target/ppc: TCG SMT support for spapr
cover target/ppc: TCG SMT support for spapr
Previous RFC here
https://lists.gnu.org/archive/html/qemu-ppc/2023-05/msg00453.html
This series drops patch 1 from the previous, which is more of a standalone bugfix.
Also accounted for Cedric's comments, except a nicer way to set cpu_index vs PIR/TIR SPRs, which is not quite trivial.
This limits support for SMT to POWER8 and newer. It is also incompatible with nested-HV so that is checked for too.
Iterating CPUs to find siblings for now I kept because similar loops exist in a few places, and it is not conceptually difficult for SMT, just fiddly code to improve. For now it should not be much performane concern.
I removed hypervisor msgsnd support from patch 3, which is not required for spapr and added significantly to the patch.
For now nobody has objected to the way shared SPR access is handled (serialised with TCG atomics support) so we'll keep going with it.
Thanks,
Nick
Nicholas Piggin (4):
target/ppc: Add initial flags and helpers for SMT support target/ppc: Add support for SMT CTRL register target/ppc: Add msgsndp and DPDES SMT support
spapr: Allow up to 8 threads SMT on POWER8 and newer
hw/ppc/ppc.c | 6 ++++ hw/ppc/spapr.c | 16 +++++++--- hw/ppc/spapr_caps.c | 14 ++++++++ hw/ppc/spapr_cpu_core.c | 7 ++-- include/hw/ppc/ppc.h | 1 + target/ppc/cpu.h | 9 ++++++ target/ppc/cpu_init.c | 5 +++ target/ppc/excp_helper.c | 30 ++++++++++++++--- target/ppc/gdbstub.c | 2 +- target/ppc/helper.h | 2 ++ target/ppc/misc_helper.c | 69 ++++++++++++++++++++++++++++++++++++---- target/ppc/translate.c | 46 ++++++++++++++++++++++++++- 12 files changed, 188 insertions(+), 19 deletions(-)
Signature · 1 line
2.40.1
PATCH 1/4 target/ppc: Add initial flags and helpers for SMT support
SMT TCG emulation needs to be able to iterate over siblings in a core, and needs to serialise core access to shared SPRs and state.
--- target/ppc/cpu.h | 9 +++++++++ target/ppc/cpu_init.c | 5 +++++ target/ppc/translate.c | 20 ++++++++++++++++++++ 3 files changed, 34 insertions(+)
target/ppc/cpu.h +9 -0
@@ -672,6 +672,8 @@ enum {POWERPC_FLAG_TM =0x00100000 ,/* Has SCV (ISA 3.00) */ POWERPC_FLAG_SCV =0x00200000 , +/* Has >1 thread per core */ + POWERPC_FLAG_SMT = 0x00400000, };/*
@@ -1266,6 +1268,13 @@ struct CPUArchState {uint64_t pmu_base_time; }; +#define _CORE_ID (cs) \ + (POWERPC_CPU(cs)->env.spr_cb[SPR_PIR].default_value & ~(cs->nr_threads - 1)) + +#define THREAD_SIBLING_FOREACH (cs, cs_sibling) \ + CPU_FOREACH(cs_sibling) \ + if (_CORE_ID(cs) == _CORE_ID(cs_sibling)) +#define SET_FIT_PERIOD (a_, b_, c_, d_) \ do { \ env->fit_period[0] = (a_); \
target/ppc/cpu_init.c +5 -0
@@ -6754,6 +6754,7 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp) {CPUState *cs =CPU (dev);PowerPCCPU *cpu =POWERPC_CPU (dev); +CPUPPCState *env = &cpu->env;PowerPCCPUClass *pcc =POWERPC_CPU_GET_CLASS (cpu);Error *local_err =NULL ;
@@ -6785,6 +6786,10 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp) pcc->parent_realize (dev, errp); +if (env_cpu (env)->nr_threads >1 ) { + env->flags |=POWERPC_FLAG_SMT ; + } +return ; unrealize:
target/ppc/translate.c +20 -0
@@ -227,6 +227,26 @@ struct opc_handler_t {void (*handler)(DisasContext *ctx); }; +static inline bool gen_serialize (DisasContext *ctx) +{ +if (tb_cflags (ctx->base.tb) &CF_PARALLEL ) { +/* Restart with exclusive lock. */ +gen_helper_exit_atomic (cpu_env); + ctx->base.is_jmp =DISAS_NORETURN ; +return false; + } +return true; +} + +static inline bool gen_serialize_core (DisasContext *ctx) +{ +if (ctx->flags &POWERPC_FLAG_SMT ) { +return gen_serialize (ctx); + } + +return true; +} +/* SPR load/store helpers */ static inline void gen_load_spr (TCGv t,int reg) {
Signature · 1 line
2.40.1
PATCH 2/4 target/ppc: Add support for SMT CTRL register
In reply to Nicholas PigginA relatively simple case to begin with, CTRL is a SMT shared register where reads and writes need to synchronise against state changes by other threads in the core.
Atomic serialisation operations are used to achieve this.
--- target/ppc/helper.h | 2 ++ target/ppc/misc_helper.c | 25 +++++++++++++++++++++++++ target/ppc/translate.c | 18 +++++++++++++++++- 3 files changed, 44 insertions(+), 1 deletion(-)
target/ppc/helper.h +2 -0
@@ -704,6 +704,8 @@ DEF_HELPER_3(store_dcr, void, env, tl, tl) DEF_HELPER_2(load_dump_spr, void, env, i32) DEF_HELPER_2(store_dump_spr, void, env, i32) +DEF_HELPER_3(spr_write_CTRL, void, env, i32, tl) + DEF_HELPER_4(fscr_facility_check, void, env, i32, i32, i32) DEF_HELPER_4(msr_facility_check, void, env, i32, i32, i32)DEF_HELPER_FLAGS_1 (load_tbl,TCG_CALL_NO_RWG , tl, env)
target/ppc/misc_helper.c +25 -0
@@ -43,6 +43,31 @@ void helper_store_dump_spr(CPUPPCState *env, uint32_t sprn) env->spr[sprn]); } +void helper_spr_write_CTRL (CPUPPCState *env,uint32_t sprn, +target_ulong val) +{ +CPUState *cs =env_cpu (env); +CPUState *ccs; +uint32_t run = val &1 ; +uint32_t ts, ts_mask; + +assert (sprn ==SPR_CTRL ); + + env->spr[sprn] &= ~1U ; + env->spr[sprn] |= run; + + ts_mask = ~(1U << (8 + env->spr[SPR_TIR ])); + ts = run << (8 + env->spr[SPR_TIR ]); + + THREAD_SIBLING_FOREACH(cs, ccs) { +CPUPPCState *cenv = &POWERPC_CPU (ccs)->env; + + cenv->spr[sprn] &= ts_mask; + cenv->spr[sprn] |= ts; + } +} + +#ifdef TARGET_PPC64 static void raise_hv_fu_exception(CPUPPCState *env, uint32_t bit, const char *caller, uint32_t cause,
target/ppc/translate.c +17 -1
@@ -443,7 +443,7 @@ void spr_write_generic32(DisasContext *ctx, int sprn, int gprn)#endif } -void spr_write_CTRL (DisasContext *ctx,int sprn,int gprn) +static void spr_write_CTRL_ST (DisasContext *ctx,int sprn,int gprn) {/* This does not implement >1 thread */ TCGv t0 =tcg_temp_new ();
@@ -452,6 +452,22 @@ void spr_write_CTRL(DisasContext *ctx, int sprn, int gprn)tcg_gen_shli_tl (t1, t0,8 );/* Duplicate the bit in TS */ tcg_gen_or_tl (t1, t1, t0);gen_store_spr (sprn, t1); +} + +void spr_write_CTRL (DisasContext *ctx,int sprn,int gprn) +{ +if (!(ctx->flags &POWERPC_FLAG_SMT )) { +spr_write_CTRL_ST (ctx, sprn, gprn); + goto out; + } + +if (!gen_serialize (ctx)) { +return ; + } + +gen_helper_spr_write_CTRL (cpu_env,tcg_constant_i32 (sprn), + cpu_gpr[gprn]); +out:spr_store_dump_spr (sprn);/*
Signature · 1 line
2.40.1
Re: [PATCH 2/4] target/ppc: Add support for SMT CTRL register
Hello Nick,
Quoted 12 lines
target/ppc/translate.c +1
-1@@ -443,7 +443,7 @@ void spr_write_generic32(DisasContext *ctx, int sprn, int gprn)#endif } -void spr_write_CTRL (DisasContext *ctx,int sprn,int gprn) +static void spr_write_CTRL_ST (DisasContext *ctx,int sprn,int gprn) {/* This does not implement >1 thread */ TCGv t0 =tcg_temp_new ();
target/ppc/translate.c void spr_write_CTRL(DisasContext *ctx, int sprn, int gprn) · line 454
Quoted 3 lines
target/ppc/translate.c +0
-0@@ -452,6 +452,22 @@ void spr_write_CTRL(DisasContext *ctx, int sprn, int gprn)tcg_gen_shli_tl (t1, t0,8 );/* Duplicate the bit in TS */ tcg_gen_or_tl (t1, t1, t0);gen_store_spr (sprn, t1);
The hunk above doesn't apply cleanly. Am I missing some patch you would have sent previously ?
Thanks,
C.
Quoted 19 lines
target/ppc/translate.c +16
-0+} + +void spr_write_CTRL (DisasContext *ctx,int sprn,int gprn) +{ +if (!(ctx->flags &POWERPC_FLAG_SMT )) { +spr_write_CTRL_ST (ctx, sprn, gprn); + goto out; + } + +if (!gen_serialize (ctx)) { +return ; + } + +gen_helper_spr_write_CTRL (cpu_env,tcg_constant_i32 (sprn), + cpu_gpr[gprn]); +out:spr_store_dump_spr (sprn);/*
PATCH 3/4 target/ppc: Add msgsndp and DPDES SMT support
In reply to Nicholas PigginDoorbells in SMT need to coordinate msgsndp/msgclrp and DPDES access from multiple threads that affect the same state.
--- hw/ppc/ppc.c | 6 ++++++ include/hw/ppc/ppc.h | 1 + target/ppc/excp_helper.c | 30 ++++++++++++++++++++++----- target/ppc/gdbstub.c | 2 +- target/ppc/misc_helper.c | 44 ++++++++++++++++++++++++++++++++++------ target/ppc/translate.c | 8 ++++++++ 6 files changed, 79 insertions(+), 12 deletions(-)
hw/ppc/ppc.c +6 -0
@@ -1434,6 +1434,12 @@ int ppc_cpu_pir(PowerPCCPU *cpu)return env->spr_cb[SPR_PIR ].default_value; } +int ppc_cpu_tir (PowerPCCPU *cpu) +{ +CPUPPCState *env = &cpu->env; +return env->spr_cb[SPR_TIR ].default_value; +} +PowerPCCPU *ppc_get_vcpu_by_pir (int pir) {CPUState *cs;
include/hw/ppc/ppc.h +1 -0
@@ -6,6 +6,7 @@void ppc_set_irq (PowerPCCPU *cpu,int n_IRQ,int level);PowerPCCPU *ppc_get_vcpu_by_pir (int pir);int ppc_cpu_pir (PowerPCCPU *cpu); +int ppc_cpu_tir (PowerPCCPU *cpu);/* PowerPC hardware exceptions management helpers */ typedef void (*clk_setup_cb )(void *opaque,uint32_t freq);
target/ppc/excp_helper.c +25 -5
@@ -3154,22 +3154,42 @@ void helper_book3s_msgclrp(CPUPPCState *env, target_ulong rb) }/* -* sends a message to other threads that are on the same +* sends a message to another thread on the same * multi-threaded processor */ void helper_book3s_msgsndp (CPUPPCState *env,target_ulong rb) { -int pir = env->spr_cb[SPR_PIR ].default_value; +CPUState *cs =env_cpu (env); +PowerPCCPU *cpu =POWERPC_CPU (cs); +CPUState *ccs; +uint32_t nr_threads = cs->nr_threads; +int ttir = rb &PPC_BITMASK (57 ,63 );helper_hfscr_facility_check (env,HFSCR_MSGP ,"msgsndp" ,HFSCR_IC_MSGP ); -if (!dbell_type_server (rb)) { +if (!dbell_type_server (rb) || ttir >= nr_threads) { +return ; + } + +if (nr_threads ==1 ) { +ppc_set_irq (cpu,PPC_INTERRUPT_DOORBELL ,1 );return ; } -/* TODO: TCG supports only one thread */ +/* WHy does iothread need to be locked for walking CPU list? */ +qemu_mutex_lock_iothread (); + THREAD_SIBLING_FOREACH(cs, ccs) { +PowerPCCPU *ccpu =POWERPC_CPU (ccs); +uint32_t thread_id =ppc_cpu_tir (ccpu); + +if (ttir == thread_id) { +ppc_set_irq (ccpu,PPC_INTERRUPT_DOORBELL ,1 ); +qemu_mutex_unlock_iothread (); +return ; + } + } -book3s_msgsnd_common (pir,PPC_INTERRUPT_DOORBELL ); +assert (0 ); }#endif /* TARGET_PPC64 */
target/ppc/gdbstub.c +1 -1
@@ -117,7 +117,7 @@ void ppc_maybe_bswap_register(CPUPPCState *env, uint8_t *mem_buf, int len)* regs and PC, MSR, CR, and so forth. We hack round this by giving * the FP regs zero size when talking to a newer gdb. */ - +/* XXX: read/write dpdes correctly */ int ppc_cpu_gdb_read_register (CPUState *cs,GByteArray *buf,int n) {PowerPCCPU *cpu =POWERPC_CPU (cs);
target/ppc/misc_helper.c +38 -6
69 lines not on this page · raw message
target/ppc/translate.c +8 -0
@@ -820,11 +820,19 @@ void spr_write_pcr(DisasContext *ctx, int sprn, int gprn)/* DPDES */ void spr_read_dpdes (DisasContext *ctx,int gprn,int sprn) { +if (!gen_serialize_core (ctx)) { +return ; + } +gen_helper_load_dpdes (cpu_gpr[gprn], cpu_env); }void spr_write_dpdes (DisasContext *ctx,int sprn,int gprn) { +if (!gen_serialize_core (ctx)) { +return ; + } +gen_helper_store_dpdes (cpu_env, cpu_gpr[gprn]); }#endif
Signature · 1 line
2.40.1
PATCH 4/4 spapr: Allow up to 8 threads SMT on POWER8 and newer
In reply to Nicholas PigginPPC TCG now supports multi-threaded CPU configuration for non-hypervisor state. This requires PIR and TIR be set, because that's how sibling thread matching is done.
spapr's nested-HV capability does not currently coexist with SMT. This is quite analogous to LPAR-per-core mode on real hardware which also does not support KVM.
--- hw/ppc/spapr.c | 16 ++++++++++++---- hw/ppc/spapr_caps.c | 14 ++++++++++++++ hw/ppc/spapr_cpu_core.c | 7 +++++-- 3 files changed, 31 insertions(+), 6 deletions(-)
hw/ppc/spapr.c +12 -4
@@ -2524,10 +2524,18 @@ static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp)int ret;unsigned int smp_threads = ms->smp.threads; -if (!kvm_enabled () && (smp_threads >1 )) { -error_setg (errp,"TCG cannot support more than 1 thread/core " -"on a pseries machine" ); -return ; +if (!kvm_enabled ()) { +if (!ppc_type_check_compat (ms->cpu_type,CPU_POWERPC_LOGICAL_2_07 ,0 , + spapr->max_compat_pvr)) { +error_setg (errp,"TCG only supports SMT on POWER8 or newer CPUs" ); +return ; + } + +if (smp_threads >8 ) { +error_setg (errp,"TCG cannot support more than 8 threads/core " +"on a pseries machine" ); +return ; + } }if (!is_power_of_2 (smp_threads)) { error_setg(errp, "Cannot support %d threads/core on a pseries "
hw/ppc/spapr_caps.c +14 -0
@@ -473,6 +473,20 @@ static void cap_nested_kvm_hv_apply(SpaprMachineState *spapr,error_append_hint (errp,"Try appending -machine cap-nested-hv=off\n" ); } + }else { +MachineState *ms =MACHINE (spapr); +unsigned int smp_threads = ms->smp.threads; + +/* +* Nested-HV vCPU env state to L2, so SMT-shared SPR updates, for +* example, do not necessarily update the correct SPR value on sibling +* threads that are in a different guest/host context. +*/ +if (smp_threads >1 ) { +error_setg (errp,"TCG does not support nested-HV with SMT" ); +error_append_hint (errp,"Try appending -machine cap-nested-hv=off " +"or use threads=1 with -smp\n" ); + } } }
hw/ppc/spapr_cpu_core.c +5 -2
@@ -255,7 +255,7 @@ static void spapr_cpu_core_unrealize(DeviceState *dev) }static bool spapr_realize_vcpu (PowerPCCPU *cpu,SpaprMachineState *spapr, -SpaprCpuCore *sc,Error **errp) +SpaprCpuCore *sc,int thread_index,Error **errp) {CPUPPCState *env = &cpu->env;CPUState *cs =CPU (cpu);
@@ -267,6 +267,9 @@ static bool spapr_realize_vcpu(PowerPCCPU *cpu, SpaprMachineState *spapr,cpu_ppc_set_vhyp (cpu,PPC_VIRTUAL_HYPERVISOR (spapr));kvmppc_set_papr (cpu); + env->spr_cb[SPR_PIR ].default_value = cs->cpu_index; + env->spr_cb[SPR_TIR ].default_value = thread_index; +/* Set time-base frequency to 512 MHz. vhyp must be set first. */ cpu_ppc_tb_init (env,SPAPR_TIMEBASE_FREQ );
@@ -337,7 +340,7 @@ static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)for (i =0 ; i < cc->nr_threads; i++) { sc->threads[i] =spapr_create_vcpu (sc, i, errp);if (!sc->threads[i] || - !spapr_realize_vcpu (sc->threads[i], spapr, sc, errp)) { + !spapr_realize_vcpu (sc->threads[i], spapr, sc, i, errp)) {spapr_cpu_core_unrealize (dev);return ; }
Signature · 1 line
2.40.1
Re: [PATCH 4/4] spapr: Allow up to 8 threads SMT on POWER8 and newer
Quoted 20 lines · Nicholas Piggin
PPC TCG now supports multi-threaded CPU configuration for non-hypervisor state. This requires PIR and TIR be set, because that's how sibling thread matching is done.
spapr's nested-HV capability does not currently coexist with SMT. This is quite analogous to LPAR-per-core mode on real hardware which also does not support KVM.
Signed-off-by Nicholas Piggin <[email protected]>--- hw/ppc/spapr.c | 16 ++++++++++++---- hw/ppc/spapr_caps.c | 14 ++++++++++++++ hw/ppc/spapr_cpu_core.c | 7 +++++-- 3 files changed, 31 insertions(+), 6 deletions(-)hw/ppc/spapr.c +0
-0
hw/ppc/spapr.c static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp) · line 2527
Quoted 8 lines
hw/ppc/spapr.c +1
-4@@ -2524,10 +2524,18 @@ static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp)int ret;unsigned int smp_threads = ms->smp.threads; -if (!kvm_enabled () && (smp_threads >1 )) { -error_setg (errp,"TCG cannot support more than 1 thread/core " -"on a pseries machine" ); -return ; +if (!kvm_enabled ()) {
Can we make it a check for tcg_enabled() which would be more appropriate or as Cedric suggested, may be include this one along with your series:
https://lore.kernel.org/qemu-devel/[email protected]/
regards,
Harsh
Quoted 71 lines
hw/ppc/spapr.c +11
-0+if (!ppc_type_check_compat (ms->cpu_type,CPU_POWERPC_LOGICAL_2_07 ,0 , + spapr->max_compat_pvr)) { +error_setg (errp,"TCG only supports SMT on POWER8 or newer CPUs" ); +return ; + } + +if (smp_threads >8 ) { +error_setg (errp,"TCG cannot support more than 8 threads/core " +"on a pseries machine" ); +return ; + } }if (!is_power_of_2 (smp_threads)) { error_setg(errp, "Cannot support %d threads/core on a pseries "hw/ppc/spapr_caps.c +14
-0@@ -473,6 +473,20 @@ static void cap_nested_kvm_hv_apply(SpaprMachineState *spapr,error_append_hint (errp,"Try appending -machine cap-nested-hv=off\n" ); } + }else { +MachineState *ms =MACHINE (spapr); +unsigned int smp_threads = ms->smp.threads; + +/* +* Nested-HV vCPU env state to L2, so SMT-shared SPR updates, for +* example, do not necessarily update the correct SPR value on sibling +* threads that are in a different guest/host context. +*/ +if (smp_threads >1 ) { +error_setg (errp,"TCG does not support nested-HV with SMT" ); +error_append_hint (errp,"Try appending -machine cap-nested-hv=off " +"or use threads=1 with -smp\n" ); + } } }hw/ppc/spapr_cpu_core.c +5
-2@@ -255,7 +255,7 @@ static void spapr_cpu_core_unrealize(DeviceState *dev) }static bool spapr_realize_vcpu (PowerPCCPU *cpu,SpaprMachineState *spapr, -SpaprCpuCore *sc,Error **errp) +SpaprCpuCore *sc,int thread_index,Error **errp) {CPUPPCState *env = &cpu->env;CPUState *cs =CPU (cpu);@@ -267,6 +267,9 @@ static bool spapr_realize_vcpu(PowerPCCPU *cpu, SpaprMachineState *spapr,cpu_ppc_set_vhyp (cpu,PPC_VIRTUAL_HYPERVISOR (spapr));kvmppc_set_papr (cpu); + env->spr_cb[SPR_PIR ].default_value = cs->cpu_index; + env->spr_cb[SPR_TIR ].default_value = thread_index; +/* Set time-base frequency to 512 MHz. vhyp must be set first. */ cpu_ppc_tb_init (env,SPAPR_TIMEBASE_FREQ );@@ -337,7 +340,7 @@ static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)for (i =0 ; i < cc->nr_threads; i++) { sc->threads[i] =spapr_create_vcpu (sc, i, errp);if (!sc->threads[i] || - !spapr_realize_vcpu (sc->threads[i], spapr, sc, errp)) { + !spapr_realize_vcpu (sc->threads[i], spapr, sc, i, errp)) {spapr_cpu_core_unrealize (dev);return ; }
Re: [PATCH 4/4] spapr: Allow up to 8 threads SMT on POWER8 and newer
Quoted 35 lines · Harsh Prateek Bora
Quoted 20 lines · Nicholas Piggin
PPC TCG now supports multi-threaded CPU configuration for non-hypervisor state. This requires PIR and TIR be set, because that's how sibling thread matching is done.
spapr's nested-HV capability does not currently coexist with SMT. This is quite analogous to LPAR-per-core mode on real hardware which also does not support KVM.
Signed-off-by Nicholas Piggin <[email protected]>--- hw/ppc/spapr.c | 16 ++++++++++++---- hw/ppc/spapr_caps.c | 14 ++++++++++++++ hw/ppc/spapr_cpu_core.c | 7 +++++-- 3 files changed, 31 insertions(+), 6 deletions(-)hw/ppc/spapr.c +0
-0hw/ppc/spapr.c static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp) · line 2527
Quoted 8 lines
hw/ppc/spapr.c +1
-4@@ -2524,10 +2524,18 @@ static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp)int ret;unsigned int smp_threads = ms->smp.threads; -if (!kvm_enabled () && (smp_threads >1 )) { -error_setg (errp,"TCG cannot support more than 1 thread/core " -"on a pseries machine" ); -return ; +if (!kvm_enabled ()) {Can we make it a check for
tcg_enabled()which would be more appropriate or as Cedric suggested, may be include this one along with your series:
Good point, I'll keep it in mind.
Thanks,
Nick
Quoted 18 lines · Nicholas Piggin
Previous RFC here
https://lists.gnu.org/archive/html/qemu-ppc/2023-05/msg00453.html
This series drops patch 1 from the previous, which is more of a standalone bugfix.
Also accounted for Cedric's comments, except a nicer way to set
cpu_indexvs PIR/TIR SPRs, which is not quite trivial.This limits support for SMT to POWER8 and newer. It is also incompatible with nested-HV so that is checked for too.
Iterating CPUs to find siblings for now I kept because similar loops exist in a few places, and it is not conceptually difficult for SMT, just fiddly code to improve. For now it should not be much performane concern.
Quoted 6 lines
I removed hypervisor msgsnd support from patch 3, which is not required for spapr and added significantly to the patch.
For now nobody has objected to the way shared SPR access is handled (serialised with TCG atomics support) so we'll keep going with it.
Thanks,
C.
Quoted 19 lines · Cédric Le Goater
Quoted 18 lines · Nicholas Piggin
Previous RFC here
https://lists.gnu.org/archive/html/qemu-ppc/2023-05/msg00453.html
This series drops patch 1 from the previous, which is more of a standalone bugfix.
Also accounted for Cedric's comments, except a nicer way to set
cpu_indexvs PIR/TIR SPRs, which is not quite trivial.This limits support for SMT to POWER8 and newer. It is also incompatible with nested-HV so that is checked for too.
Iterating CPUs to find siblings for now I kept because similar loops exist in a few places, and it is not conceptually difficult for SMT, just fiddly code to improve. For now it should not be much performane concern.
Quoted 8 lines
Quoted 6 lines
I removed hypervisor msgsnd support from patch 3, which is not required for spapr and added significantly to the patch.
For now nobody has objected to the way shared SPR access is handled (serialised with TCG atomics support) so we'll keep going with it.
Cc ing more people for possible feedback.
Not much feedback so I'll plan to go with this.
A more performant implementation might try to synchronize threads at the register level rather than serialize everything, but SMT shared registers are not too performance critical so this should do for now.
Thanks,
Nick
Quoted 27 lines · Nicholas Piggin
Quoted 26 lines · Cédric Le Goater
Quoted 24 lines · Nicholas Piggin
Previous RFC here
https://lists.gnu.org/archive/html/qemu-ppc/2023-05/msg00453.html
This series drops patch 1 from the previous, which is more of a standalone bugfix.
Also accounted for Cedric's comments, except a nicer way to set
cpu_indexvs PIR/TIR SPRs, which is not quite trivial.This limits support for SMT to POWER8 and newer. It is also incompatible with nested-HV so that is checked for too.
Iterating CPUs to find siblings for now I kept because similar loops exist in a few places, and it is not conceptually difficult for SMT, just fiddly code to improve. For now it should not be much performane concern.
I removed hypervisor msgsnd support from patch 3, which is not required for spapr and added significantly to the patch.
For now nobody has objected to the way shared SPR access is handled (serialised with TCG atomics support) so we'll keep going with it.
Quoted 8 lines
Quoted 1 line
Cc ing more people for possible feedback.Not much feedback so I'll plan to go with this.
A more performant implementation might try to synchronize threads at the register level rather than serialize everything, but SMT shared registers are not too performance critical so this should do for now.
yes. Could you please rebase this series on upstream ?
It would be good to add tests for SMT. May be we could extend :
tests/avocado/ppc_pseries.py
with a couple of extra QEMU configs adding 'threads=' (if possible) and check :
"CPU maps initialized for Y threads per core"
and
"smp: Brought up 1 node, X*Y CPUs"
?
Thanks,
C.
Quoted 30 lines · Cédric Le Goater
Quoted 29 lines · Nicholas Piggin
Quoted 27 lines · Cédric Le Goater
Quoted 18 lines · Nicholas Piggin
Previous RFC here
https://lists.gnu.org/archive/html/qemu-ppc/2023-05/msg00453.html
This series drops patch 1 from the previous, which is more of a standalone bugfix.
Also accounted for Cedric's comments, except a nicer way to set
cpu_indexvs PIR/TIR SPRs, which is not quite trivial.This limits support for SMT to POWER8 and newer. It is also incompatible with nested-HV so that is checked for too.
Iterating CPUs to find siblings for now I kept because similar loops exist in a few places, and it is not conceptually difficult for SMT, just fiddly code to improve. For now it should not be much performane concern.
Quoted 6 lines
I removed hypervisor msgsnd support from patch 3, which is not required for spapr and added significantly to the patch.
For now nobody has objected to the way shared SPR access is handled (serialised with TCG atomics support) so we'll keep going with it.
Cc ing more people for possible feedback.
Quoted 8 lines
Quoted 6 lines
Not much feedback so I'll plan to go with this.
A more performant implementation might try to synchronize threads at the register level rather than serialize everything, but SMT shared registers are not too performance critical so this should do for now.
yes. Could you please rebase this series on upstream ?
Oh yeah I should have said, I will rebase and resend.
Quoted 7 lines
It would be good to add tests for SMT. May be we could extend :
tests/avocado/ppc_pseries.pywith a couple of extra QEMU configs adding 'threads=' (if possible) and check :
Quoted 7 lines
"CPU maps initialized for Y threads per core"
and
"smp: Brought up 1 node, X*Y CPUs"
?
Yeah that could be a good idea, I'll try it.
Thanks,
Nick
Quoted 33 lines · Nicholas Piggin
Quoted 32 lines · Cédric Le Goater
Quoted 31 lines · Nicholas Piggin
Quoted 19 lines · Cédric Le Goater
Quoted 18 lines · Nicholas Piggin
Previous RFC here
https://lists.gnu.org/archive/html/qemu-ppc/2023-05/msg00453.html
This series drops patch 1 from the previous, which is more of a standalone bugfix.
Also accounted for Cedric's comments, except a nicer way to set
cpu_indexvs PIR/TIR SPRs, which is not quite trivial.This limits support for SMT to POWER8 and newer. It is also incompatible with nested-HV so that is checked for too.
Iterating CPUs to find siblings for now I kept because similar loops exist in a few places, and it is not conceptually difficult for SMT, just fiddly code to improve. For now it should not be much performane concern.
Quoted 8 lines
Quoted 6 lines
I removed hypervisor msgsnd support from patch 3, which is not required for spapr and added significantly to the patch.
For now nobody has objected to the way shared SPR access is handled (serialised with TCG atomics support) so we'll keep going with it.
Cc ing more people for possible feedback.Not much feedback so I'll plan to go with this.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
A more performant implementation might try to synchronize threads at the register level rather than serialize everything, but SMT shared registers are not too performance critical so this should do for now.
yes. Could you please rebase this series on upstream ?
Oh yeah I should have said, I will rebase and resend.
Here is a tree to check the rebase on :
https://github.com/legoater/qemu/commits/ppc-next
Quoted 9 lines
Quoted 9 lines
It would be good to add tests for SMT. May be we could extend :
tests/avocado/ppc_pseries.pywith a couple of extra QEMU configs adding 'threads=' (if possible) and check :
"CPU maps initialized for Y threads per core"
Quoted 7 lines
Quoted 5 lines
and
"smp: Brought up 1 node, X*Y CPUs"
?
Yeah that could be a good idea, I'll try it.
It can come later.
Thanks,
C.