Skip to contentSkip to conversation

ppc: improve some memory ordering issues

PATCH Applied Nicholas Piggin 3 participants 9 messages May 19, 2022 – May 23, 2022

lore
Share & tools
lore
#0 lore raw

cover ppc: improve some memory ordering issues

Since RFC[*], this fixes a compile issue noticed by Richard, and has survived some basic stressing with mttcg.

Thanks,
Nick

[*] https://lists.nongnu.org/archive/html/qemu-ppc/2022-05/msg00046.html

Nicholas Piggin (4):
target/ppc: Fix eieio memory ordering semantics tcg/ppc: ST_ST memory ordering is not provided with eieio tcg/ppc: Optimize memory ordering generation with lwsync target/ppc: Implement lwsync with weaker memory ordering

 target/ppc/cpu.h         |  4 +++-
 target/ppc/cpu_init.c    | 13 +++++++------
 target/ppc/machine.c     |  3 ++-
 target/ppc/translate.c   | 35 +++++++++++++++++++++++++++++++++--
 tcg/ppc/tcg-target.c.inc | 11 ++++++-----
 5 files changed, 51 insertions(+), 15 deletions(-)
Signature · 1 line
2.35.1
#3 lore raw

PATCH 2/4 tcg/ppc: ST_ST memory ordering is not provided with eieio

eieio does not provide ordering between stores to CI memory and stores to cacheable memory so it can't be used as a general ST_ST barrier.

---
 tcg/ppc/tcg-target.c.inc | 2 --
 1 file changed, 2 deletions(-)
tcg/ppc/tcg-target.c.inc +0 -2
index cfcd121f9c..3ff845d063 100644
@@ -1836,8 +1836,6 @@ static void tcg_out_mb(TCGContext *s, TCGArg a0)
     a0 &= TCG_MO_ALL;
     if (a0 == TCG_MO_LD_LD) {
         insn = LWSYNC;
-    } else if (a0 == TCG_MO_ST_ST) {
-        insn = EIEIO;
     }
     tcg_out32(s, insn);
 }
Signature · 1 line
2.35.1
#1 lore raw

PATCH 1/4 target/ppc: Fix eieio memory ordering semantics

In reply to Nicholas Piggin

The generated eieio memory ordering semantics do not match the instruction definition in the architecture. Add a big comment to explain this strange instruction and correct the memory ordering behaviour.

Signed-off: Nicholas Piggin <[email protected]>

---
 target/ppc/translate.c | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)
target/ppc/translate.c +26 -1
index fa34f81c30..eb42f7e459 100644
@@ -3513,7 +3513,32 @@ static void gen_stswx(DisasContext *ctx)
 /* eieio */
 static void gen_eieio(DisasContext *ctx)
 {
-    TCGBar bar = TCG_MO_LD_ST;
+    TCGBar bar = TCG_MO_ALL;
+
+    /*
+     * eieio has complex semanitcs. It provides memory ordering between
+     * operations in the set:
+     * - loads from CI memory.
+     * - stores to CI memory.
+     * - stores to WT memory.
+     *
+     * It separately also orders memory for operations in the set:
+     * - stores to cacheble memory.
+     *
+     * It also serializes instructions:
+     * - dcbt and dcbst.
+     *
+     * It separately serializes:
+     * - tlbie and tlbsync.
+     *
+     * And separately serializes:
+     * - slbieg, slbiag, and slbsync.
+     *
+     * The end result is that CI memory ordering requires TCG_MO_ALL
+     * and it is not possible to special-case more relaxed ordering for
+     * cacheable accesses. TCG_BAR_SC is required to provide this
+     * serialization.
+     */
 
     /*
      * POWER9 has a eieio instruction variant using bit 6 as a hint to
Signature · 1 line
2.35.1
#4 lore raw

PATCH 3/4 tcg/ppc: Optimize memory ordering generation with lwsync

In reply to Nicholas Piggin

lwsync orders more than just LD_LD, importantly it matches x86 and s390 default memory ordering.

---
 tcg/ppc/tcg-target.c.inc | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tcg/ppc/tcg-target.c.inc +6 -3
index 3ff845d063..c0a5bca34f 100644
@@ -1832,11 +1832,14 @@ static void tcg_out_brcond2 (TCGContext *s, const TCGArg *args,
 
 static void tcg_out_mb(TCGContext *s, TCGArg a0)
 {
-    uint32_t insn = HWSYNC;
-    a0 &= TCG_MO_ALL;
-    if (a0 == TCG_MO_LD_LD) {
+    uint32_t insn;
+
+    if (a0 & TCG_MO_ST_LD) {
+        insn = HWSYNC;
+    } else {
         insn = LWSYNC;
     }
+
     tcg_out32(s, insn);
 }
 
Signature · 1 line
2.35.1
#6 lore raw

PATCH 4/4 target/ppc: Implement lwsync with weaker memory ordering

In reply to Nicholas Piggin

This allows an x86 host to no-op lwsyncs, and ppc host can use lwsync rather than sync.

---
 target/ppc/cpu.h       |  4 +++-
 target/ppc/cpu_init.c  | 13 +++++++------
 target/ppc/machine.c   |  3 ++-
 target/ppc/translate.c |  8 +++++++-
 4 files changed, 19 insertions(+), 9 deletions(-)
target/ppc/cpu.h +3 -1
index 48596cfb25..b9b2536394 100644
@@ -2271,6 +2271,8 @@ enum {
     PPC2_ISA300        = 0x0000000000080000ULL,
     /* POWER ISA 3.1                                                         */
     PPC2_ISA310        = 0x0000000000100000ULL,
+    /*   lwsync instruction                                                  */
+    PPC2_MEM_LWSYNC    = 0x0000000000200000ULL,
 
 #define PPC_TCG_INSNS2 (PPC2_BOOKE206 | PPC2_VSX | PPC2_PRCNTL | PPC2_DBRX | \
                         PPC2_ISA205 | PPC2_VSX207 | PPC2_PERM_ISA206 | \
@@ -2279,7 +2281,7 @@ enum {
                         PPC2_BCTAR_ISA207 | PPC2_LSQ_ISA207 | \
                         PPC2_ALTIVEC_207 | PPC2_ISA207S | PPC2_DFP | \
                         PPC2_FP_CVT_S64 | PPC2_TM | PPC2_PM_ISA206 | \
-                        PPC2_ISA300 | PPC2_ISA310)
+                        PPC2_ISA300 | PPC2_ISA310 | PPC2_MEM_LWSYNC)
 };
 
 /*****************************************************************************/
target/ppc/cpu_init.c +7 -6
index 527ad40fcb..0f891afa04 100644
@@ -5769,7 +5769,7 @@ POWERPC_FAMILY(970)(ObjectClass *oc, void *data)
                        PPC_MEM_TLBIE | PPC_MEM_TLBSYNC |
                        PPC_64B | PPC_ALTIVEC |
                        PPC_SEGMENT_64B | PPC_SLBI;
-    pcc->insns_flags2 = PPC2_FP_CVT_S64;
+    pcc->insns_flags2 = PPC2_FP_CVT_S64 | PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_VR) |
                     (1ull << MSR_POW) |
@@ -5846,7 +5846,7 @@ POWERPC_FAMILY(POWER5P)(ObjectClass *oc, void *data)
                        PPC_64B |
                        PPC_POPCNTB |
                        PPC_SEGMENT_64B | PPC_SLBI;
-    pcc->insns_flags2 = PPC2_FP_CVT_S64;
+    pcc->insns_flags2 = PPC2_FP_CVT_S64 | PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_VR) |
                     (1ull << MSR_POW) |
@@ -5985,7 +5985,7 @@ POWERPC_FAMILY(POWER7)(ObjectClass *oc, void *data)
                         PPC2_PERM_ISA206 | PPC2_DIVE_ISA206 |
                         PPC2_ATOMIC_ISA206 | PPC2_FP_CVT_ISA206 |
                         PPC2_FP_TST_ISA206 | PPC2_FP_CVT_S64 |
-                        PPC2_PM_ISA206;
+                        PPC2_PM_ISA206 | PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_VR) |
                     (1ull << MSR_VSX) |
@@ -6159,7 +6159,7 @@ POWERPC_FAMILY(POWER8)(ObjectClass *oc, void *data)
                         PPC2_FP_TST_ISA206 | PPC2_BCTAR_ISA207 |
                         PPC2_LSQ_ISA207 | PPC2_ALTIVEC_207 |
                         PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 |
-                        PPC2_TM | PPC2_PM_ISA206;
+                        PPC2_TM | PPC2_PM_ISA206 | PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_HV) |
                     (1ull << MSR_TM) |
@@ -6379,7 +6379,7 @@ POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data)
                         PPC2_FP_TST_ISA206 | PPC2_BCTAR_ISA207 |
                         PPC2_LSQ_ISA207 | PPC2_ALTIVEC_207 |
                         PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 |
-                        PPC2_TM | PPC2_ISA300 | PPC2_PRCNTL;
+                        PPC2_TM | PPC2_ISA300 | PPC2_PRCNTL | PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_HV) |
                     (1ull << MSR_TM) |
@@ -6596,7 +6596,8 @@ POWERPC_FAMILY(POWER10)(ObjectClass *oc, void *data)
                         PPC2_FP_TST_ISA206 | PPC2_BCTAR_ISA207 |
                         PPC2_LSQ_ISA207 | PPC2_ALTIVEC_207 |
                         PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 |
-                        PPC2_TM | PPC2_ISA300 | PPC2_PRCNTL | PPC2_ISA310;
+                        PPC2_TM | PPC2_ISA300 | PPC2_PRCNTL | PPC2_ISA310 |
+                        PPC2_MEM_LWSYNC;
     pcc->msr_mask = (1ull << MSR_SF) |
                     (1ull << MSR_HV) |
                     (1ull << MSR_TM) |
target/ppc/machine.c +2 -1
index 7104a5c67e..a7d9036c09 100644
@@ -157,7 +157,8 @@ static int cpu_pre_save(void *opaque)
         | PPC2_ATOMIC_ISA206 | PPC2_FP_CVT_ISA206
         | PPC2_FP_TST_ISA206 | PPC2_BCTAR_ISA207
         | PPC2_LSQ_ISA207 | PPC2_ALTIVEC_207
-        | PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 | PPC2_TM;
+        | PPC2_ISA205 | PPC2_ISA207S | PPC2_FP_CVT_S64 | PPC2_TM
+        | PPC2_MEM_LWSYNC;
 
     env->spr[SPR_LR] = env->lr;
     env->spr[SPR_CTR] = env->ctr;
target/ppc/translate.c +7 -1
index eb42f7e459..1d6daa4608 100644
@@ -4041,8 +4041,13 @@ static void gen_stqcx_(DisasContext *ctx)
 /* sync */
 static void gen_sync(DisasContext *ctx)
 {
+    TCGBar bar = TCG_MO_ALL;
     uint32_t l = (ctx->opcode >> 21) & 3;
 
+    if ((l == 1) && (ctx->insns_flags2 & PPC2_MEM_LWSYNC)) {
+        bar = TCG_MO_LD_LD | TCG_MO_LD_ST | TCG_MO_ST_ST;
+    }
+
     /*
      * We may need to check for a pending TLB flush.
      *
@@ -4054,7 +4059,8 @@ static void gen_sync(DisasContext *ctx)
     if (((l == 2) || !(ctx->insns_flags & PPC_64B)) && !ctx->pr) {
         gen_check_tlb_flush(ctx, true);
     }
-    tcg_gen_mb(TCG_MO_ALL | TCG_BAR_SC);
+
+    tcg_gen_mb(bar | TCG_BAR_SC);
 }
 
 /* wait */
Signature · 1 line
2.35.1
Reviewed-by on v1 1/4
#2 lore raw

Re: [PATCH 1/4] target/ppc: Fix eieio memory ordering semantics

In reply to Nicholas Piggin
Quoted 9 lines · Nicholas Piggin

The generated eieio memory ordering semantics do not match the instruction definition in the architecture. Add a big comment to explain this strange instruction and correct the memory ordering behaviour.

Signed-off: Nicholas Piggin<[email protected]>

---
  target/ppc/translate.c | 27 ++++++++++++++++++++++++++-
  1 file changed, 26 insertions(+), 1 deletion(-)

r~

Reviewed-by on v1 4/4
#7 lore raw

Re: [PATCH 4/4] target/ppc: Implement lwsync with weaker memory ordering

In reply to Nicholas Piggin
Quoted 10 lines · Nicholas Piggin

This allows an x86 host to no-op lwsyncs, and ppc host can use lwsync rather than sync.

---
  target/ppc/cpu.h       |  4 +++-
  target/ppc/cpu_init.c  | 13 +++++++------
  target/ppc/machine.c   |  3 ++-
  target/ppc/translate.c |  8 +++++++-
  4 files changed, 19 insertions(+), 9 deletions(-)

to the translate part, and I'll trust you on the set of cpus adjusted.

r~

#8 lore raw
In reply to Nicholas Piggin

Queued in gitlab.com/danielhb/qemu/tree/ppc-next. Thanks,

Daniel

Quoted 21 lines · Nicholas Piggin

Since RFC[*], this fixes a compile issue noticed by Richard, and has survived some basic stressing with mttcg.

Thanks,
Nick

[*] https://lists.nongnu.org/archive/html/qemu-ppc/2022-05/msg00046.html

Nicholas Piggin (4):
target/ppc: Fix eieio memory ordering semantics tcg/ppc: ST_ST memory ordering is not provided with eieio tcg/ppc: Optimize memory ordering generation with lwsync target/ppc: Implement lwsync with weaker memory ordering

  target/ppc/cpu.h         |  4 +++-
  target/ppc/cpu_init.c    | 13 +++++++------
  target/ppc/machine.c     |  3 ++-
  target/ppc/translate.c   | 35 +++++++++++++++++++++++++++++++++--
  tcg/ppc/tcg-target.c.inc | 11 ++++++-----
  5 files changed, 51 insertions(+), 15 deletions(-)