Skip to contentSkip to conversation

ppc/pnv: add XIVE support for KVM guests

PATCH Applied Cédric Le Goater 3 participants 64 messages Nov 15, 2019 – Nov 23, 2019

v5 of 6 Applied Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v5 cover ppc/pnv: add XIVE support for KVM guests

Hello,

The QEMU PowerNV machine emulates a baremetal OpenPOWER system and acts as an hypervisor (L0). Supporting emulation of KVM to run guests (L1) requires a few more extensions, among which guest support for the XIVE interrupt controller on POWER9 processor.

The following changes extend the XIVE models with the new XiveFabric and XivePresenter interfaces to provide support for XIVE escalations and interrupt resend. This mechanism is used by XIVE to notify the hypervisor that a vCPU is not dispatched on a HW thread. Tested on a QEMU PowerNV machine and a simple QEMU pseries guest doing network on a local bridge.

The XIVE interrupt controller offers a way to increase the XIVE resources per chip by configuring multiple XIVE blocks on a chip. This is not currently supported by the model. However, some configurations, such as OPAL/skiboot, use one block-per-chip configuration with some optimizations. One of them is to override the hardwired chip ID by the block id in the PowerBUS operations and for CAM line compares. This patchset improves the support for this setup. Tested with 4 chips.

A series from Suraj adding guest support in the Radix MMU model of the QEMU PowerNV machine is still required and will be send later. The whole patchset can be found under :

https://github.com/legoater/qemu/tree/powernv-4.2

Based on top of Greg's patchset "ppc: Consolidate QOM links and pointers to the same object".

Thanks,

C.

Changes since v4:

- rebased on QEMU 4.2-rc1
- better commit logs
- moved fixes at the beginning of the patchset
- reworked pnv_xive_match_nvt() handler to loop on the all threads of a PnvChip

Changes since v3:

- reworked the patches introducing the XiveFabric and XivePresenter interfaces
- moved the get_block_id() handler to the XiveRouter
- new small addons related to the format of the trigger data

Changes since v2:

- introduced the XiveFabric and XivePresenter interfaces
- removed the need of a XiveRouter pointer under XiveTCTX

Changes since v1:

- minor extra fixes
- split the escalation support in different patches
- kept the XiveRouter type for XiveTCTX back pointer (will address this in P10)
- removed pnv_xive_vst_size(). Really broken on indirect tables.
- improved the dump of the NVT table
- introduce pnv_xive_get_block_id()

Cédric Le Goater (23):
ppc/xive: Record the IPB in the associated NVT ppc/xive: Introduce helpers for the NVT id
ppc/pnv: Remove pnv_xive_vst_size() routine
ppc/pnv: Dump the XIVE NVT table
ppc/pnv: Quiesce some XIVE errors
ppc/xive: Introduce OS CAM line helpers
ppc/xive: Check V bit in TM_PULL_POOL_CTX
ppc/xive: Introduce a XivePresenter interface
ppc/xive: Implement the XivePresenter interface ppc/pnv: Loop on the threads of the chip to find a matching NVT ppc/pnv: Introduce a pnv_xive_is_cpu_enabled() helper ppc/xive: Introduce a XiveFabric interface
ppc/pnv: Implement the XiveFabric interface
ppc/spapr: Implement the XiveFabric interface
ppc/xive: Use the XiveFabric and XivePresenter interfaces ppc/xive: Extend the TIMA operation with a XivePresenter parameter ppc/pnv: Clarify how the TIMA is accessed on a multichip system ppc/xive: Move the TIMA operations to the controller model ppc/xive: Remove the get_tctx() XiveRouter handler ppc/xive: Introduce a xive_tctx_ipb_update() helper ppc/xive: Synthesize interrupt from the saved IPB in the NVT ppc/pnv: Introduce a pnv_xive_block_id() helper ppc/pnv: Extend XiveRouter with a get_block_id() handler

 include/hw/ppc/pnv.h       |  15 ++
 include/hw/ppc/pnv_xive.h  |   3 -
 include/hw/ppc/xive.h      |  72 ++++++--
 include/hw/ppc/xive_regs.h |  24 +++
 hw/intc/pnv_xive.c         | 360 ++++++++++++++++++++++++-------------
 hw/intc/spapr_xive.c       |  88 ++++++++-
 hw/intc/xive.c             | 350 ++++++++++++++++++++----------------
 hw/ppc/pnv.c               |  37 +++-
 hw/ppc/spapr.c             |  36 ++++
 9 files changed, 691 insertions(+), 294 deletions(-)
Signature · 1 line
2.21.0
#1 lore raw

PATCH v5 1/23 ppc/xive: Record the IPB in the associated NVT

When an interrupt can not be presented to a vCPU, because it is not running on any of the HW treads, the XIVE presenter updates the Interrupt Pending Buffer register of the associated XIVE NVT structure. This is only done if backlog is activated in the END but this is generally the case.

The current code assumes that the fields of the NVT structure is architected with the same layout of the thread interrupt context registers. Fix this assumption and define an offset for the IPB register backup value in the NVT.

---
 include/hw/ppc/xive_regs.h |  1 +
 hw/intc/xive.c             | 11 +++++++++--
 2 files changed, 10 insertions(+), 2 deletions(-)
include/hw/ppc/xive_regs.h +1 -0
index 55307cd1533c..530f232b04f8 100644
@@ -255,6 +255,7 @@ typedef struct XiveNVT {
         uint32_t        w2;
         uint32_t        w3;
         uint32_t        w4;
+#define NVT_W4_IPB               PPC_BITMASK32(16, 23)
         uint32_t        w5;
         uint32_t        w6;
         uint32_t        w7;
hw/intc/xive.c +9 -2
index 3d472e29c858..177663d2b43e 100644
@@ -1607,14 +1607,21 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
      * - logical server : forward request to IVPE (not supported)
      */
     if (xive_end_is_backlog(&end)) {
+        uint8_t ipb;
+
         if (format == 1) {
             qemu_log_mask(LOG_GUEST_ERROR,
                           "XIVE: END %x/%x invalid config: F1 & backlog\n",
                           end_blk, end_idx);
             return;
         }
-        /* Record the IPB in the associated NVT structure */
-        ipb_update((uint8_t *) &nvt.w4, priority);
+        /*
+         * Record the IPB in the associated NVT structure for later
+         * use. The presenter will resend the interrupt when the vCPU
+         * is dispatched again on a HW thread.
+         */
+        ipb = xive_get_field32(NVT_W4_IPB, nvt.w4) | priority_to_ipb(priority);
+        nvt.w4 = xive_set_field32(NVT_W4_IPB, nvt.w4, ipb);
         xive_router_write_nvt(xrtr, nvt_blk, nvt_idx, &nvt, 4);
 
         /*
Signature · 1 line
2.21.0
Reviewed-by on v5 1/23
#2 lore raw

Re: [PATCH for-5.0 v5 01/23] ppc/xive: Record the IPB in the associated NVT

Quoted 11 lines · On Fri, 15 Nov 2019 17:24:14 +0100 Cédric Le Goater <[email protected]> wrote:

When an interrupt can not be presented to a vCPU, because it is not running on any of the HW treads, the XIVE presenter updates the Interrupt Pending Buffer register of the associated XIVE NVT structure. This is only done if backlog is activated in the END but this is generally the case.

The current code assumes that the fields of the NVT structure is architected with the same layout of the thread interrupt context registers. Fix this assumption and define an offset for the IPB register backup value in the NVT.

Does this fix a visible bug in the powernv machine ? If so, maybe worth describing the symptoms.

Anyway, this seems conforment to the XIVE spec I have, so FWIW:

Quoted 46 lines
---
 include/hw/ppc/xive_regs.h |  1 +
 hw/intc/xive.c             | 11 +++++++++--
 2 files changed, 10 insertions(+), 2 deletions(-)
include/hw/ppc/xive_regs.h +1 -0
index 55307cd1533c..530f232b04f8 100644
@@ -255,6 +255,7 @@ typedef struct XiveNVT {
         uint32_t        w2;
         uint32_t        w3;
         uint32_t        w4;
+#define NVT_W4_IPB               PPC_BITMASK32(16, 23)
         uint32_t        w5;
         uint32_t        w6;
         uint32_t        w7;
hw/intc/xive.c +9 -2
index 3d472e29c858..177663d2b43e 100644
@@ -1607,14 +1607,21 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
      * - logical server : forward request to IVPE (not supported)
      */
     if (xive_end_is_backlog(&end)) {
+        uint8_t ipb;
+
         if (format == 1) {
             qemu_log_mask(LOG_GUEST_ERROR,
                           "XIVE: END %x/%x invalid config: F1 & backlog\n",
                           end_blk, end_idx);
             return;
         }
-        /* Record the IPB in the associated NVT structure */
-        ipb_update((uint8_t *) &nvt.w4, priority);
+        /*
+         * Record the IPB in the associated NVT structure for later
+         * use. The presenter will resend the interrupt when the vCPU
+         * is dispatched again on a HW thread.
+         */
+        ipb = xive_get_field32(NVT_W4_IPB, nvt.w4) | priority_to_ipb(priority);
+        nvt.w4 = xive_set_field32(NVT_W4_IPB, nvt.w4, ipb);
         xive_router_write_nvt(xrtr, nvt_blk, nvt_idx, &nvt, 4);
 
         /*
#3 lore raw

Re: [PATCH for-5.0 v5 01/23] ppc/xive: Record the IPB in the associated NVT

Quoted 9 lines · Greg Kurz
Quoted 6 lines · On Fri, 15 Nov 2019 17:24:14 +0100 Cédric Le Goater <[email protected]> wrote:

When an interrupt can not be presented to a vCPU, because it is not running on any of the HW treads, the XIVE presenter updates the Interrupt Pending Buffer register of the associated XIVE NVT structure. This is only done if backlog is activated in the END but this is generally the case.

Quoted 8 lines
Quoted 5 lines

The current code assumes that the fields of the NVT structure is architected with the same layout of the thread interrupt context registers. Fix this assumption and define an offset for the IPB register backup value in the NVT.

Does this fix a visible bug in the powernv machine ? If so, maybe worth describing the symptoms.

no. this is a preliminary work for escalation suppport and to dump the contents of the NVT table.

Quoted 1 line

Anyway, this seems conforment to the XIVE spec I have, so FWIW:

FYI, the NVT structure evolves in the XIVE2 specs.

C.

Quoted 49 lines
Quoted 46 lines
---
 include/hw/ppc/xive_regs.h |  1 +
 hw/intc/xive.c             | 11 +++++++++--
 2 files changed, 10 insertions(+), 2 deletions(-)
include/hw/ppc/xive_regs.h +1 -0
index 55307cd1533c..530f232b04f8 100644
@@ -255,6 +255,7 @@ typedef struct XiveNVT {
         uint32_t        w2;
         uint32_t        w3;
         uint32_t        w4;
+#define NVT_W4_IPB               PPC_BITMASK32(16, 23)
         uint32_t        w5;
         uint32_t        w6;
         uint32_t        w7;
hw/intc/xive.c +9 -2
index 3d472e29c858..177663d2b43e 100644
@@ -1607,14 +1607,21 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
      * - logical server : forward request to IVPE (not supported)
      */
     if (xive_end_is_backlog(&end)) {
+        uint8_t ipb;
+
         if (format == 1) {
             qemu_log_mask(LOG_GUEST_ERROR,
                           "XIVE: END %x/%x invalid config: F1 & backlog\n",
                           end_blk, end_idx);
             return;
         }
-        /* Record the IPB in the associated NVT structure */
-        ipb_update((uint8_t *) &nvt.w4, priority);
+        /*
+         * Record the IPB in the associated NVT structure for later
+         * use. The presenter will resend the interrupt when the vCPU
+         * is dispatched again on a HW thread.
+         */
+        ipb = xive_get_field32(NVT_W4_IPB, nvt.w4) | priority_to_ipb(priority);
+        nvt.w4 = xive_set_field32(NVT_W4_IPB, nvt.w4, ipb);
         xive_router_write_nvt(xrtr, nvt_blk, nvt_idx, &nvt, 4);
 
         /*
Applied
#4 lore raw

Re: [PATCH for-5.0 v5 01/23] ppc/xive: Record the IPB in the associated NVT

In reply to Cédric Le Goater
Quoted 6 lines · Cédric Le Goater

When an interrupt can not be presented to a vCPU, because it is not running on any of the HW treads, the XIVE presenter updates the Interrupt Pending Buffer register of the associated XIVE NVT structure. This is only done if backlog is activated in the END but this is generally the case.

Quoted 6 lines

The current code assumes that the fields of the NVT structure is architected with the same layout of the thread interrupt context registers. Fix this assumption and define an offset for the IPB register backup value in the NVT.

Applied to ppc-for-5.0, thanks.

Quoted 45 lines
---
 include/hw/ppc/xive_regs.h |  1 +
 hw/intc/xive.c             | 11 +++++++++--
 2 files changed, 10 insertions(+), 2 deletions(-)
include/hw/ppc/xive_regs.h +1 -0
index 55307cd1533c..530f232b04f8 100644
@@ -255,6 +255,7 @@ typedef struct XiveNVT {
         uint32_t        w2;
         uint32_t        w3;
         uint32_t        w4;
+#define NVT_W4_IPB               PPC_BITMASK32(16, 23)
         uint32_t        w5;
         uint32_t        w6;
         uint32_t        w7;
hw/intc/xive.c +9 -2
index 3d472e29c858..177663d2b43e 100644
@@ -1607,14 +1607,21 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
      * - logical server : forward request to IVPE (not supported)
      */
     if (xive_end_is_backlog(&end)) {
+        uint8_t ipb;
+
         if (format == 1) {
             qemu_log_mask(LOG_GUEST_ERROR,
                           "XIVE: END %x/%x invalid config: F1 & backlog\n",
                           end_blk, end_idx);
             return;
         }
-        /* Record the IPB in the associated NVT structure */
-        ipb_update((uint8_t *) &nvt.w4, priority);
+        /*
+         * Record the IPB in the associated NVT structure for later
+         * use. The presenter will resend the interrupt when the vCPU
+         * is dispatched again on a HW thread.
+         */
+        ipb = xive_get_field32(NVT_W4_IPB, nvt.w4) | priority_to_ipb(priority);
+        nvt.w4 = xive_set_field32(NVT_W4_IPB, nvt.w4, ipb);
         xive_router_write_nvt(xrtr, nvt_blk, nvt_idx, &nvt, 4);
 
         /*
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#5 lore raw

PATCH v5 2/23 ppc/xive: Introduce helpers for the NVT id

In reply to Cédric Le Goater

Each vCPU in the system is identified with an NVT identifier which is pushed in the OS CAM line (QW1W2) of the HW thread interrupt context register when the vCPU is dispatched on a HW thread. This identifier is used by the presenter subengine to find a matching target to notify of an event. It is also used to fetch the associate NVT structure which may contain pending interrupts that need a resend.

Add a couple of helpers for the NVT ids. The NVT space is 19 bits wide, giving a maximum of 512K per chip.

---
 include/hw/ppc/xive.h      |  5 -----
 include/hw/ppc/xive_regs.h | 21 +++++++++++++++++++++
 2 files changed, 21 insertions(+), 5 deletions(-)
include/hw/ppc/xive.h +0 -5
index 8fd439ec9bba..fa7adf87feb2 100644
@@ -418,11 +418,6 @@ Object *xive_tctx_create(Object *cpu, XiveRouter *xrtr, Error **errp);
 void xive_tctx_reset(XiveTCTX *tctx);
 void xive_tctx_destroy(XiveTCTX *tctx);
 
-static inline uint32_t xive_nvt_cam_line(uint8_t nvt_blk, uint32_t nvt_idx)
-{
-    return (nvt_blk << 19) | nvt_idx;
-}
-
 /*
  * KVM XIVE device helpers
  */
include/hw/ppc/xive_regs.h +21 -0
index 530f232b04f8..1a5622f8ded8 100644
@@ -272,4 +272,25 @@ typedef struct XiveNVT {
 
 #define xive_nvt_is_valid(nvt)    (be32_to_cpu((nvt)->w0) & NVT_W0_VALID)
 
+/*
+ * The VP number space in a block is defined by the END_W6_NVT_INDEX
+ * field of the XIVE END
+ */
+#define XIVE_NVT_SHIFT                19
+
+static inline uint32_t xive_nvt_cam_line(uint8_t nvt_blk, uint32_t nvt_idx)
+{
+    return (nvt_blk << XIVE_NVT_SHIFT) | nvt_idx;
+}
+
+static inline uint32_t xive_nvt_idx(uint32_t cam_line)
+{
+    return cam_line & ((1 << XIVE_NVT_SHIFT) - 1);
+}
+
+static inline uint32_t xive_nvt_blk(uint32_t cam_line)
+{
+    return (cam_line >> XIVE_NVT_SHIFT) & 0xf;
+}
+
 #endif /* PPC_XIVE_REGS_H */
Signature · 1 line
2.21.0
Applied
#6 lore raw

Re: [PATCH for-5.0 v5 02/23] ppc/xive: Introduce helpers for the NVT id

Quoted 11 lines · Cédric Le Goater

Each vCPU in the system is identified with an NVT identifier which is pushed in the OS CAM line (QW1W2) of the HW thread interrupt context register when the vCPU is dispatched on a HW thread. This identifier is used by the presenter subengine to find a matching target to notify of an event. It is also used to fetch the associate NVT structure which may contain pending interrupts that need a resend.

Add a couple of helpers for the NVT ids. The NVT space is 19 bits wide, giving a maximum of 512K per chip.

Applied to ppc-for-5.0, thanks.

Quoted 51 lines
---
 include/hw/ppc/xive.h      |  5 -----
 include/hw/ppc/xive_regs.h | 21 +++++++++++++++++++++
 2 files changed, 21 insertions(+), 5 deletions(-)
include/hw/ppc/xive.h +0 -5
index 8fd439ec9bba..fa7adf87feb2 100644
@@ -418,11 +418,6 @@ Object *xive_tctx_create(Object *cpu, XiveRouter *xrtr, Error **errp);
 void xive_tctx_reset(XiveTCTX *tctx);
 void xive_tctx_destroy(XiveTCTX *tctx);
 
-static inline uint32_t xive_nvt_cam_line(uint8_t nvt_blk, uint32_t nvt_idx)
-{
-    return (nvt_blk << 19) | nvt_idx;
-}
-
 /*
  * KVM XIVE device helpers
  */
include/hw/ppc/xive_regs.h +21 -0
index 530f232b04f8..1a5622f8ded8 100644
@@ -272,4 +272,25 @@ typedef struct XiveNVT {
 
 #define xive_nvt_is_valid(nvt)    (be32_to_cpu((nvt)->w0) & NVT_W0_VALID)
 
+/*
+ * The VP number space in a block is defined by the END_W6_NVT_INDEX
+ * field of the XIVE END
+ */
+#define XIVE_NVT_SHIFT                19
+
+static inline uint32_t xive_nvt_cam_line(uint8_t nvt_blk, uint32_t nvt_idx)
+{
+    return (nvt_blk << XIVE_NVT_SHIFT) | nvt_idx;
+}
+
+static inline uint32_t xive_nvt_idx(uint32_t cam_line)
+{
+    return cam_line & ((1 << XIVE_NVT_SHIFT) - 1);
+}
+
+static inline uint32_t xive_nvt_blk(uint32_t cam_line)
+{
+    return (cam_line >> XIVE_NVT_SHIFT) & 0xf;
+}
+
 #endif /* PPC_XIVE_REGS_H */
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#7 lore raw

Re: [PATCH for-5.0 v5 02/23] ppc/xive: Introduce helpers for the NVT id

In reply to Cédric Le Goater
Quoted 39 lines · On Fri, 15 Nov 2019 17:24:15 +0100 Cédric Le Goater <[email protected]> wrote:

Each vCPU in the system is identified with an NVT identifier which is pushed in the OS CAM line (QW1W2) of the HW thread interrupt context register when the vCPU is dispatched on a HW thread. This identifier is used by the presenter subengine to find a matching target to notify of an event. It is also used to fetch the associate NVT structure which may contain pending interrupts that need a resend.

Add a couple of helpers for the NVT ids. The NVT space is 19 bits wide, giving a maximum of 512K per chip.

---
 include/hw/ppc/xive.h      |  5 -----
 include/hw/ppc/xive_regs.h | 21 +++++++++++++++++++++
 2 files changed, 21 insertions(+), 5 deletions(-)
include/hw/ppc/xive.h +0 -5
index 8fd439ec9bba..fa7adf87feb2 100644
@@ -418,11 +418,6 @@ Object *xive_tctx_create(Object *cpu, XiveRouter *xrtr, Error **errp);
 void xive_tctx_reset(XiveTCTX *tctx);
 void xive_tctx_destroy(XiveTCTX *tctx);
 
-static inline uint32_t xive_nvt_cam_line(uint8_t nvt_blk, uint32_t nvt_idx)
-{
-    return (nvt_blk << 19) | nvt_idx;
-}
-
 /*
  * KVM XIVE device helpers
  */
include/hw/ppc/xive_regs.h +0 -0
index 530f232b04f8..1a5622f8ded8 100644
@@ -272,4 +272,25 @@ typedef struct XiveNVT {
 
 #define xive_nvt_is_valid(nvt)    (be32_to_cpu((nvt)->w0) & NVT_W0_VALID)

include/hw/ppc/xive_regs.h typedef struct XiveNVT { · line 283

Quoted 10 lines
include/hw/ppc/xive_regs.h +9 -0
 
+/*
+ * The VP number space in a block is defined by the END_W6_NVT_INDEX
+ * field of the XIVE END
+ */
+#define XIVE_NVT_SHIFT                19
+
+static inline uint32_t xive_nvt_cam_line(uint8_t nvt_blk, uint32_t nvt_idx)
+{
+    return (nvt_blk << XIVE_NVT_SHIFT) | nvt_idx;

Shouldn't we ensure nvt_idx fits in the 19 bits ?

Quoted 13 lines
include/hw/ppc/xive_regs.h +12 -0
+}
+
+static inline uint32_t xive_nvt_idx(uint32_t cam_line)
+{
+    return cam_line & ((1 << XIVE_NVT_SHIFT) - 1);
+}
+
+static inline uint32_t xive_nvt_blk(uint32_t cam_line)
+{
+    return (cam_line >> XIVE_NVT_SHIFT) & 0xf;
+}
+
 #endif /* PPC_XIVE_REGS_H */
#8 lore raw

Re: [PATCH for-5.0 v5 02/23] ppc/xive: Introduce helpers for the NVT id

Quoted 43 lines · Greg Kurz
Quoted 40 lines · On Fri, 15 Nov 2019 17:24:15 +0100 Cédric Le Goater <[email protected]> wrote:

Each vCPU in the system is identified with an NVT identifier which is pushed in the OS CAM line (QW1W2) of the HW thread interrupt context register when the vCPU is dispatched on a HW thread. This identifier is used by the presenter subengine to find a matching target to notify of an event. It is also used to fetch the associate NVT structure which may contain pending interrupts that need a resend.

Add a couple of helpers for the NVT ids. The NVT space is 19 bits wide, giving a maximum of 512K per chip.

---
 include/hw/ppc/xive.h      |  5 -----
 include/hw/ppc/xive_regs.h | 21 +++++++++++++++++++++
 2 files changed, 21 insertions(+), 5 deletions(-)
include/hw/ppc/xive.h +0 -5
index 8fd439ec9bba..fa7adf87feb2 100644
@@ -418,11 +418,6 @@ Object *xive_tctx_create(Object *cpu, XiveRouter *xrtr, Error **errp);
 void xive_tctx_reset(XiveTCTX *tctx);
 void xive_tctx_destroy(XiveTCTX *tctx);
 
-static inline uint32_t xive_nvt_cam_line(uint8_t nvt_blk, uint32_t nvt_idx)
-{
-    return (nvt_blk << 19) | nvt_idx;
-}
-
 /*
  * KVM XIVE device helpers
  */
include/hw/ppc/xive_regs.h +0 -0
index 530f232b04f8..1a5622f8ded8 100644
@@ -272,4 +272,25 @@ typedef struct XiveNVT {
 
 #define xive_nvt_is_valid(nvt)    (be32_to_cpu((nvt)->w0) & NVT_W0_VALID)
 
Quoted 11 lines

include/hw/ppc/xive_regs.h typedef struct XiveNVT { · line 283

Quoted 9 lines
include/hw/ppc/xive_regs.h +9 -0
+/*
+ * The VP number space in a block is defined by the END_W6_NVT_INDEX
+ * field of the XIVE END
+ */
+#define XIVE_NVT_SHIFT                19
+
+static inline uint32_t xive_nvt_cam_line(uint8_t nvt_blk, uint32_t nvt_idx)
+{
+    return (nvt_blk << XIVE_NVT_SHIFT) | nvt_idx;

Shouldn't we ensure nvt_idx fits in the 19 bits ?

yes. We should use the END_W6_NVT_INDEX mask. We are fine today because the NVT index is extracted from the end w6 using xive_get_field32() or computed from the PIR using the appropriate mask.

Something to improve.

Quoted 15 lines
Quoted 14 lines
include/hw/ppc/xive_regs.h +12 -0
+}
+
+static inline uint32_t xive_nvt_idx(uint32_t cam_line)
+{
+    return cam_line & ((1 << XIVE_NVT_SHIFT) - 1);
+}
+
+static inline uint32_t xive_nvt_blk(uint32_t cam_line)
+{
+    return (cam_line >> XIVE_NVT_SHIFT) & 0xf;
+}
+
 #endif /* PPC_XIVE_REGS_H */
#9 lore raw

PATCH v5 3/23 ppc/pnv: Remove pnv_xive_vst_size() routine

In reply to Cédric Le Goater

pnv_xive_vst_size() tries to compute the size of a VSD table from the information given by FW. The number of entries of the table are deduced from the result and the MMIO regions of the ESBs and the END ESBs are then resized accordingly with the computed value. This reduces the number of elements that can be addressed by the ESB pages.

The maximum number of elements of a direct table can contain is simply:

Table size / sizeof(XIVE structure)

An indirect table is a one page array of VSDs pointing to subpages containing XIVE virtual structures and the maximum number of elements an indirect table can contain :

(PAGE_SIZE / sizeof(vsd)) * (PAGE_SIZE / sizeof(XIVE structure))

which gives us 16M for XiveENDs, 8M for XiveNVTs. That's more than the associated VC and PC BARS can address.

The result returned by pnv_xive_vst_size() for indirect tables is incorrect and can not be used to reduce the size of the MMIO region of a XIVE resource using an indirect table, such as ENDs in skiboot.

Remove pnv_xive_vst_size() and use a simpler form for direct tables only. Keep the resizing of the MMIO region for direct tables only as this is still useful for the ESB MMIO window.

---
 hw/intc/pnv_xive.c | 112 +++++++++++++++++----------------------------
 1 file changed, 43 insertions(+), 69 deletions(-)
hw/intc/pnv_xive.c +43 -69
index 4e56c2e4689c..a4d80fd5e79c 100644
@@ -123,36 +123,22 @@ static uint64_t pnv_xive_vst_page_size_allowed(uint32_t page_shift)
          page_shift == 21 || page_shift == 24;
 }
 
-static uint64_t pnv_xive_vst_size(uint64_t vsd)
-{
-    uint64_t vst_tsize = 1ull << (GETFIELD(VSD_TSIZE, vsd) + 12);
-
-    /*
-     * Read the first descriptor to get the page size of the indirect
-     * table.
-     */
-    if (VSD_INDIRECT & vsd) {
-        uint32_t nr_pages = vst_tsize / XIVE_VSD_SIZE;
-        uint32_t page_shift;
-
-        vsd = ldq_be_dma(&address_space_memory, vsd & VSD_ADDRESS_MASK);
-        page_shift = GETFIELD(VSD_TSIZE, vsd) + 12;
-
-        if (!pnv_xive_vst_page_size_allowed(page_shift)) {
-            return 0;
-        }
-
-        return nr_pages * (1ull << page_shift);
-    }
-
-    return vst_tsize;
-}
-
 static uint64_t pnv_xive_vst_addr_direct(PnvXive *xive, uint32_t type,
                                          uint64_t vsd, uint32_t idx)
 {
     const XiveVstInfo *info = &vst_infos[type];
     uint64_t vst_addr = vsd & VSD_ADDRESS_MASK;
+    uint64_t vst_tsize = 1ull << (GETFIELD(VSD_TSIZE, vsd) + 12);
+    uint32_t idx_max;
+
+    idx_max = vst_tsize / info->size - 1;
+    if (idx > idx_max) {
+#ifdef XIVE_DEBUG
+        xive_error(xive, "VST: %s entry %x out of range [ 0 .. %x ] !?",
+                   info->name, idx, idx_max);
+#endif
+        return 0;
+    }
 
     return vst_addr + idx * info->size;
 }
@@ -215,7 +201,6 @@ static uint64_t pnv_xive_vst_addr(PnvXive *xive, uint32_t type, uint8_t blk,
 {
     const XiveVstInfo *info = &vst_infos[type];
     uint64_t vsd;
-    uint32_t idx_max;
 
     if (blk >= info->max_blocks) {
         xive_error(xive, "VST: invalid block id %d for VST %s %d !?",
@@ -232,15 +217,6 @@ static uint64_t pnv_xive_vst_addr(PnvXive *xive, uint32_t type, uint8_t blk,
         return xive ? pnv_xive_vst_addr(xive, type, blk, idx) : 0;
     }
 
-    idx_max = pnv_xive_vst_size(vsd) / info->size - 1;
-    if (idx > idx_max) {
-#ifdef XIVE_DEBUG
-        xive_error(xive, "VST: %s entry %x/%x out of range [ 0 .. %x ] !?",
-                   info->name, blk, idx, idx_max);
-#endif
-        return 0;
-    }
-
     if (VSD_INDIRECT & vsd) {
         return pnv_xive_vst_addr_indirect(xive, type, vsd, idx);
     }
@@ -453,19 +429,12 @@ static uint64_t pnv_xive_pc_size(PnvXive *xive)
     return (~xive->regs[CQ_PC_BARM >> 3] + 1) & CQ_PC_BARM_MASK;
 }
 
-static uint32_t pnv_xive_nr_ipis(PnvXive *xive)
+static uint32_t pnv_xive_nr_ipis(PnvXive *xive, uint8_t blk)
 {
-    uint8_t blk = xive->chip->chip_id;
-
-    return pnv_xive_vst_size(xive->vsds[VST_TSEL_SBE][blk]) * SBE_PER_BYTE;
-}
-
-static uint32_t pnv_xive_nr_ends(PnvXive *xive)
-{
-    uint8_t blk = xive->chip->chip_id;
+    uint64_t vsd = xive->vsds[VST_TSEL_SBE][blk];
+    uint64_t vst_tsize = 1ull << (GETFIELD(VSD_TSIZE, vsd) + 12);
 
-    return pnv_xive_vst_size(xive->vsds[VST_TSEL_EQDT][blk])
-        / vst_infos[VST_TSEL_EQDT].size;
+    return VSD_INDIRECT & vsd ? 0 : vst_tsize * SBE_PER_BYTE;
 }
 
 /*
@@ -598,6 +567,7 @@ static void pnv_xive_vst_set_exclusive(PnvXive *xive, uint8_t type,
     XiveSource *xsrc = &xive->ipi_source;
     const XiveVstInfo *info = &vst_infos[type];
     uint32_t page_shift = GETFIELD(VSD_TSIZE, vsd) + 12;
+    uint64_t vst_tsize = 1ull << page_shift;
     uint64_t vst_addr = vsd & VSD_ADDRESS_MASK;
 
     /* Basic checks */
@@ -633,11 +603,16 @@ static void pnv_xive_vst_set_exclusive(PnvXive *xive, uint8_t type,
 
     case VST_TSEL_EQDT:
         /*
-         * Backing store pages for the END. Compute the number of ENDs
-         * provisioned by FW and resize the END ESB window accordingly.
+         * Backing store pages for the END.
+         *
+         * If the table is direct, we can compute the number of PQ
+         * entries provisioned by FW (such as skiboot) and resize the
+         * END ESB window accordingly.
          */
-        memory_region_set_size(&end_xsrc->esb_mmio, pnv_xive_nr_ends(xive) *
-                               (1ull << (end_xsrc->esb_shift + 1)));
+        if (!(VSD_INDIRECT & vsd)) {
+            memory_region_set_size(&end_xsrc->esb_mmio, (vst_tsize / info->size)
+                                   * (1ull << xsrc->esb_shift));
+        }
         memory_region_add_subregion(&xive->end_edt_mmio, 0,
                                     &end_xsrc->esb_mmio);
         break;
@@ -646,11 +621,16 @@ static void pnv_xive_vst_set_exclusive(PnvXive *xive, uint8_t type,
         /*
          * Backing store pages for the source PQ bits. The model does
          * not use these PQ bits backed in RAM because the XiveSource
-         * model has its own. Compute the number of IRQs provisioned
-         * by FW and resize the IPI ESB window accordingly.
+         * model has its own.
+         *
+         * If the table is direct, we can compute the number of PQ
+         * entries provisioned by FW (such as skiboot) and resize the
+         * ESB window accordingly.
          */
-        memory_region_set_size(&xsrc->esb_mmio, pnv_xive_nr_ipis(xive) *
-                               (1ull << xsrc->esb_shift));
+        if (!(VSD_INDIRECT & vsd)) {
+            memory_region_set_size(&xsrc->esb_mmio, vst_tsize * SBE_PER_BYTE
+                                   * (1ull << xsrc->esb_shift));
+        }
         memory_region_add_subregion(&xive->ipi_edt_mmio, 0, &xsrc->esb_mmio);
         break;
 
@@ -1579,8 +1559,7 @@ void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
     XiveRouter *xrtr = XIVE_ROUTER(xive);
     uint8_t blk = xive->chip->chip_id;
     uint32_t srcno0 = XIVE_EAS(blk, 0);
-    uint32_t nr_ipis = pnv_xive_nr_ipis(xive);
-    uint32_t nr_ends = pnv_xive_nr_ends(xive);
+    uint32_t nr_ipis = pnv_xive_nr_ipis(xive, blk);
     XiveEAS eas;
     XiveEND end;
     int i;
@@ -1600,21 +1579,16 @@ void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
         }
     }
 
-    monitor_printf(mon, "XIVE[%x] ENDT %08x .. %08x\n", blk, 0, nr_ends - 1);
-    for (i = 0; i < nr_ends; i++) {
-        if (xive_router_get_end(xrtr, blk, i, &end)) {
-            break;
-        }
-        xive_end_pic_print_info(&end, i, mon);
+    monitor_printf(mon, "XIVE[%x] ENDT\n", blk);
+    i = 0;
+    while (!xive_router_get_end(xrtr, blk, i, &end)) {
+        xive_end_pic_print_info(&end, i++, mon);
     }
 
-    monitor_printf(mon, "XIVE[%x] END Escalation %08x .. %08x\n", blk, 0,
-                   nr_ends - 1);
-    for (i = 0; i < nr_ends; i++) {
-        if (xive_router_get_end(xrtr, blk, i, &end)) {
-            break;
-        }
-        xive_end_eas_pic_print_info(&end, i, mon);
+    monitor_printf(mon, "XIVE[%x] END Escalation EAT\n", blk);
+    i = 0;
+    while (!xive_router_get_end(xrtr, blk, i, &end)) {
+        xive_end_eas_pic_print_info(&end, i++, mon);
     }
 }
 
Signature · 1 line
2.21.0
Applied
#10 lore raw

Re: [PATCH for-5.0 v5 03/23] ppc/pnv: Remove pnv_xive_vst_size() routine

Quoted 23 lines · Cédric Le Goater

pnv_xive_vst_size() tries to compute the size of a VSD table from the information given by FW. The number of entries of the table are deduced from the result and the MMIO regions of the ESBs and the END ESBs are then resized accordingly with the computed value. This reduces the number of elements that can be addressed by the ESB pages.

The maximum number of elements of a direct table can contain is simply:

Table size / sizeof(XIVE structure)

An indirect table is a one page array of VSDs pointing to subpages containing XIVE virtual structures and the maximum number of elements an indirect table can contain :

(PAGE_SIZE / sizeof(vsd)) * (PAGE_SIZE / sizeof(XIVE structure))

which gives us 16M for XiveENDs, 8M for XiveNVTs. That's more than the associated VC and PC BARS can address.

The result returned by pnv_xive_vst_size() for indirect tables is incorrect and can not be used to reduce the size of the MMIO region of a XIVE resource using an indirect table, such as ENDs in skiboot.

Quoted 5 lines

Remove pnv_xive_vst_size() and use a simpler form for direct tables only. Keep the resizing of the MMIO region for direct tables only as this is still useful for the ESB MMIO window.

Applied to ppc-for-5.0, thanks.

Quoted 194 lines
---
 hw/intc/pnv_xive.c | 112 +++++++++++++++++----------------------------
 1 file changed, 43 insertions(+), 69 deletions(-)
hw/intc/pnv_xive.c +43 -69
index 4e56c2e4689c..a4d80fd5e79c 100644
@@ -123,36 +123,22 @@ static uint64_t pnv_xive_vst_page_size_allowed(uint32_t page_shift)
          page_shift == 21 || page_shift == 24;
 }
 
-static uint64_t pnv_xive_vst_size(uint64_t vsd)
-{
-    uint64_t vst_tsize = 1ull << (GETFIELD(VSD_TSIZE, vsd) + 12);
-
-    /*
-     * Read the first descriptor to get the page size of the indirect
-     * table.
-     */
-    if (VSD_INDIRECT & vsd) {
-        uint32_t nr_pages = vst_tsize / XIVE_VSD_SIZE;
-        uint32_t page_shift;
-
-        vsd = ldq_be_dma(&address_space_memory, vsd & VSD_ADDRESS_MASK);
-        page_shift = GETFIELD(VSD_TSIZE, vsd) + 12;
-
-        if (!pnv_xive_vst_page_size_allowed(page_shift)) {
-            return 0;
-        }
-
-        return nr_pages * (1ull << page_shift);
-    }
-
-    return vst_tsize;
-}
-
 static uint64_t pnv_xive_vst_addr_direct(PnvXive *xive, uint32_t type,
                                          uint64_t vsd, uint32_t idx)
 {
     const XiveVstInfo *info = &vst_infos[type];
     uint64_t vst_addr = vsd & VSD_ADDRESS_MASK;
+    uint64_t vst_tsize = 1ull << (GETFIELD(VSD_TSIZE, vsd) + 12);
+    uint32_t idx_max;
+
+    idx_max = vst_tsize / info->size - 1;
+    if (idx > idx_max) {
+#ifdef XIVE_DEBUG
+        xive_error(xive, "VST: %s entry %x out of range [ 0 .. %x ] !?",
+                   info->name, idx, idx_max);
+#endif
+        return 0;
+    }
 
     return vst_addr + idx * info->size;
 }
@@ -215,7 +201,6 @@ static uint64_t pnv_xive_vst_addr(PnvXive *xive, uint32_t type, uint8_t blk,
 {
     const XiveVstInfo *info = &vst_infos[type];
     uint64_t vsd;
-    uint32_t idx_max;
 
     if (blk >= info->max_blocks) {
         xive_error(xive, "VST: invalid block id %d for VST %s %d !?",
@@ -232,15 +217,6 @@ static uint64_t pnv_xive_vst_addr(PnvXive *xive, uint32_t type, uint8_t blk,
         return xive ? pnv_xive_vst_addr(xive, type, blk, idx) : 0;
     }
 
-    idx_max = pnv_xive_vst_size(vsd) / info->size - 1;
-    if (idx > idx_max) {
-#ifdef XIVE_DEBUG
-        xive_error(xive, "VST: %s entry %x/%x out of range [ 0 .. %x ] !?",
-                   info->name, blk, idx, idx_max);
-#endif
-        return 0;
-    }
-
     if (VSD_INDIRECT & vsd) {
         return pnv_xive_vst_addr_indirect(xive, type, vsd, idx);
     }
@@ -453,19 +429,12 @@ static uint64_t pnv_xive_pc_size(PnvXive *xive)
     return (~xive->regs[CQ_PC_BARM >> 3] + 1) & CQ_PC_BARM_MASK;
 }
 
-static uint32_t pnv_xive_nr_ipis(PnvXive *xive)
+static uint32_t pnv_xive_nr_ipis(PnvXive *xive, uint8_t blk)
 {
-    uint8_t blk = xive->chip->chip_id;
-
-    return pnv_xive_vst_size(xive->vsds[VST_TSEL_SBE][blk]) * SBE_PER_BYTE;
-}
-
-static uint32_t pnv_xive_nr_ends(PnvXive *xive)
-{
-    uint8_t blk = xive->chip->chip_id;
+    uint64_t vsd = xive->vsds[VST_TSEL_SBE][blk];
+    uint64_t vst_tsize = 1ull << (GETFIELD(VSD_TSIZE, vsd) + 12);
 
-    return pnv_xive_vst_size(xive->vsds[VST_TSEL_EQDT][blk])
-        / vst_infos[VST_TSEL_EQDT].size;
+    return VSD_INDIRECT & vsd ? 0 : vst_tsize * SBE_PER_BYTE;
 }
 
 /*
@@ -598,6 +567,7 @@ static void pnv_xive_vst_set_exclusive(PnvXive *xive, uint8_t type,
     XiveSource *xsrc = &xive->ipi_source;
     const XiveVstInfo *info = &vst_infos[type];
     uint32_t page_shift = GETFIELD(VSD_TSIZE, vsd) + 12;
+    uint64_t vst_tsize = 1ull << page_shift;
     uint64_t vst_addr = vsd & VSD_ADDRESS_MASK;
 
     /* Basic checks */
@@ -633,11 +603,16 @@ static void pnv_xive_vst_set_exclusive(PnvXive *xive, uint8_t type,
 
     case VST_TSEL_EQDT:
         /*
-         * Backing store pages for the END. Compute the number of ENDs
-         * provisioned by FW and resize the END ESB window accordingly.
+         * Backing store pages for the END.
+         *
+         * If the table is direct, we can compute the number of PQ
+         * entries provisioned by FW (such as skiboot) and resize the
+         * END ESB window accordingly.
          */
-        memory_region_set_size(&end_xsrc->esb_mmio, pnv_xive_nr_ends(xive) *
-                               (1ull << (end_xsrc->esb_shift + 1)));
+        if (!(VSD_INDIRECT & vsd)) {
+            memory_region_set_size(&end_xsrc->esb_mmio, (vst_tsize / info->size)
+                                   * (1ull << xsrc->esb_shift));
+        }
         memory_region_add_subregion(&xive->end_edt_mmio, 0,
                                     &end_xsrc->esb_mmio);
         break;
@@ -646,11 +621,16 @@ static void pnv_xive_vst_set_exclusive(PnvXive *xive, uint8_t type,
         /*
          * Backing store pages for the source PQ bits. The model does
          * not use these PQ bits backed in RAM because the XiveSource
-         * model has its own. Compute the number of IRQs provisioned
-         * by FW and resize the IPI ESB window accordingly.
+         * model has its own.
+         *
+         * If the table is direct, we can compute the number of PQ
+         * entries provisioned by FW (such as skiboot) and resize the
+         * ESB window accordingly.
          */
-        memory_region_set_size(&xsrc->esb_mmio, pnv_xive_nr_ipis(xive) *
-                               (1ull << xsrc->esb_shift));
+        if (!(VSD_INDIRECT & vsd)) {
+            memory_region_set_size(&xsrc->esb_mmio, vst_tsize * SBE_PER_BYTE
+                                   * (1ull << xsrc->esb_shift));
+        }
         memory_region_add_subregion(&xive->ipi_edt_mmio, 0, &xsrc->esb_mmio);
         break;
 
@@ -1579,8 +1559,7 @@ void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
     XiveRouter *xrtr = XIVE_ROUTER(xive);
     uint8_t blk = xive->chip->chip_id;
     uint32_t srcno0 = XIVE_EAS(blk, 0);
-    uint32_t nr_ipis = pnv_xive_nr_ipis(xive);
-    uint32_t nr_ends = pnv_xive_nr_ends(xive);
+    uint32_t nr_ipis = pnv_xive_nr_ipis(xive, blk);
     XiveEAS eas;
     XiveEND end;
     int i;
@@ -1600,21 +1579,16 @@ void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
         }
     }
 
-    monitor_printf(mon, "XIVE[%x] ENDT %08x .. %08x\n", blk, 0, nr_ends - 1);
-    for (i = 0; i < nr_ends; i++) {
-        if (xive_router_get_end(xrtr, blk, i, &end)) {
-            break;
-        }
-        xive_end_pic_print_info(&end, i, mon);
+    monitor_printf(mon, "XIVE[%x] ENDT\n", blk);
+    i = 0;
+    while (!xive_router_get_end(xrtr, blk, i, &end)) {
+        xive_end_pic_print_info(&end, i++, mon);
     }
 
-    monitor_printf(mon, "XIVE[%x] END Escalation %08x .. %08x\n", blk, 0,
-                   nr_ends - 1);
-    for (i = 0; i < nr_ends; i++) {
-        if (xive_router_get_end(xrtr, blk, i, &end)) {
-            break;
-        }
-        xive_end_eas_pic_print_info(&end, i, mon);
+    monitor_printf(mon, "XIVE[%x] END Escalation EAT\n", blk);
+    i = 0;
+    while (!xive_router_get_end(xrtr, blk, i, &end)) {
+        xive_end_eas_pic_print_info(&end, i++, mon);
     }
 }
 
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#11 lore raw

PATCH v5 4/23 ppc/pnv: Dump the XIVE NVT table

In reply to Cédric Le Goater

This is useful to dump the saved contexts of the vCPUs : configuration of the base END index of the vCPU and the Interrupt Pending Buffer register, which is updated when an interrupt can not be presented.

When dumping the NVT table, we skip empty indirect pages which are not necessarily allocated.

---
 include/hw/ppc/xive_regs.h |  2 ++
 hw/intc/pnv_xive.c         | 30 ++++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+)
include/hw/ppc/xive_regs.h +2 -0
index 1a5622f8ded8..94338b4b551e 100644
@@ -252,6 +252,8 @@ typedef struct XiveNVT {
         uint32_t        w0;
 #define NVT_W0_VALID             PPC_BIT32(0)
         uint32_t        w1;
+#define NVT_W1_EQ_BLOCK          PPC_BITMASK32(0, 3)
+#define NVT_W1_EQ_INDEX          PPC_BITMASK32(4, 31)
         uint32_t        w2;
         uint32_t        w3;
         uint32_t        w4;
hw/intc/pnv_xive.c +30 -0
index a4d80fd5e79c..02faf4135e48 100644
@@ -1554,6 +1554,27 @@ static const MemoryRegionOps pnv_xive_pc_ops = {
     },
 };
 
+/*
+ * skiboot uses an indirect NVT table with 64k subpages
+ */
+#define XIVE_NVT_COUNT          (1 << XIVE_NVT_SHIFT)
+#define XIVE_NVT_PER_PAGE       (0x10000 / sizeof(XiveNVT))
+
+static void xive_nvt_pic_print_info(XiveNVT *nvt, uint32_t nvt_idx,
+                                    Monitor *mon)
+{
+    uint8_t  eq_blk = xive_get_field32(NVT_W1_EQ_BLOCK, nvt->w1);
+    uint32_t eq_idx = xive_get_field32(NVT_W1_EQ_INDEX, nvt->w1);
+
+    if (!xive_nvt_is_valid(nvt)) {
+        return;
+    }
+
+    monitor_printf(mon, "  %08x end:%02x/%04x IPB:%02x\n", nvt_idx,
+                   eq_blk, eq_idx,
+                   xive_get_field32(NVT_W4_IPB, nvt->w4));
+}
+
 void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
 {
     XiveRouter *xrtr = XIVE_ROUTER(xive);
@@ -1562,6 +1583,7 @@ void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
     uint32_t nr_ipis = pnv_xive_nr_ipis(xive, blk);
     XiveEAS eas;
     XiveEND end;
+    XiveNVT nvt;
     int i;
 
     monitor_printf(mon, "XIVE[%x] Source %08x .. %08x\n", blk, srcno0,
@@ -1590,6 +1612,14 @@ void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
     while (!xive_router_get_end(xrtr, blk, i, &end)) {
         xive_end_eas_pic_print_info(&end, i++, mon);
     }
+
+    monitor_printf(mon, "XIVE[%x] NVTT %08x .. %08x\n", blk, 0,
+                   XIVE_NVT_COUNT - 1);
+    for (i = 0; i < XIVE_NVT_COUNT; i += XIVE_NVT_PER_PAGE) {
+        while (!xive_router_get_nvt(xrtr, blk, i, &nvt)) {
+            xive_nvt_pic_print_info(&nvt, i++, mon);
+        }
+    }
 }
 
 static void pnv_xive_reset(void *dev)
Signature · 1 line
2.21.0
#12 lore raw

Re: [PATCH for-5.0 v5 04/23] ppc/pnv: Dump the XIVE NVT table

Quoted 31 lines · Cédric Le Goater

This is useful to dump the saved contexts of the vCPUs : configuration of the base END index of the vCPU and the Interrupt Pending Buffer register, which is updated when an interrupt can not be presented.

When dumping the NVT table, we skip empty indirect pages which are not necessarily allocated.

---
 include/hw/ppc/xive_regs.h |  2 ++
 hw/intc/pnv_xive.c         | 30 ++++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+)
include/hw/ppc/xive_regs.h +2 -0
index 1a5622f8ded8..94338b4b551e 100644
@@ -252,6 +252,8 @@ typedef struct XiveNVT {
         uint32_t        w0;
 #define NVT_W0_VALID             PPC_BIT32(0)
         uint32_t        w1;
+#define NVT_W1_EQ_BLOCK          PPC_BITMASK32(0, 3)
+#define NVT_W1_EQ_INDEX          PPC_BITMASK32(4, 31)
         uint32_t        w2;
         uint32_t        w3;
         uint32_t        w4;
hw/intc/pnv_xive.c +0 -0
index a4d80fd5e79c..02faf4135e48 100644

hw/intc/pnv_xive.c static const MemoryRegionOps pnv_xive_pc_ops = { · line 1561

Quoted 8 lines
hw/intc/pnv_xive.c +5 -0
@@ -1554,6 +1554,27 @@ static const MemoryRegionOps pnv_xive_pc_ops = {
     },
 };
 
+/*
+ * skiboot uses an indirect NVT table with 64k subpages
+ */
+#define XIVE_NVT_COUNT          (1 << XIVE_NVT_SHIFT)
+#define XIVE_NVT_PER_PAGE       (0x10000 / sizeof(XiveNVT))

Hrm. So skiboot uses 64kiB pages, but how does the hardware know that? Is the hw fixed to 64kiB pages, or is there some register controlling the subpage size? In which case shouldn't you be looking at that rather than assuming 64kiB?

Quoted 41 lines
hw/intc/pnv_xive.c +24 -0
+static void xive_nvt_pic_print_info(XiveNVT *nvt, uint32_t nvt_idx,
+                                    Monitor *mon)
+{
+    uint8_t  eq_blk = xive_get_field32(NVT_W1_EQ_BLOCK, nvt->w1);
+    uint32_t eq_idx = xive_get_field32(NVT_W1_EQ_INDEX, nvt->w1);
+
+    if (!xive_nvt_is_valid(nvt)) {
+        return;
+    }
+
+    monitor_printf(mon, "  %08x end:%02x/%04x IPB:%02x\n", nvt_idx,
+                   eq_blk, eq_idx,
+                   xive_get_field32(NVT_W4_IPB, nvt->w4));
+}
+
 void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
 {
     XiveRouter *xrtr = XIVE_ROUTER(xive);
@@ -1562,6 +1583,7 @@ void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
     uint32_t nr_ipis = pnv_xive_nr_ipis(xive, blk);
     XiveEAS eas;
     XiveEND end;
+    XiveNVT nvt;
     int i;
 
     monitor_printf(mon, "XIVE[%x] Source %08x .. %08x\n", blk, srcno0,
@@ -1590,6 +1612,14 @@ void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
     while (!xive_router_get_end(xrtr, blk, i, &end)) {
         xive_end_eas_pic_print_info(&end, i++, mon);
     }
+
+    monitor_printf(mon, "XIVE[%x] NVTT %08x .. %08x\n", blk, 0,
+                   XIVE_NVT_COUNT - 1);
+    for (i = 0; i < XIVE_NVT_COUNT; i += XIVE_NVT_PER_PAGE) {
+        while (!xive_router_get_nvt(xrtr, blk, i, &nvt)) {
+            xive_nvt_pic_print_info(&nvt, i++, mon);
+        }
+    }
 }
 
 static void pnv_xive_reset(void *dev)
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#13 lore raw

Re: [PATCH for-5.0 v5 04/23] ppc/pnv: Dump the XIVE NVT table

Quoted 35 lines · David Gibson
Quoted 34 lines · Cédric Le Goater

This is useful to dump the saved contexts of the vCPUs : configuration of the base END index of the vCPU and the Interrupt Pending Buffer register, which is updated when an interrupt can not be presented.

When dumping the NVT table, we skip empty indirect pages which are not necessarily allocated.

---
 include/hw/ppc/xive_regs.h |  2 ++
 hw/intc/pnv_xive.c         | 30 ++++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+)
include/hw/ppc/xive_regs.h +2 -0
index 1a5622f8ded8..94338b4b551e 100644
@@ -252,6 +252,8 @@ typedef struct XiveNVT {
         uint32_t        w0;
 #define NVT_W0_VALID             PPC_BIT32(0)
         uint32_t        w1;
+#define NVT_W1_EQ_BLOCK          PPC_BITMASK32(0, 3)
+#define NVT_W1_EQ_INDEX          PPC_BITMASK32(4, 31)
         uint32_t        w2;
         uint32_t        w3;
         uint32_t        w4;
hw/intc/pnv_xive.c +0 -0
index a4d80fd5e79c..02faf4135e48 100644
@@ -1554,6 +1554,27 @@ static const MemoryRegionOps pnv_xive_pc_ops = {
     },
 };
 
Quoted 5 lines

hw/intc/pnv_xive.c static const MemoryRegionOps pnv_xive_pc_ops = { · line 1561

Quoted 5 lines
hw/intc/pnv_xive.c +5 -0
+/*
+ * skiboot uses an indirect NVT table with 64k subpages
+ */
+#define XIVE_NVT_COUNT          (1 << XIVE_NVT_SHIFT)
+#define XIVE_NVT_PER_PAGE       (0x10000 / sizeof(XiveNVT))

First of all, this is an optimization to skip ranges of NVTs when an indirect page is empty. two assumptions are made : indirect table and page size.

Quoted 3 lines

Hrm. So skiboot uses 64kiB pages, but how does the hardware know that? Is the hw fixed to 64kiB pages, or is there some register controlling the subpage size?

The indirect page size is a field of the VSD structure describing the pages.

Quoted 2 lines

In which case shouldn't you be looking at that rather than assuming 64kiB?

Something cleaner would be :

+static uint64_t pnv_xive_vst_per_page(PnvXive *xive, uint32_t type) +{

+    uint8_t blk = pnv_xive_block_id(xive);
+    uint64_t vsd = xive->vsds[type][blk];
+    const XiveVstInfo *info = &vst_infos[type];
+    uint64_t vsd_addr;
+    uint32_t page_shift;
+
+    /* Get the page size of the indirect table. */
+    vsd_addr = vsd & VSD_ADDRESS_MASK;
+    vsd = ldq_be_dma(&address_space_memory, vsd_addr);

+
+ if (!(vsd & VSD_ADDRESS_MASK)) {
+#ifdef XIVE_DEBUG
+ xive_error(xive, "VST: invalid %s entry %x !?", info->name, idx);
+#endif
+ return 0;
+ }
+
+ page_shift = GETFIELD(VSD_TSIZE, vsd) + 12;
+
+ if (!pnv_xive_vst_page_size_allowed(page_shift)) {
+ xive_error(xive, "VST: invalid %s page shift %d", info->name,
+ page_shift);
+ return 0;
+ }
+
+ return (1ull << page_shift) / info->size;
+}
+
+
#define XIVE_NVT_COUNT (1 << XIVE_NVT_SHIFT)
-#define XIVE_NVT_PER_PAGE (0x10000 / sizeof(XiveNVT))
+#define XIVE_NVT_PER_PAGE(xive) pnv_xive_vst_per_page(xive, VST_TSEL_VPDT)

I will rework that patch and try to keep the info pic command simple.

C.

Quoted 42 lines
Quoted 41 lines
hw/intc/pnv_xive.c +24 -0
+static void xive_nvt_pic_print_info(XiveNVT *nvt, uint32_t nvt_idx,
+                                    Monitor *mon)
+{
+    uint8_t  eq_blk = xive_get_field32(NVT_W1_EQ_BLOCK, nvt->w1);
+    uint32_t eq_idx = xive_get_field32(NVT_W1_EQ_INDEX, nvt->w1);
+
+    if (!xive_nvt_is_valid(nvt)) {
+        return;
+    }
+
+    monitor_printf(mon, "  %08x end:%02x/%04x IPB:%02x\n", nvt_idx,
+                   eq_blk, eq_idx,
+                   xive_get_field32(NVT_W4_IPB, nvt->w4));
+}
+
 void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
 {
     XiveRouter *xrtr = XIVE_ROUTER(xive);
@@ -1562,6 +1583,7 @@ void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
     uint32_t nr_ipis = pnv_xive_nr_ipis(xive, blk);
     XiveEAS eas;
     XiveEND end;
+    XiveNVT nvt;
     int i;
 
     monitor_printf(mon, "XIVE[%x] Source %08x .. %08x\n", blk, srcno0,
@@ -1590,6 +1612,14 @@ void pnv_xive_pic_print_info(PnvXive *xive, Monitor *mon)
     while (!xive_router_get_end(xrtr, blk, i, &end)) {
         xive_end_eas_pic_print_info(&end, i++, mon);
     }
+
+    monitor_printf(mon, "XIVE[%x] NVTT %08x .. %08x\n", blk, 0,
+                   XIVE_NVT_COUNT - 1);
+    for (i = 0; i < XIVE_NVT_COUNT; i += XIVE_NVT_PER_PAGE) {
+        while (!xive_router_get_nvt(xrtr, blk, i, &nvt)) {
+            xive_nvt_pic_print_info(&nvt, i++, mon);
+        }
+    }
 }
 
 static void pnv_xive_reset(void *dev)
#14 lore raw

PATCH v5 5/23 ppc/pnv: Quiesce some XIVE errors

In reply to Cédric Le Goater

When dumping the END and NVT tables, the error logging is too noisy.

---
 hw/intc/pnv_xive.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
hw/intc/pnv_xive.c +5 -1
index 02faf4135e48..a394331ddd6a 100644
@@ -29,7 +29,7 @@
 
 #include "pnv_xive_regs.h"
 
-#define XIVE_DEBUG
+#undef XIVE_DEBUG
 
 /*
  * Virtual structures table (VST)
@@ -157,7 +157,9 @@ static uint64_t pnv_xive_vst_addr_indirect(PnvXive *xive, uint32_t type,
     vsd = ldq_be_dma(&address_space_memory, vsd_addr);
 
     if (!(vsd & VSD_ADDRESS_MASK)) {
+#ifdef XIVE_DEBUG
         xive_error(xive, "VST: invalid %s entry %x !?", info->name, idx);
+#endif
         return 0;
     }
 
@@ -178,7 +180,9 @@ static uint64_t pnv_xive_vst_addr_indirect(PnvXive *xive, uint32_t type,
         vsd = ldq_be_dma(&address_space_memory, vsd_addr);
 
         if (!(vsd & VSD_ADDRESS_MASK)) {
+#ifdef XIVE_DEBUG
             xive_error(xive, "VST: invalid %s entry %x !?", info->name, idx);
+#endif
             return 0;
         }
 
Signature · 1 line
2.21.0
Applied
#15 lore raw

Re: [PATCH for-5.0 v5 05/23] ppc/pnv: Quiesce some XIVE errors

Quoted 3 lines · Cédric Le Goater

When dumping the END and NVT tables, the error logging is too noisy.

Applied, thanks.

Quoted 37 lines
---
 hw/intc/pnv_xive.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
hw/intc/pnv_xive.c +5 -1
index 02faf4135e48..a394331ddd6a 100644
@@ -29,7 +29,7 @@
 
 #include "pnv_xive_regs.h"
 
-#define XIVE_DEBUG
+#undef XIVE_DEBUG
 
 /*
  * Virtual structures table (VST)
@@ -157,7 +157,9 @@ static uint64_t pnv_xive_vst_addr_indirect(PnvXive *xive, uint32_t type,
     vsd = ldq_be_dma(&address_space_memory, vsd_addr);
 
     if (!(vsd & VSD_ADDRESS_MASK)) {
+#ifdef XIVE_DEBUG
         xive_error(xive, "VST: invalid %s entry %x !?", info->name, idx);
+#endif
         return 0;
     }
 
@@ -178,7 +180,9 @@ static uint64_t pnv_xive_vst_addr_indirect(PnvXive *xive, uint32_t type,
         vsd = ldq_be_dma(&address_space_memory, vsd_addr);
 
         if (!(vsd & VSD_ADDRESS_MASK)) {
+#ifdef XIVE_DEBUG
             xive_error(xive, "VST: invalid %s entry %x !?", info->name, idx);
+#endif
             return 0;
         }
 
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#16 lore raw

PATCH v5 6/23 ppc/xive: Introduce OS CAM line helpers

In reply to Cédric Le Goater

The OS CAM line has a special encoding exploited by the HW. Provide helper routines to hide the details to the TIMA command handlers. This also clarifies the endianness of different variables : 'qw1w2' is big-endian and 'cam' is native.

---
 hw/intc/xive.c | 41 ++++++++++++++++++++++++++++++++++++++---
 1 file changed, 38 insertions(+), 3 deletions(-)
hw/intc/xive.c +38 -3
index 177663d2b43e..42e9a11ef731 100644
@@ -337,14 +337,49 @@ static void xive_tm_set_os_pending(XiveTCTX *tctx, hwaddr offset,
     xive_tctx_notify(tctx, TM_QW1_OS);
 }
 
+static void xive_os_cam_decode(uint32_t cam, uint8_t *nvt_blk,
+                               uint32_t *nvt_idx, bool *vo)
+{
+    if (nvt_blk) {
+        *nvt_blk = xive_nvt_blk(cam);
+    }
+    if (nvt_idx) {
+        *nvt_idx = xive_nvt_idx(cam);
+    }
+    if (vo) {
+        *vo = !!(cam & TM_QW1W2_VO);
+    }
+}
+
+static uint32_t xive_tctx_get_os_cam(XiveTCTX *tctx, uint8_t *nvt_blk,
+                                     uint32_t *nvt_idx, bool *vo)
+{
+    uint32_t qw1w2 = xive_tctx_word2(&tctx->regs[TM_QW1_OS]);
+    uint32_t cam = be32_to_cpu(qw1w2);
+
+    xive_os_cam_decode(cam, nvt_blk, nvt_idx, vo);
+    return qw1w2;
+}
+
+static void xive_tctx_set_os_cam(XiveTCTX *tctx, uint32_t qw1w2)
+{
+    memcpy(&tctx->regs[TM_QW1_OS + TM_WORD2], &qw1w2, 4);
+}
+
 static uint64_t xive_tm_pull_os_ctx(XiveTCTX *tctx, hwaddr offset,
                                     unsigned size)
 {
-    uint32_t qw1w2_prev = xive_tctx_word2(&tctx->regs[TM_QW1_OS]);
     uint32_t qw1w2;
+    uint32_t qw1w2_new;
+    uint8_t nvt_blk;
+    uint32_t nvt_idx;
+    bool vo;
 
-    qw1w2 = xive_set_field32(TM_QW1W2_VO, qw1w2_prev, 0);
-    memcpy(&tctx->regs[TM_QW1_OS + TM_WORD2], &qw1w2, 4);
+    qw1w2 = xive_tctx_get_os_cam(tctx, &nvt_blk, &nvt_idx, &vo);
+
+    /* Invalidate CAM line */
+    qw1w2_new = xive_set_field32(TM_QW1W2_VO, qw1w2, 0);
+    xive_tctx_set_os_cam(tctx, qw1w2_new);
     return qw1w2;
 }
 
Signature · 1 line
2.21.0
Applied
#17 lore raw

Re: [PATCH for-5.0 v5 06/23] ppc/xive: Introduce OS CAM line helpers

Quoted 6 lines · Cédric Le Goater

The OS CAM line has a special encoding exploited by the HW. Provide helper routines to hide the details to the TIMA command handlers. This also clarifies the endianness of different variables : 'qw1w2' is big-endian and 'cam' is native.

Applied, thanks.

Quoted 61 lines
---
 hw/intc/xive.c | 41 ++++++++++++++++++++++++++++++++++++++---
 1 file changed, 38 insertions(+), 3 deletions(-)
hw/intc/xive.c +38 -3
index 177663d2b43e..42e9a11ef731 100644
@@ -337,14 +337,49 @@ static void xive_tm_set_os_pending(XiveTCTX *tctx, hwaddr offset,
     xive_tctx_notify(tctx, TM_QW1_OS);
 }
 
+static void xive_os_cam_decode(uint32_t cam, uint8_t *nvt_blk,
+                               uint32_t *nvt_idx, bool *vo)
+{
+    if (nvt_blk) {
+        *nvt_blk = xive_nvt_blk(cam);
+    }
+    if (nvt_idx) {
+        *nvt_idx = xive_nvt_idx(cam);
+    }
+    if (vo) {
+        *vo = !!(cam & TM_QW1W2_VO);
+    }
+}
+
+static uint32_t xive_tctx_get_os_cam(XiveTCTX *tctx, uint8_t *nvt_blk,
+                                     uint32_t *nvt_idx, bool *vo)
+{
+    uint32_t qw1w2 = xive_tctx_word2(&tctx->regs[TM_QW1_OS]);
+    uint32_t cam = be32_to_cpu(qw1w2);
+
+    xive_os_cam_decode(cam, nvt_blk, nvt_idx, vo);
+    return qw1w2;
+}
+
+static void xive_tctx_set_os_cam(XiveTCTX *tctx, uint32_t qw1w2)
+{
+    memcpy(&tctx->regs[TM_QW1_OS + TM_WORD2], &qw1w2, 4);
+}
+
 static uint64_t xive_tm_pull_os_ctx(XiveTCTX *tctx, hwaddr offset,
                                     unsigned size)
 {
-    uint32_t qw1w2_prev = xive_tctx_word2(&tctx->regs[TM_QW1_OS]);
     uint32_t qw1w2;
+    uint32_t qw1w2_new;
+    uint8_t nvt_blk;
+    uint32_t nvt_idx;
+    bool vo;
 
-    qw1w2 = xive_set_field32(TM_QW1W2_VO, qw1w2_prev, 0);
-    memcpy(&tctx->regs[TM_QW1_OS + TM_WORD2], &qw1w2, 4);
+    qw1w2 = xive_tctx_get_os_cam(tctx, &nvt_blk, &nvt_idx, &vo);
+
+    /* Invalidate CAM line */
+    qw1w2_new = xive_set_field32(TM_QW1W2_VO, qw1w2, 0);
+    xive_tctx_set_os_cam(tctx, qw1w2_new);
     return qw1w2;
 }
 
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#18 lore raw

PATCH v5 7/23 ppc/xive: Check V bit in TM_PULL_POOL_CTX

In reply to Cédric Le Goater

A context should be 'valid' when pulled from the thread interrupt context registers.

---
 hw/intc/xive.c | 5 +++++
 1 file changed, 5 insertions(+)
hw/intc/xive.c +5 -0
index 42e9a11ef731..511e1a936347 100644
@@ -377,6 +377,11 @@ static uint64_t xive_tm_pull_os_ctx(XiveTCTX *tctx, hwaddr offset,
 
     qw1w2 = xive_tctx_get_os_cam(tctx, &nvt_blk, &nvt_idx, &vo);
 
+    if (!vo) {
+        qemu_log_mask(LOG_GUEST_ERROR, "XIVE: pulling invalid NVT %x/%x !?\n",
+                      nvt_blk, nvt_idx);
+    }
+
     /* Invalidate CAM line */
     qw1w2_new = xive_set_field32(TM_QW1W2_VO, qw1w2, 0);
     xive_tctx_set_os_cam(tctx, qw1w2_new);
Signature · 1 line
2.21.0
Applied
#19 lore raw

Re: [PATCH for-5.0 v5 07/23] ppc/xive: Check V bit in TM_PULL_POOL_CTX

Quoted 4 lines · Cédric Le Goater

A context should be 'valid' when pulled from the thread interrupt context registers.

Applied, thanks.

Quoted 20 lines
---
 hw/intc/xive.c | 5 +++++
 1 file changed, 5 insertions(+)
hw/intc/xive.c +5 -0
index 42e9a11ef731..511e1a936347 100644
@@ -377,6 +377,11 @@ static uint64_t xive_tm_pull_os_ctx(XiveTCTX *tctx, hwaddr offset,
 
     qw1w2 = xive_tctx_get_os_cam(tctx, &nvt_blk, &nvt_idx, &vo);
 
+    if (!vo) {
+        qemu_log_mask(LOG_GUEST_ERROR, "XIVE: pulling invalid NVT %x/%x !?\n",
+                      nvt_blk, nvt_idx);
+    }
+
     /* Invalidate CAM line */
     qw1w2_new = xive_set_field32(TM_QW1W2_VO, qw1w2, 0);
     xive_tctx_set_os_cam(tctx, qw1w2_new);
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#20 lore raw

PATCH v5 8/23 ppc/xive: Introduce a XivePresenter interface

In reply to Cédric Le Goater

When the XIVE IVRE sub-engine (XiveRouter) looks for a Notification Virtual Target (NVT) to notify, it broadcasts a message on the PowerBUS to find an XIVE IVPE sub-engine (Presenter) with the NVT dispatched on one of its HW threads, and then forwards the notification if any response was received.

The current XIVE presenter model is sufficient for the pseries machine because it has a single interrupt controller device, but the PowerNV machine can have multiple chips each having its own interrupt controller. In this case, the XIVE presenter model is too simple and the CAM line matching should scan all chips of the system.

To start fixing this issue, we first extend the XIVE Router model with a new XivePresenter QOM interface representing the XIVE IVPE sub-engine. This interface exposes a 'match_nvt' handler which the sPAPR and PowerNV XIVE Router models will need to implement to perform the CAM line matching.

---
 include/hw/ppc/xive.h | 32 ++++++++++++++++++++++++++++++++
 hw/intc/xive.c        | 26 +++++++++++++++++---------
 2 files changed, 49 insertions(+), 9 deletions(-)
include/hw/ppc/xive.h +32 -0
index fa7adf87feb2..f9aa0fa0dac3 100644
@@ -367,6 +367,38 @@ int xive_router_write_nvt(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
 XiveTCTX *xive_router_get_tctx(XiveRouter *xrtr, CPUState *cs);
 void xive_router_notify(XiveNotifier *xn, uint32_t lisn);
 
+/*
+ * XIVE Presenter
+ */
+
+typedef struct XiveTCTXMatch {
+    XiveTCTX *tctx;
+    uint8_t ring;
+} XiveTCTXMatch;
+
+typedef struct XivePresenter XivePresenter;
+
+#define TYPE_XIVE_PRESENTER "xive-presenter"
+#define XIVE_PRESENTER(obj)                                     \
+    INTERFACE_CHECK(XivePresenter, (obj), TYPE_XIVE_PRESENTER)
+#define XIVE_PRESENTER_CLASS(klass)                                     \
+    OBJECT_CLASS_CHECK(XivePresenterClass, (klass), TYPE_XIVE_PRESENTER)
+#define XIVE_PRESENTER_GET_CLASS(obj)                                   \
+    OBJECT_GET_CLASS(XivePresenterClass, (obj), TYPE_XIVE_PRESENTER)
+
+typedef struct XivePresenterClass {
+    InterfaceClass parent;
+    int (*match_nvt)(XivePresenter *xptr, uint8_t format,
+                     uint8_t nvt_blk, uint32_t nvt_idx,
+                     bool cam_ignore, uint8_t priority,
+                     uint32_t logic_serv, XiveTCTXMatch *match);
+} XivePresenterClass;
+
+int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
+                              uint8_t format,
+                              uint8_t nvt_blk, uint32_t nvt_idx,
+                              bool cam_ignore, uint32_t logic_serv);
+
 /*
  * XIVE END ESBs
  */
hw/intc/xive.c +17 -9
index 511e1a936347..344bb3f3bc4b 100644
@@ -1363,9 +1363,10 @@ static uint32_t xive_tctx_hw_cam_line(XiveTCTX *tctx)
 /*
  * The thread context register words are in big-endian format.
  */
-static int xive_presenter_tctx_match(XiveTCTX *tctx, uint8_t format,
-                                     uint8_t nvt_blk, uint32_t nvt_idx,
-                                     bool cam_ignore, uint32_t logic_serv)
+int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
+                              uint8_t format,
+                              uint8_t nvt_blk, uint32_t nvt_idx,
+                              bool cam_ignore, uint32_t logic_serv)
 {
     uint32_t cam = xive_nvt_cam_line(nvt_blk, nvt_idx);
     uint32_t qw3w2 = xive_tctx_word2(&tctx->regs[TM_QW3_HV_PHYS]);
@@ -1422,11 +1423,6 @@ static int xive_presenter_tctx_match(XiveTCTX *tctx, uint8_t format,
     return -1;
 }
 
-typedef struct XiveTCTXMatch {
-    XiveTCTX *tctx;
-    uint8_t ring;
-} XiveTCTXMatch;
-
 static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
                                  uint8_t nvt_blk, uint32_t nvt_idx,
                                  bool cam_ignore, uint8_t priority,
@@ -1460,7 +1456,8 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
          * Check the thread context CAM lines and record matches. We
          * will handle CPU exception delivery later
          */
-        ring = xive_presenter_tctx_match(tctx, format, nvt_blk, nvt_idx,
+        ring = xive_presenter_tctx_match(XIVE_PRESENTER(xrtr), tctx, format,
+                                         nvt_blk, nvt_idx,
                                          cam_ignore, logic_serv);
         /*
          * Save the context and follow on to catch duplicates, that we
@@ -1754,6 +1751,7 @@ static const TypeInfo xive_router_info = {
     .class_init    = xive_router_class_init,
     .interfaces    = (InterfaceInfo[]) {
         { TYPE_XIVE_NOTIFIER },
+        { TYPE_XIVE_PRESENTER },
         { }
     }
 };
@@ -1923,10 +1921,20 @@ static const TypeInfo xive_notifier_info = {
     .class_size = sizeof(XiveNotifierClass),
 };
 
+/*
+ * XIVE Presenter
+ */
+static const TypeInfo xive_presenter_info = {
+    .name = TYPE_XIVE_PRESENTER,
+    .parent = TYPE_INTERFACE,
+    .class_size = sizeof(XivePresenterClass),
+};
+
 static void xive_register_types(void)
 {
     type_register_static(&xive_source_info);
     type_register_static(&xive_notifier_info);
+    type_register_static(&xive_presenter_info);
     type_register_static(&xive_router_info);
     type_register_static(&xive_end_source_info);
     type_register_static(&xive_tctx_info);
Signature · 1 line
2.21.0
Reviewed-by on v5 8/23
#21 lore raw

Re: [PATCH for-5.0 v5 08/23] ppc/xive: Introduce a XivePresenter interface

Quoted 31 lines · On Fri, 15 Nov 2019 17:24:21 +0100 Cédric Le Goater <[email protected]> wrote:

When the XIVE IVRE sub-engine (XiveRouter) looks for a Notification Virtual Target (NVT) to notify, it broadcasts a message on the PowerBUS to find an XIVE IVPE sub-engine (Presenter) with the NVT dispatched on one of its HW threads, and then forwards the notification if any response was received.

The current XIVE presenter model is sufficient for the pseries machine because it has a single interrupt controller device, but the PowerNV machine can have multiple chips each having its own interrupt controller. In this case, the XIVE presenter model is too simple and the CAM line matching should scan all chips of the system.

To start fixing this issue, we first extend the XIVE Router model with a new XivePresenter QOM interface representing the XIVE IVPE sub-engine. This interface exposes a 'match_nvt' handler which the sPAPR and PowerNV XIVE Router models will need to implement to perform the CAM line matching.

---
 include/hw/ppc/xive.h | 32 ++++++++++++++++++++++++++++++++
 hw/intc/xive.c        | 26 +++++++++++++++++---------
 2 files changed, 49 insertions(+), 9 deletions(-)
include/hw/ppc/xive.h +0 -0
index fa7adf87feb2..f9aa0fa0dac3 100644
@@ -367,6 +367,38 @@ int xive_router_write_nvt(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
 XiveTCTX *xive_router_get_tctx(XiveRouter *xrtr, CPUState *cs);
 void xive_router_notify(XiveNotifier *xn, uint32_t lisn);

include/hw/ppc/xive.h int xive_router_write_nvt(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx, · line 378

Quoted 10 lines
include/hw/ppc/xive.h +9 -0
 
+/*
+ * XIVE Presenter
+ */
+
+typedef struct XiveTCTXMatch {
+    XiveTCTX *tctx;
+    uint8_t ring;
+} XiveTCTXMatch;
+

We recently decided to follow a strict CamelCase naming scheme in spapr. We should probably do the same in other areas. Can you please make this XiveTctxMatch ? I can send follow-up patches to convert the existing code later on, ie. after this series is merged at last :)

The patch looks good anyway so:

Quoted 95 lines
include/hw/ppc/xive.h +23 -0
+typedef struct XivePresenter XivePresenter;
+
+#define TYPE_XIVE_PRESENTER "xive-presenter"
+#define XIVE_PRESENTER(obj)                                     \
+    INTERFACE_CHECK(XivePresenter, (obj), TYPE_XIVE_PRESENTER)
+#define XIVE_PRESENTER_CLASS(klass)                                     \
+    OBJECT_CLASS_CHECK(XivePresenterClass, (klass), TYPE_XIVE_PRESENTER)
+#define XIVE_PRESENTER_GET_CLASS(obj)                                   \
+    OBJECT_GET_CLASS(XivePresenterClass, (obj), TYPE_XIVE_PRESENTER)
+
+typedef struct XivePresenterClass {
+    InterfaceClass parent;
+    int (*match_nvt)(XivePresenter *xptr, uint8_t format,
+                     uint8_t nvt_blk, uint32_t nvt_idx,
+                     bool cam_ignore, uint8_t priority,
+                     uint32_t logic_serv, XiveTCTXMatch *match);
+} XivePresenterClass;
+
+int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
+                              uint8_t format,
+                              uint8_t nvt_blk, uint32_t nvt_idx,
+                              bool cam_ignore, uint32_t logic_serv);
+
 /*
  * XIVE END ESBs
  */
hw/intc/xive.c +17 -9
index 511e1a936347..344bb3f3bc4b 100644
@@ -1363,9 +1363,10 @@ static uint32_t xive_tctx_hw_cam_line(XiveTCTX *tctx)
 /*
  * The thread context register words are in big-endian format.
  */
-static int xive_presenter_tctx_match(XiveTCTX *tctx, uint8_t format,
-                                     uint8_t nvt_blk, uint32_t nvt_idx,
-                                     bool cam_ignore, uint32_t logic_serv)
+int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
+                              uint8_t format,
+                              uint8_t nvt_blk, uint32_t nvt_idx,
+                              bool cam_ignore, uint32_t logic_serv)
 {
     uint32_t cam = xive_nvt_cam_line(nvt_blk, nvt_idx);
     uint32_t qw3w2 = xive_tctx_word2(&tctx->regs[TM_QW3_HV_PHYS]);
@@ -1422,11 +1423,6 @@ static int xive_presenter_tctx_match(XiveTCTX *tctx, uint8_t format,
     return -1;
 }
 
-typedef struct XiveTCTXMatch {
-    XiveTCTX *tctx;
-    uint8_t ring;
-} XiveTCTXMatch;
-
 static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
                                  uint8_t nvt_blk, uint32_t nvt_idx,
                                  bool cam_ignore, uint8_t priority,
@@ -1460,7 +1456,8 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
          * Check the thread context CAM lines and record matches. We
          * will handle CPU exception delivery later
          */
-        ring = xive_presenter_tctx_match(tctx, format, nvt_blk, nvt_idx,
+        ring = xive_presenter_tctx_match(XIVE_PRESENTER(xrtr), tctx, format,
+                                         nvt_blk, nvt_idx,
                                          cam_ignore, logic_serv);
         /*
          * Save the context and follow on to catch duplicates, that we
@@ -1754,6 +1751,7 @@ static const TypeInfo xive_router_info = {
     .class_init    = xive_router_class_init,
     .interfaces    = (InterfaceInfo[]) {
         { TYPE_XIVE_NOTIFIER },
+        { TYPE_XIVE_PRESENTER },
         { }
     }
 };
@@ -1923,10 +1921,20 @@ static const TypeInfo xive_notifier_info = {
     .class_size = sizeof(XiveNotifierClass),
 };
 
+/*
+ * XIVE Presenter
+ */
+static const TypeInfo xive_presenter_info = {
+    .name = TYPE_XIVE_PRESENTER,
+    .parent = TYPE_INTERFACE,
+    .class_size = sizeof(XivePresenterClass),
+};
+
 static void xive_register_types(void)
 {
     type_register_static(&xive_source_info);
     type_register_static(&xive_notifier_info);
+    type_register_static(&xive_presenter_info);
     type_register_static(&xive_router_info);
     type_register_static(&xive_end_source_info);
     type_register_static(&xive_tctx_info);
#22 lore raw

PATCH v5 9/23 ppc/xive: Implement the XivePresenter interface

In reply to Cédric Le Goater

Each XIVE Router model, sPAPR and PowerNV, now implements the 'match_nvt' handler of the XivePresenter QOM interface. This is simply moving code and taking into account the new API.

To be noted that the xive_router_get_tctx() helper is not used anymore when doing CAM matching and will be removed later on after other changes.

The XIVE presenter model is still too simple for the PowerNV machine and the CAM matching algo is not correct on multichip system. Subsequent patches will introduce more changes to scan all chips of the system.

---
 hw/intc/pnv_xive.c   | 41 +++++++++++++++++++++++++++++++++++
 hw/intc/spapr_xive.c | 49 ++++++++++++++++++++++++++++++++++++++++++
 hw/intc/xive.c       | 51 ++++++--------------------------------------
 3 files changed, 97 insertions(+), 44 deletions(-)
hw/intc/pnv_xive.c +41 -0
index a394331ddd6a..087cbfbaad48 100644
@@ -372,6 +372,45 @@ static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx,
     return pnv_xive_vst_read(xive, VST_TSEL_IVT, blk, idx, eas);
 }
 
+static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
+                              uint8_t nvt_blk, uint32_t nvt_idx,
+                              bool cam_ignore, uint8_t priority,
+                              uint32_t logic_serv, XiveTCTXMatch *match)
+{
+    CPUState *cs;
+    int count = 0;
+
+    CPU_FOREACH(cs) {
+        PowerPCCPU *cpu = POWERPC_CPU(cs);
+        XiveTCTX *tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
+        int ring;
+
+        /*
+         * Check the thread context CAM lines and record matches.
+         */
+        ring = xive_presenter_tctx_match(xptr, tctx, format, nvt_blk, nvt_idx,
+                                         cam_ignore, logic_serv);
+        /*
+         * Save the context and follow on to catch duplicates, that we
+         * don't support yet.
+         */
+        if (ring != -1) {
+            if (match->tctx) {
+                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a "
+                              "thread context NVT %x/%x\n",
+                              nvt_blk, nvt_idx);
+                return -1;
+            }
+
+            match->ring = ring;
+            match->tctx = tctx;
+            count++;
+        }
+    }
+
+    return count;
+}
+
 static XiveTCTX *pnv_xive_get_tctx(XiveRouter *xrtr, CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
@@ -1810,6 +1849,7 @@ static void pnv_xive_class_init(ObjectClass *klass, void *data)
     PnvXScomInterfaceClass *xdc = PNV_XSCOM_INTERFACE_CLASS(klass);
     XiveRouterClass *xrc = XIVE_ROUTER_CLASS(klass);
     XiveNotifierClass *xnc = XIVE_NOTIFIER_CLASS(klass);
+    XivePresenterClass *xpc = XIVE_PRESENTER_CLASS(klass);
 
     xdc->dt_xscom = pnv_xive_dt_xscom;
 
@@ -1825,6 +1865,7 @@ static void pnv_xive_class_init(ObjectClass *klass, void *data)
     xrc->get_tctx = pnv_xive_get_tctx;
 
     xnc->notify = pnv_xive_notify;
+    xpc->match_nvt  = pnv_xive_match_nvt;
 };
 
 static const TypeInfo pnv_xive_info = {
hw/intc/spapr_xive.c +49 -0

67 lines not on this page · raw message

hw/intc/xive.c +7 -44

58 lines not on this page · raw message

Signature · 1 line
2.21.0
Reviewed-by on v5 9/23
#23 lore raw

Re: [PATCH for-5.0 v5 09/23] ppc/xive: Implement the XivePresenter interface

Quoted 29 lines · On Fri, 15 Nov 2019 17:24:22 +0100 Cédric Le Goater <[email protected]> wrote:

Each XIVE Router model, sPAPR and PowerNV, now implements the 'match_nvt' handler of the XivePresenter QOM interface. This is simply moving code and taking into account the new API.

To be noted that the xive_router_get_tctx() helper is not used anymore when doing CAM matching and will be removed later on after other changes.

The XIVE presenter model is still too simple for the PowerNV machine and the CAM matching algo is not correct on multichip system. Subsequent patches will introduce more changes to scan all chips of the system.

---
 hw/intc/pnv_xive.c   | 41 +++++++++++++++++++++++++++++++++++
 hw/intc/spapr_xive.c | 49 ++++++++++++++++++++++++++++++++++++++++++
 hw/intc/xive.c       | 51 ++++++--------------------------------------
 3 files changed, 97 insertions(+), 44 deletions(-)
hw/intc/pnv_xive.c +3 -0
index a394331ddd6a..087cbfbaad48 100644
@@ -372,6 +372,45 @@ static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx,
     return pnv_xive_vst_read(xive, VST_TSEL_IVT, blk, idx, eas);
 }
 
+static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
+                              uint8_t nvt_blk, uint32_t nvt_idx,
+                              bool cam_ignore, uint8_t priority,

hw/intc/pnv_xive.c static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx, · line 387

Quoted 10 lines
hw/intc/pnv_xive.c +10 -0
+                              uint32_t logic_serv, XiveTCTXMatch *match)
+{
+    CPUState *cs;
+    int count = 0;
+
+    CPU_FOREACH(cs) {
+        PowerPCCPU *cpu = POWERPC_CPU(cs);
+        XiveTCTX *tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
+        int ring;
+

I guess it's ok not to check tctx here because the powernv machine type doesn't support cpu hotplug.

LGTM despite the non-strict CamelCase type :)

Quoted 182 lines
hw/intc/pnv_xive.c +28 -0
+        /*
+         * Check the thread context CAM lines and record matches.
+         */
+        ring = xive_presenter_tctx_match(xptr, tctx, format, nvt_blk, nvt_idx,
+                                         cam_ignore, logic_serv);
+        /*
+         * Save the context and follow on to catch duplicates, that we
+         * don't support yet.
+         */
+        if (ring != -1) {
+            if (match->tctx) {
+                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a "
+                              "thread context NVT %x/%x\n",
+                              nvt_blk, nvt_idx);
+                return -1;
+            }
+
+            match->ring = ring;
+            match->tctx = tctx;
+            count++;
+        }
+    }
+
+    return count;
+}
+
 static XiveTCTX *pnv_xive_get_tctx(XiveRouter *xrtr, CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
@@ -1810,6 +1849,7 @@ static void pnv_xive_class_init(ObjectClass *klass, void *data)
     PnvXScomInterfaceClass *xdc = PNV_XSCOM_INTERFACE_CLASS(klass);
     XiveRouterClass *xrc = XIVE_ROUTER_CLASS(klass);
     XiveNotifierClass *xnc = XIVE_NOTIFIER_CLASS(klass);
+    XivePresenterClass *xpc = XIVE_PRESENTER_CLASS(klass);
 
     xdc->dt_xscom = pnv_xive_dt_xscom;
 
@@ -1825,6 +1865,7 @@ static void pnv_xive_class_init(ObjectClass *klass, void *data)
     xrc->get_tctx = pnv_xive_get_tctx;
 
     xnc->notify = pnv_xive_notify;
+    xpc->match_nvt  = pnv_xive_match_nvt;
 };
 
 static const TypeInfo pnv_xive_info = {
hw/intc/spapr_xive.c +49 -0
index 729246e906c9..bb3b2dfdb77f 100644
@@ -405,6 +405,52 @@ static XiveTCTX *spapr_xive_get_tctx(XiveRouter *xrtr, CPUState *cs)
     return spapr_cpu_state(cpu)->tctx;
 }
 
+static int spapr_xive_match_nvt(XivePresenter *xptr, uint8_t format,
+                                uint8_t nvt_blk, uint32_t nvt_idx,
+                                bool cam_ignore, uint8_t priority,
+                                uint32_t logic_serv, XiveTCTXMatch *match)
+{
+    CPUState *cs;
+    int count = 0;
+
+    CPU_FOREACH(cs) {
+        PowerPCCPU *cpu = POWERPC_CPU(cs);
+        XiveTCTX *tctx = spapr_cpu_state(cpu)->tctx;
+        int ring;
+
+        /*
+         * Skip partially initialized vCPUs. This can happen when
+         * vCPUs are hotplugged.
+         */
+        if (!tctx) {
+            continue;
+        }
+
+        /*
+         * Check the thread context CAM lines and record matches.
+         */
+        ring = xive_presenter_tctx_match(xptr, tctx, format, nvt_blk, nvt_idx,
+                                         cam_ignore, logic_serv);
+        /*
+         * Save the matching thread interrupt context and follow on to
+         * check for duplicates which are invalid.
+         */
+        if (ring != -1) {
+            if (match->tctx) {
+                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a thread "
+                              "context NVT %x/%x\n", nvt_blk, nvt_idx);
+                return -1;
+            }
+
+            match->ring = ring;
+            match->tctx = tctx;
+            count++;
+        }
+    }
+
+    return count;
+}
+
 static const VMStateDescription vmstate_spapr_xive_end = {
     .name = TYPE_SPAPR_XIVE "/end",
     .version_id = 1,
@@ -684,6 +730,7 @@ static void spapr_xive_class_init(ObjectClass *klass, void *data)
     DeviceClass *dc = DEVICE_CLASS(klass);
     XiveRouterClass *xrc = XIVE_ROUTER_CLASS(klass);
     SpaprInterruptControllerClass *sicc = SPAPR_INTC_CLASS(klass);
+    XivePresenterClass *xpc = XIVE_PRESENTER_CLASS(klass);
 
     dc->desc    = "sPAPR XIVE Interrupt Controller";
     dc->props   = spapr_xive_properties;
@@ -708,6 +755,8 @@ static void spapr_xive_class_init(ObjectClass *klass, void *data)
     sicc->print_info = spapr_xive_print_info;
     sicc->dt = spapr_xive_dt;
     sicc->post_load = spapr_xive_post_load;
+
+    xpc->match_nvt  = spapr_xive_match_nvt;
 }
 
 static const TypeInfo spapr_xive_info = {
hw/intc/xive.c +7 -44
index 344bb3f3bc4b..da6196ca958f 100644
@@ -1428,51 +1428,14 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
                                  bool cam_ignore, uint8_t priority,
                                  uint32_t logic_serv, XiveTCTXMatch *match)
 {
-    CPUState *cs;
+    XivePresenter *xptr = XIVE_PRESENTER(xrtr);
+    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
+    int count;
 
-    /*
-     * TODO (PowerNV): handle chip_id overwrite of block field for
-     * hardwired CAM compares
-     */
-
-    CPU_FOREACH(cs) {
-        XiveTCTX *tctx = xive_router_get_tctx(xrtr, cs);
-        int ring;
-
-        /*
-         * Skip partially initialized vCPUs. This can happen when
-         * vCPUs are hotplugged.
-         */
-        if (!tctx) {
-            continue;
-        }
-
-        /*
-         * HW checks that the CPU is enabled in the Physical Thread
-         * Enable Register (PTER).
-         */
-
-        /*
-         * Check the thread context CAM lines and record matches. We
-         * will handle CPU exception delivery later
-         */
-        ring = xive_presenter_tctx_match(XIVE_PRESENTER(xrtr), tctx, format,
-                                         nvt_blk, nvt_idx,
-                                         cam_ignore, logic_serv);
-        /*
-         * Save the context and follow on to catch duplicates, that we
-         * don't support yet.
-         */
-        if (ring != -1) {
-            if (match->tctx) {
-                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a thread "
-                              "context NVT %x/%x\n", nvt_blk, nvt_idx);
-                return false;
-            }
-
-            match->ring = ring;
-            match->tctx = tctx;
-        }
+    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
+                           priority, logic_serv, match);
+    if (count < 0) {
+        return false;
     }
 
     if (!match->tctx) {
#24 lore raw

Re: [PATCH for-5.0 v5 09/23] ppc/xive: Implement the XivePresenter interface

Quoted 29 lines · Greg Kurz
Quoted 26 lines · On Fri, 15 Nov 2019 17:24:22 +0100 Cédric Le Goater <[email protected]> wrote:

Each XIVE Router model, sPAPR and PowerNV, now implements the 'match_nvt' handler of the XivePresenter QOM interface. This is simply moving code and taking into account the new API.

To be noted that the xive_router_get_tctx() helper is not used anymore when doing CAM matching and will be removed later on after other changes.

The XIVE presenter model is still too simple for the PowerNV machine and the CAM matching algo is not correct on multichip system. Subsequent patches will introduce more changes to scan all chips of the system.

---
 hw/intc/pnv_xive.c   | 41 +++++++++++++++++++++++++++++++++++
 hw/intc/spapr_xive.c | 49 ++++++++++++++++++++++++++++++++++++++++++
 hw/intc/xive.c       | 51 ++++++--------------------------------------
 3 files changed, 97 insertions(+), 44 deletions(-)
hw/intc/pnv_xive.c +0 -0
index a394331ddd6a..087cbfbaad48 100644
@@ -372,6 +372,45 @@ static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx,
     return pnv_xive_vst_read(xive, VST_TSEL_IVT, blk, idx, eas);
 }
 
Quoted 16 lines

hw/intc/pnv_xive.c static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx, · line 387

Quoted 13 lines
hw/intc/pnv_xive.c +13 -0
+static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
+                              uint8_t nvt_blk, uint32_t nvt_idx,
+                              bool cam_ignore, uint8_t priority,
+                              uint32_t logic_serv, XiveTCTXMatch *match)
+{
+    CPUState *cs;
+    int count = 0;
+
+    CPU_FOREACH(cs) {
+        PowerPCCPU *cpu = POWERPC_CPU(cs);
+        XiveTCTX *tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
+        int ring;
+

I guess it's ok not to check tctx here because the powernv machine type doesn't support cpu hotplug.

patch 10 and patch 11 add some more changes in that area.

C.

Quoted 188 lines

LGTM despite the non-strict CamelCase type :)

Quoted 182 lines
hw/intc/pnv_xive.c +28 -0
+        /*
+         * Check the thread context CAM lines and record matches.
+         */
+        ring = xive_presenter_tctx_match(xptr, tctx, format, nvt_blk, nvt_idx,
+                                         cam_ignore, logic_serv);
+        /*
+         * Save the context and follow on to catch duplicates, that we
+         * don't support yet.
+         */
+        if (ring != -1) {
+            if (match->tctx) {
+                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a "
+                              "thread context NVT %x/%x\n",
+                              nvt_blk, nvt_idx);
+                return -1;
+            }
+
+            match->ring = ring;
+            match->tctx = tctx;
+            count++;
+        }
+    }
+
+    return count;
+}
+
 static XiveTCTX *pnv_xive_get_tctx(XiveRouter *xrtr, CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
@@ -1810,6 +1849,7 @@ static void pnv_xive_class_init(ObjectClass *klass, void *data)
     PnvXScomInterfaceClass *xdc = PNV_XSCOM_INTERFACE_CLASS(klass);
     XiveRouterClass *xrc = XIVE_ROUTER_CLASS(klass);
     XiveNotifierClass *xnc = XIVE_NOTIFIER_CLASS(klass);
+    XivePresenterClass *xpc = XIVE_PRESENTER_CLASS(klass);
 
     xdc->dt_xscom = pnv_xive_dt_xscom;
 
@@ -1825,6 +1865,7 @@ static void pnv_xive_class_init(ObjectClass *klass, void *data)
     xrc->get_tctx = pnv_xive_get_tctx;
 
     xnc->notify = pnv_xive_notify;
+    xpc->match_nvt  = pnv_xive_match_nvt;
 };
 
 static const TypeInfo pnv_xive_info = {
hw/intc/spapr_xive.c +49 -0
index 729246e906c9..bb3b2dfdb77f 100644
@@ -405,6 +405,52 @@ static XiveTCTX *spapr_xive_get_tctx(XiveRouter *xrtr, CPUState *cs)
     return spapr_cpu_state(cpu)->tctx;
 }
 
+static int spapr_xive_match_nvt(XivePresenter *xptr, uint8_t format,
+                                uint8_t nvt_blk, uint32_t nvt_idx,
+                                bool cam_ignore, uint8_t priority,
+                                uint32_t logic_serv, XiveTCTXMatch *match)
+{
+    CPUState *cs;
+    int count = 0;
+
+    CPU_FOREACH(cs) {
+        PowerPCCPU *cpu = POWERPC_CPU(cs);
+        XiveTCTX *tctx = spapr_cpu_state(cpu)->tctx;
+        int ring;
+
+        /*
+         * Skip partially initialized vCPUs. This can happen when
+         * vCPUs are hotplugged.
+         */
+        if (!tctx) {
+            continue;
+        }
+
+        /*
+         * Check the thread context CAM lines and record matches.
+         */
+        ring = xive_presenter_tctx_match(xptr, tctx, format, nvt_blk, nvt_idx,
+                                         cam_ignore, logic_serv);
+        /*
+         * Save the matching thread interrupt context and follow on to
+         * check for duplicates which are invalid.
+         */
+        if (ring != -1) {
+            if (match->tctx) {
+                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a thread "
+                              "context NVT %x/%x\n", nvt_blk, nvt_idx);
+                return -1;
+            }
+
+            match->ring = ring;
+            match->tctx = tctx;
+            count++;
+        }
+    }
+
+    return count;
+}
+
 static const VMStateDescription vmstate_spapr_xive_end = {
     .name = TYPE_SPAPR_XIVE "/end",
     .version_id = 1,
@@ -684,6 +730,7 @@ static void spapr_xive_class_init(ObjectClass *klass, void *data)
     DeviceClass *dc = DEVICE_CLASS(klass);
     XiveRouterClass *xrc = XIVE_ROUTER_CLASS(klass);
     SpaprInterruptControllerClass *sicc = SPAPR_INTC_CLASS(klass);
+    XivePresenterClass *xpc = XIVE_PRESENTER_CLASS(klass);
 
     dc->desc    = "sPAPR XIVE Interrupt Controller";
     dc->props   = spapr_xive_properties;
@@ -708,6 +755,8 @@ static void spapr_xive_class_init(ObjectClass *klass, void *data)
     sicc->print_info = spapr_xive_print_info;
     sicc->dt = spapr_xive_dt;
     sicc->post_load = spapr_xive_post_load;
+
+    xpc->match_nvt  = spapr_xive_match_nvt;
 }
 
 static const TypeInfo spapr_xive_info = {
hw/intc/xive.c +7 -44
index 344bb3f3bc4b..da6196ca958f 100644
@@ -1428,51 +1428,14 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
                                  bool cam_ignore, uint8_t priority,
                                  uint32_t logic_serv, XiveTCTXMatch *match)
 {
-    CPUState *cs;
+    XivePresenter *xptr = XIVE_PRESENTER(xrtr);
+    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
+    int count;
 
-    /*
-     * TODO (PowerNV): handle chip_id overwrite of block field for
-     * hardwired CAM compares
-     */
-
-    CPU_FOREACH(cs) {
-        XiveTCTX *tctx = xive_router_get_tctx(xrtr, cs);
-        int ring;
-
-        /*
-         * Skip partially initialized vCPUs. This can happen when
-         * vCPUs are hotplugged.
-         */
-        if (!tctx) {
-            continue;
-        }
-
-        /*
-         * HW checks that the CPU is enabled in the Physical Thread
-         * Enable Register (PTER).
-         */
-
-        /*
-         * Check the thread context CAM lines and record matches. We
-         * will handle CPU exception delivery later
-         */
-        ring = xive_presenter_tctx_match(XIVE_PRESENTER(xrtr), tctx, format,
-                                         nvt_blk, nvt_idx,
-                                         cam_ignore, logic_serv);
-        /*
-         * Save the context and follow on to catch duplicates, that we
-         * don't support yet.
-         */
-        if (ring != -1) {
-            if (match->tctx) {
-                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a thread "
-                              "context NVT %x/%x\n", nvt_blk, nvt_idx);
-                return false;
-            }
-
-            match->ring = ring;
-            match->tctx = tctx;
-        }
+    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
+                           priority, logic_serv, match);
+    if (count < 0) {
+        return false;
     }
 
     if (!match->tctx) {
#25 lore raw

PATCH v5 10/23 ppc/pnv: Loop on the threads of the chip to find a matching NVT

In reply to Cédric Le Goater

CPU_FOREACH() loops on all the CPUs of the machine which is incorrect. Each XIVE Presenter should scan only the HW threads of the chip it belongs to.

---
 include/hw/ppc/pnv.h |  2 ++
 hw/intc/pnv_xive.c   | 63 ++++++++++++++++++++++++++------------------
 hw/ppc/pnv.c         |  2 +-
 3 files changed, 40 insertions(+), 27 deletions(-)
include/hw/ppc/pnv.h +2 -0
index 07c56c05ad30..58f4dcc0b71d 100644
@@ -150,6 +150,8 @@ typedef struct PnvChipClass {
  */
 #define PNV_CHIP_HWID(i) ((((i) & 0x3e) << 3) | ((i) & 0x1))
 
+const char *pnv_chip_core_typename(const PnvChip *chip);
+
 /*
  * Converts back a HW chip id to an index. This is useful to calculate
  * the MMIO addresses of some controllers which depend on the chip id.
hw/intc/pnv_xive.c +37 -26

71 lines not on this page · raw message

hw/ppc/pnv.c +1 -1
index 57f924ba0466..94c9f536413f 100644
@@ -64,7 +64,7 @@
 #define INITRD_LOAD_ADDR        0x60000000
 #define INITRD_MAX_SIZE         (256 * MiB)
 
-static const char *pnv_chip_core_typename(const PnvChip *o)
+const char *pnv_chip_core_typename(const PnvChip *o)
 {
     const char *chip_type = object_class_get_name(object_get_class(OBJECT(o)));
     int len = strlen(chip_type) - strlen(PNV_CHIP_TYPE_SUFFIX);
Signature · 1 line
2.21.0
Reviewed-by on v5 10/23
#26 lore raw

Re: [PATCH for-5.0 v5 10/23] ppc/pnv: Loop on the threads of the chip to find a matching NVT

Quoted 29 lines · On Fri, 15 Nov 2019 17:24:23 +0100 Cédric Le Goater <[email protected]> wrote:

CPU_FOREACH() loops on all the CPUs of the machine which is incorrect. Each XIVE Presenter should scan only the HW threads of the chip it belongs to.

---
 include/hw/ppc/pnv.h |  2 ++
 hw/intc/pnv_xive.c   | 63 ++++++++++++++++++++++++++------------------
 hw/ppc/pnv.c         |  2 +-
 3 files changed, 40 insertions(+), 27 deletions(-)
include/hw/ppc/pnv.h +2 -0
index 07c56c05ad30..58f4dcc0b71d 100644
@@ -150,6 +150,8 @@ typedef struct PnvChipClass {
  */
 #define PNV_CHIP_HWID(i) ((((i) & 0x3e) << 3) | ((i) & 0x1))
 
+const char *pnv_chip_core_typename(const PnvChip *chip);
+
 /*
  * Converts back a HW chip id to an index. This is useful to calculate
  * the MMIO addresses of some controllers which depend on the chip id.
hw/intc/pnv_xive.c +0 -0
index 087cbfbaad48..71ca4961b6b1 100644

hw/intc/pnv_xive.c static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format, · line 383

Quoted 8 lines
hw/intc/pnv_xive.c +4 -1
@@ -377,34 +377,45 @@ static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
                               bool cam_ignore, uint8_t priority,
                               uint32_t logic_serv, XiveTCTXMatch *match)
 {
-    CPUState *cs;
+    PnvXive *xive = PNV_XIVE(xptr);
+    PnvChip *chip = xive->chip;
+    const char *typename = pnv_chip_core_typename(chip);
+    size_t typesize = object_type_get_instance_size(typename);

Ahh the same boiler plate is needed again because the cores are stored in an void *cores array in PnvChip... which is an ugly and fragile pattern we got rid of in spapr (commit 94ad93bd976841). We probably don't have the same risk here because the powernv machine doesn't support hot-unplug, but this is still ugly anyway.

I certainly don't want to hold this series any longer for the sake of Cedric's health :) but I'll be glad to turn the void *cores array into a PnvCore **cores as a follow-up cleanup.

Rest LGTM.

Quoted 76 lines
hw/intc/pnv_xive.c +33 -25
     int count = 0;
-
-    CPU_FOREACH(cs) {
-        PowerPCCPU *cpu = POWERPC_CPU(cs);
-        XiveTCTX *tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
-        int ring;
-
-        /*
-         * Check the thread context CAM lines and record matches.
-         */
-        ring = xive_presenter_tctx_match(xptr, tctx, format, nvt_blk, nvt_idx,
-                                         cam_ignore, logic_serv);
-        /*
-         * Save the context and follow on to catch duplicates, that we
-         * don't support yet.
-         */
-        if (ring != -1) {
-            if (match->tctx) {
-                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a "
-                              "thread context NVT %x/%x\n",
-                              nvt_blk, nvt_idx);
-                return -1;
+    int i, j;
+
+    for (i = 0; i < chip->nr_cores; i++) {
+        PnvCore *pc = PNV_CORE(chip->cores + i * typesize);
+        CPUCore *cc = CPU_CORE(pc);
+
+        for (j = 0; j < cc->nr_threads; j++) {
+            PowerPCCPU *cpu = pc->threads[j];
+            XiveTCTX *tctx;
+            int ring;
+
+            tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
+
+            /*
+             * Check the thread context CAM lines and record matches.
+             */
+            ring = xive_presenter_tctx_match(xptr, tctx, format, nvt_blk,
+                                             nvt_idx, cam_ignore, logic_serv);
+            /*
+             * Save the context and follow on to catch duplicates, that we
+             * don't support yet.
+             */
+            if (ring != -1) {
+                if (match->tctx) {
+                    qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a "
+                                  "thread context NVT %x/%x\n",
+                                  nvt_blk, nvt_idx);
+                    return -1;
+                }
+
+                match->ring = ring;
+                match->tctx = tctx;
+                count++;
             }
-
-            match->ring = ring;
-            match->tctx = tctx;
-            count++;
         }
     }
 
hw/ppc/pnv.c +1 -1
index 57f924ba0466..94c9f536413f 100644
@@ -64,7 +64,7 @@
 #define INITRD_LOAD_ADDR        0x60000000
 #define INITRD_MAX_SIZE         (256 * MiB)
 
-static const char *pnv_chip_core_typename(const PnvChip *o)
+const char *pnv_chip_core_typename(const PnvChip *o)
 {
     const char *chip_type = object_class_get_name(object_get_class(OBJECT(o)));
     int len = strlen(chip_type) - strlen(PNV_CHIP_TYPE_SUFFIX);
#27 lore raw

PATCH v5 11/23 ppc/pnv: Introduce a pnv_xive_is_cpu_enabled() helper

In reply to Cédric Le Goater

and use this helper to exclude CPUs which are not enabled in the XIVE controller.

---
 hw/intc/pnv_xive.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
hw/intc/pnv_xive.c +18 -0
index 71ca4961b6b1..4c8c6e51c20f 100644
@@ -372,6 +372,20 @@ static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx,
     return pnv_xive_vst_read(xive, VST_TSEL_IVT, blk, idx, eas);
 }
 
+static int cpu_pir(PowerPCCPU *cpu)
+{
+    CPUPPCState *env = &cpu->env;
+    return env->spr_cb[SPR_PIR].default_value;
+}
+
+static bool pnv_xive_is_cpu_enabled(PnvXive *xive, PowerPCCPU *cpu)
+{
+    int pir = cpu_pir(cpu);
+    int thrd_id = pir & 0x7f;
+
+    return xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(thrd_id);
+}
+
 static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
                               uint8_t nvt_blk, uint32_t nvt_idx,
                               bool cam_ignore, uint8_t priority,
@@ -393,6 +407,10 @@ static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
             XiveTCTX *tctx;
             int ring;
 
+            if (!pnv_xive_is_cpu_enabled(xive, cpu)) {
+                continue;
+            }
+
             tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
 
             /*
Signature · 1 line
2.21.0
#28 lore raw

Re: [PATCH for-5.0 v5 11/23] ppc/pnv: Introduce a pnv_xive_is_cpu_enabled() helper

Quoted 18 lines · On Fri, 15 Nov 2019 17:24:24 +0100 Cédric Le Goater <[email protected]> wrote:

and use this helper to exclude CPUs which are not enabled in the XIVE controller.

---
 hw/intc/pnv_xive.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
hw/intc/pnv_xive.c +2 -0
index 71ca4961b6b1..4c8c6e51c20f 100644
@@ -372,6 +372,20 @@ static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx,
     return pnv_xive_vst_read(xive, VST_TSEL_IVT, blk, idx, eas);
 }
 
+static int cpu_pir(PowerPCCPU *cpu)
+{

hw/intc/pnv_xive.c static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx, · line 386

Quoted 10 lines
hw/intc/pnv_xive.c +10 -0
+    CPUPPCState *env = &cpu->env;
+    return env->spr_cb[SPR_PIR].default_value;
+}
+
+static bool pnv_xive_is_cpu_enabled(PnvXive *xive, PowerPCCPU *cpu)
+{
+    int pir = cpu_pir(cpu);
+    int thrd_id = pir & 0x7f;
+
+    return xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(thrd_id);

A similar check is open-coded in pnv_xive_get_indirect_tctx() :

    /* Check that HW thread is XIVE enabled */
    if (!(xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(pir & 0x3f))) {
        xive_error(xive, "IC: CPU %x is not enabled", pir);
    }

The thread id is only the 6 lower bits of the PIR there, and so seems to indicate the skiboot sources:

        /* Get bit in register */
        bit = c->pir & 0x3f;

Why make it pir & 0x7f here ? If it should actually be 0x3f, maybe also use the helper in pnv_xive_get_indirect_tctx().

Quoted 16 lines
hw/intc/pnv_xive.c +6 -0
+}
+
 static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
                               uint8_t nvt_blk, uint32_t nvt_idx,
                               bool cam_ignore, uint8_t priority,
@@ -393,6 +407,10 @@ static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
             XiveTCTX *tctx;
             int ring;
 
+            if (!pnv_xive_is_cpu_enabled(xive, cpu)) {
+                continue;
+            }
+
             tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
 
             /*
#29 lore raw

Re: [PATCH for-5.0 v5 11/23] ppc/pnv: Introduce a pnv_xive_is_cpu_enabled() helper

Quoted 39 lines · Greg Kurz
Quoted 18 lines · On Fri, 15 Nov 2019 17:24:24 +0100 Cédric Le Goater <[email protected]> wrote:

and use this helper to exclude CPUs which are not enabled in the XIVE controller.

---
 hw/intc/pnv_xive.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
hw/intc/pnv_xive.c +2 -0
index 71ca4961b6b1..4c8c6e51c20f 100644
@@ -372,6 +372,20 @@ static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx,
     return pnv_xive_vst_read(xive, VST_TSEL_IVT, blk, idx, eas);
 }
 
+static int cpu_pir(PowerPCCPU *cpu)
+{

hw/intc/pnv_xive.c static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx, · line 386

Quoted 10 lines
hw/intc/pnv_xive.c +10 -0
+    CPUPPCState *env = &cpu->env;
+    return env->spr_cb[SPR_PIR].default_value;
+}
+
+static bool pnv_xive_is_cpu_enabled(PnvXive *xive, PowerPCCPU *cpu)
+{
+    int pir = cpu_pir(cpu);
+    int thrd_id = pir & 0x7f;
+
+    return xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(thrd_id);

A similar check is open-coded in pnv_xive_get_indirect_tctx() :

    /* Check that HW thread is XIVE enabled */
    if (!(xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(pir & 0x3f))) {
        xive_error(xive, "IC: CPU %x is not enabled", pir);
    }
Quoted 5 lines

The thread id is only the 6 lower bits of the PIR there, and so seems to indicate the skiboot sources:

        /* Get bit in register */
        bit = c->pir & 0x3f;

skiboot uses 0x3f when enabling the TCTXT of a CPU because register INT_TCTXT_EN0 covers cores 0-15 (normal) and 0-7 (fused) and register INT_TCTXT_EN1 covers cores 16-23 (normal) and 8-11 (fused). The encoding in the registers is a bit different.

Quoted 1 line

Why make it pir & 0x7f here ?

See pnv_chip_core_pir_p9 comments for some details on the CPU ID layout.

Quoted 1 line

If it should actually be 0x3f,

but yes, we should fix the mask in the register setting.

Quoted 1 line

maybe also use the helper in pnv_xive_get_indirect_tctx().

This is getting changed later on. So I rather not.

C.

Quoted 18 lines
Quoted 17 lines
hw/intc/pnv_xive.c +6 -0
+}
+
 static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
                               uint8_t nvt_blk, uint32_t nvt_idx,
                               bool cam_ignore, uint8_t priority,
@@ -393,6 +407,10 @@ static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
             XiveTCTX *tctx;
             int ring;
 
+            if (!pnv_xive_is_cpu_enabled(xive, cpu)) {
+                continue;
+            }
+
             tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
 
             /*
Reviewed-by
#30 lore raw

Re: [PATCH for-5.0 v5 11/23] ppc/pnv: Introduce a pnv_xive_is_cpu_enabled() helper

Quoted 51 lines · On Wed, 20 Nov 2019 22:40:31 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 39 lines · Greg Kurz
Quoted 18 lines · On Fri, 15 Nov 2019 17:24:24 +0100 Cédric Le Goater <[email protected]> wrote:

and use this helper to exclude CPUs which are not enabled in the XIVE controller.

---
 hw/intc/pnv_xive.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
hw/intc/pnv_xive.c +2 -0
index 71ca4961b6b1..4c8c6e51c20f 100644
@@ -372,6 +372,20 @@ static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx,
     return pnv_xive_vst_read(xive, VST_TSEL_IVT, blk, idx, eas);
 }
 
+static int cpu_pir(PowerPCCPU *cpu)
+{

hw/intc/pnv_xive.c static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx, · line 386

Quoted 10 lines
hw/intc/pnv_xive.c +10 -0
+    CPUPPCState *env = &cpu->env;
+    return env->spr_cb[SPR_PIR].default_value;
+}
+
+static bool pnv_xive_is_cpu_enabled(PnvXive *xive, PowerPCCPU *cpu)
+{
+    int pir = cpu_pir(cpu);
+    int thrd_id = pir & 0x7f;
+
+    return xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(thrd_id);

A similar check is open-coded in pnv_xive_get_indirect_tctx() :

    /* Check that HW thread is XIVE enabled */
    if (!(xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(pir & 0x3f))) {
        xive_error(xive, "IC: CPU %x is not enabled", pir);
    }
Quoted 5 lines

The thread id is only the 6 lower bits of the PIR there, and so seems to indicate the skiboot sources:

        /* Get bit in register */
        bit = c->pir & 0x3f;

skiboot uses 0x3f when enabling the TCTXT of a CPU because register INT_TCTXT_EN0 covers cores 0-15 (normal) and 0-7 (fused) and register INT_TCTXT_EN1 covers cores 16-23 (normal) and 8-11 (fused). The encoding in the registers is a bit different.

Quoted 5 lines
Quoted 1 line

Why make it pir & 0x7f here ?

See pnv_chip_core_pir_p9 comments for some details on the CPU ID layout.

*   57:61  Core number
*   62:63  Thread ID

Ok, so the CPU ID within the socket is 7 bits, ie. pir & 0x7f

Quoted 7 lines
Quoted 1 line

If it should actually be 0x3f,

but yes, we should fix the mask in the register setting.

Quoted 1 line

maybe also use the helper in pnv_xive_get_indirect_tctx().

This is getting changed later on. So I rather not.

I don't see any later change there, neither in this series, nor in your powernv-4.2 on github, but nevermind, this patch is good enough for the purpose of CAM line matching.

Quoted 21 lines

C.

Quoted 18 lines
Quoted 17 lines
hw/intc/pnv_xive.c +6 -0
+}
+
 static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
                               uint8_t nvt_blk, uint32_t nvt_idx,
                               bool cam_ignore, uint8_t priority,
@@ -393,6 +407,10 @@ static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
             XiveTCTX *tctx;
             int ring;
 
+            if (!pnv_xive_is_cpu_enabled(xive, cpu)) {
+                continue;
+            }
+
             tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
 
             /*
#31 lore raw

Re: [PATCH for-5.0 v5 11/23] ppc/pnv: Introduce a pnv_xive_is_cpu_enabled() helper

Quoted 66 lines · Greg Kurz
Quoted 51 lines · On Wed, 20 Nov 2019 22:40:31 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 39 lines · Greg Kurz
Quoted 18 lines · On Fri, 15 Nov 2019 17:24:24 +0100 Cédric Le Goater <[email protected]> wrote:

and use this helper to exclude CPUs which are not enabled in the XIVE controller.

---
 hw/intc/pnv_xive.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
hw/intc/pnv_xive.c +2 -0
index 71ca4961b6b1..4c8c6e51c20f 100644
@@ -372,6 +372,20 @@ static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx,
     return pnv_xive_vst_read(xive, VST_TSEL_IVT, blk, idx, eas);
 }
 
+static int cpu_pir(PowerPCCPU *cpu)
+{

hw/intc/pnv_xive.c static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx, · line 386

Quoted 10 lines
hw/intc/pnv_xive.c +10 -0
+    CPUPPCState *env = &cpu->env;
+    return env->spr_cb[SPR_PIR].default_value;
+}
+
+static bool pnv_xive_is_cpu_enabled(PnvXive *xive, PowerPCCPU *cpu)
+{
+    int pir = cpu_pir(cpu);
+    int thrd_id = pir & 0x7f;
+
+    return xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(thrd_id);

A similar check is open-coded in pnv_xive_get_indirect_tctx() :

    /* Check that HW thread is XIVE enabled */
    if (!(xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(pir & 0x3f))) {
        xive_error(xive, "IC: CPU %x is not enabled", pir);
    }
Quoted 5 lines

The thread id is only the 6 lower bits of the PIR there, and so seems to indicate the skiboot sources:

        /* Get bit in register */
        bit = c->pir & 0x3f;

skiboot uses 0x3f when enabling the TCTXT of a CPU because register INT_TCTXT_EN0 covers cores 0-15 (normal) and 0-7 (fused) and register INT_TCTXT_EN1 covers cores 16-23 (normal) and 8-11 (fused). The encoding in the registers is a bit different.

Quoted 5 lines
Quoted 1 line

Why make it pir & 0x7f here ?

See pnv_chip_core_pir_p9 comments for some details on the CPU ID layout.

*   57:61  Core number
*   62:63  Thread ID

Ok, so the CPU ID within the socket is 7 bits, ie. pir & 0x7f

Quoted 1 line
Quoted 1 line

If it should actually be 0x3f,

Quoted 8 lines
Quoted 6 lines

but yes, we should fix the mask in the register setting.

Quoted 1 line

maybe also use the helper in pnv_xive_get_indirect_tctx().

This is getting changed later on. So I rather not.

I don't see any later change there, neither in this series,

Patch "ppc/pnv: Clarify how the TIMA is accessed on a multichip system"

changes a few things in that area.

Quoted 2 lines

nor in your powernv-4.2 on github, but nevermind, this patch is good enough for the purpose of CAM line matching.

Yes. It could be better though and it's a localized change.

the INT_TCTXT_EN0 (PC_THREAD_EN_REG0) needs a small fix on the mask. We don't use the EN1 register also for cores > 15.

It works today because we don't start the machine with all the possible cores. Although it should be possible to start a QEMU PowerNV machine with 24 cores on each socket.

I would like to have stricter checks on CAM line accesses because it is an OS interface. The INT_TCTXT_EN0 (PC_THREAD_EN_REG0) is the first level (HW) but we need to check also the 'V' bit of each ring. That's more complex. For later.

C.

Quoted 24 lines
Quoted 21 lines

C.

Quoted 18 lines
Quoted 17 lines
hw/intc/pnv_xive.c +6 -0
+}
+
 static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
                               uint8_t nvt_blk, uint32_t nvt_idx,
                               bool cam_ignore, uint8_t priority,
@@ -393,6 +407,10 @@ static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
             XiveTCTX *tctx;
             int ring;
 
+            if (!pnv_xive_is_cpu_enabled(xive, cpu)) {
+                continue;
+            }
+
             tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
 
             /*
#32 lore raw

Re: [PATCH for-5.0 v5 11/23] ppc/pnv: Introduce a pnv_xive_is_cpu_enabled() helper

Quoted 71 lines · On Thu, 21 Nov 2019 10:16:14 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 70 lines · Greg Kurz
Quoted 51 lines · On Wed, 20 Nov 2019 22:40:31 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 39 lines · Greg Kurz
Quoted 18 lines · On Fri, 15 Nov 2019 17:24:24 +0100 Cédric Le Goater <[email protected]> wrote:

and use this helper to exclude CPUs which are not enabled in the XIVE controller.

---
 hw/intc/pnv_xive.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
hw/intc/pnv_xive.c +2 -0
index 71ca4961b6b1..4c8c6e51c20f 100644
@@ -372,6 +372,20 @@ static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx,
     return pnv_xive_vst_read(xive, VST_TSEL_IVT, blk, idx, eas);
 }
 
+static int cpu_pir(PowerPCCPU *cpu)
+{

hw/intc/pnv_xive.c static int pnv_xive_get_eas(XiveRouter *xrtr, uint8_t blk, uint32_t idx, · line 386

Quoted 10 lines
hw/intc/pnv_xive.c +10 -0
+    CPUPPCState *env = &cpu->env;
+    return env->spr_cb[SPR_PIR].default_value;
+}
+
+static bool pnv_xive_is_cpu_enabled(PnvXive *xive, PowerPCCPU *cpu)
+{
+    int pir = cpu_pir(cpu);
+    int thrd_id = pir & 0x7f;
+
+    return xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(thrd_id);

A similar check is open-coded in pnv_xive_get_indirect_tctx() :

    /* Check that HW thread is XIVE enabled */
    if (!(xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(pir & 0x3f))) {
        xive_error(xive, "IC: CPU %x is not enabled", pir);
    }
Quoted 5 lines

The thread id is only the 6 lower bits of the PIR there, and so seems to indicate the skiboot sources:

        /* Get bit in register */
        bit = c->pir & 0x3f;

skiboot uses 0x3f when enabling the TCTXT of a CPU because register INT_TCTXT_EN0 covers cores 0-15 (normal) and 0-7 (fused) and register INT_TCTXT_EN1 covers cores 16-23 (normal) and 8-11 (fused). The encoding in the registers is a bit different.

Quoted 5 lines
Quoted 1 line

Why make it pir & 0x7f here ?

See pnv_chip_core_pir_p9 comments for some details on the CPU ID layout.

*   57:61  Core number
*   62:63  Thread ID

Ok, so the CPU ID within the socket is 7 bits, ie. pir & 0x7f

Quoted 5 lines
Quoted 1 line

If it should actually be 0x3f,

but yes, we should fix the mask in the register setting.

Quoted 1 line

maybe also use the helper in pnv_xive_get_indirect_tctx().

Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

This is getting changed later on. So I rather not.

I don't see any later change there, neither in this series,

Patch "ppc/pnv: Clarify how the TIMA is accessed on a multichip system"

changes a few things in that area.

No, it changes pnv_xive_get_tctx() which gets removed later by

"[PATCH for-5.0 v5 19/23] ppc/xive: Remove the get_tctx() XiveRouter handler"

My remark was about pnv_xive_get_indirect_tctx(), but nevermind :)

Quoted 47 lines
Quoted 2 lines

nor in your powernv-4.2 on github, but nevermind, this patch is good enough for the purpose of CAM line matching.

Yes. It could be better though and it's a localized change.

the INT_TCTXT_EN0 (PC_THREAD_EN_REG0) needs a small fix on the mask. We don't use the EN1 register also for cores > 15.

It works today because we don't start the machine with all the possible cores. Although it should be possible to start a QEMU PowerNV machine with 24 cores on each socket.

I would like to have stricter checks on CAM line accesses because it is an OS interface. The INT_TCTXT_EN0 (PC_THREAD_EN_REG0) is the first level (HW) but we need to check also the 'V' bit of each ring. That's more complex. For later.

C.

Quoted 24 lines
Quoted 21 lines

C.

Quoted 18 lines
Quoted 17 lines
hw/intc/pnv_xive.c +6 -0
+}
+
 static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
                               uint8_t nvt_blk, uint32_t nvt_idx,
                               bool cam_ignore, uint8_t priority,
@@ -393,6 +407,10 @@ static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
             XiveTCTX *tctx;
             int ring;
 
+            if (!pnv_xive_is_cpu_enabled(xive, cpu)) {
+                continue;
+            }
+
             tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
 
             /*
#33 lore raw

PATCH v5 12/23 ppc/xive: Introduce a XiveFabric interface

In reply to Cédric Le Goater

The XiveFabric QOM interface acts as the PowerBUS interface between the interrupt controller and the system and should be implemented by the QEMU machine. On HW, the XIVE sub-engine is responsible for the communication with the other chip is the Common Queue (CQ) bridge unit.

This interface offers a 'match_nvt' handler to perform the CAM line matching when looking for a XIVE Presenter with a dispatched NVT.

---
 include/hw/ppc/xive.h | 22 ++++++++++++++++++++++
 hw/intc/xive.c        | 10 ++++++++++
 2 files changed, 32 insertions(+)
include/hw/ppc/xive.h +22 -0
index f9aa0fa0dac3..b00af988779b 100644
@@ -399,6 +399,28 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
                               uint8_t nvt_blk, uint32_t nvt_idx,
                               bool cam_ignore, uint32_t logic_serv);
 
+/*
+ * XIVE Fabric (Interface between Interrupt Controller and Machine)
+ */
+
+typedef struct XiveFabric XiveFabric;
+
+#define TYPE_XIVE_FABRIC "xive-fabric"
+#define XIVE_FABRIC(obj)                                     \
+    INTERFACE_CHECK(XiveFabric, (obj), TYPE_XIVE_FABRIC)
+#define XIVE_FABRIC_CLASS(klass)                                     \
+    OBJECT_CLASS_CHECK(XiveFabricClass, (klass), TYPE_XIVE_FABRIC)
+#define XIVE_FABRIC_GET_CLASS(obj)                                   \
+    OBJECT_GET_CLASS(XiveFabricClass, (obj), TYPE_XIVE_FABRIC)
+
+typedef struct XiveFabricClass {
+    InterfaceClass parent;
+    int (*match_nvt)(XiveFabric *xfb, uint8_t format,
+                     uint8_t nvt_blk, uint32_t nvt_idx,
+                     bool cam_ignore, uint8_t priority,
+                     uint32_t logic_serv, XiveTCTXMatch *match);
+} XiveFabricClass;
+
 /*
  * XIVE END ESBs
  */
hw/intc/xive.c +10 -0
index da6196ca958f..1c9e58f8deac 100644
@@ -1893,8 +1893,18 @@ static const TypeInfo xive_presenter_info = {
     .class_size = sizeof(XivePresenterClass),
 };
 
+/*
+ * XIVE Fabric
+ */
+static const TypeInfo xive_fabric_info = {
+    .name = TYPE_XIVE_FABRIC,
+    .parent = TYPE_INTERFACE,
+    .class_size = sizeof(XiveFabricClass),
+};
+
 static void xive_register_types(void)
 {
+    type_register_static(&xive_fabric_info);
     type_register_static(&xive_source_info);
     type_register_static(&xive_notifier_info);
     type_register_static(&xive_presenter_info);
Signature · 1 line
2.21.0
Reviewed-by on v5 12/23
#34 lore raw

Re: [PATCH for-5.0 v5 12/23] ppc/xive: Introduce a XiveFabric interface

Quoted 11 lines · On Fri, 15 Nov 2019 17:24:25 +0100 Cédric Le Goater <[email protected]> wrote:

The XiveFabric QOM interface acts as the PowerBUS interface between the interrupt controller and the system and should be implemented by the QEMU machine. On HW, the XIVE sub-engine is responsible for the communication with the other chip is the Common Queue (CQ) bridge unit.

This interface offers a 'match_nvt' handler to perform the CAM line matching when looking for a XIVE Presenter with a dispatched NVT.

---

Quoted 60 lines
 include/hw/ppc/xive.h | 22 ++++++++++++++++++++++
 hw/intc/xive.c        | 10 ++++++++++
 2 files changed, 32 insertions(+)
include/hw/ppc/xive.h +22 -0
index f9aa0fa0dac3..b00af988779b 100644
@@ -399,6 +399,28 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
                               uint8_t nvt_blk, uint32_t nvt_idx,
                               bool cam_ignore, uint32_t logic_serv);
 
+/*
+ * XIVE Fabric (Interface between Interrupt Controller and Machine)
+ */
+
+typedef struct XiveFabric XiveFabric;
+
+#define TYPE_XIVE_FABRIC "xive-fabric"
+#define XIVE_FABRIC(obj)                                     \
+    INTERFACE_CHECK(XiveFabric, (obj), TYPE_XIVE_FABRIC)
+#define XIVE_FABRIC_CLASS(klass)                                     \
+    OBJECT_CLASS_CHECK(XiveFabricClass, (klass), TYPE_XIVE_FABRIC)
+#define XIVE_FABRIC_GET_CLASS(obj)                                   \
+    OBJECT_GET_CLASS(XiveFabricClass, (obj), TYPE_XIVE_FABRIC)
+
+typedef struct XiveFabricClass {
+    InterfaceClass parent;
+    int (*match_nvt)(XiveFabric *xfb, uint8_t format,
+                     uint8_t nvt_blk, uint32_t nvt_idx,
+                     bool cam_ignore, uint8_t priority,
+                     uint32_t logic_serv, XiveTCTXMatch *match);
+} XiveFabricClass;
+
 /*
  * XIVE END ESBs
  */
hw/intc/xive.c +10 -0
index da6196ca958f..1c9e58f8deac 100644
@@ -1893,8 +1893,18 @@ static const TypeInfo xive_presenter_info = {
     .class_size = sizeof(XivePresenterClass),
 };
 
+/*
+ * XIVE Fabric
+ */
+static const TypeInfo xive_fabric_info = {
+    .name = TYPE_XIVE_FABRIC,
+    .parent = TYPE_INTERFACE,
+    .class_size = sizeof(XiveFabricClass),
+};
+
 static void xive_register_types(void)
 {
+    type_register_static(&xive_fabric_info);
     type_register_static(&xive_source_info);
     type_register_static(&xive_notifier_info);
     type_register_static(&xive_presenter_info);
#35 lore raw

PATCH v5 13/23 ppc/pnv: Implement the XiveFabric interface

In reply to Cédric Le Goater

The CAM line matching on the PowerNV machine now scans all chips of the system and all CPUs of a chip to find a dispatched NVT in the thread contexts.

---
 hw/ppc/pnv.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
hw/ppc/pnv.c +35 -0
index 94c9f536413f..207a5cf2c650 100644
@@ -1446,6 +1446,35 @@ static void pnv_pic_print_info(InterruptStatsProvider *obj,
     }
 }
 
+static int pnv_xive_match_nvt(XiveFabric *xfb, uint8_t format,
+                               uint8_t nvt_blk, uint32_t nvt_idx,
+                               bool cam_ignore, uint8_t priority,
+                               uint32_t logic_serv,
+                               XiveTCTXMatch *match)
+{
+    PnvMachineState *pnv = PNV_MACHINE(xfb);
+    int total_count = 0;
+    int i;
+
+    for (i = 0; i < pnv->num_chips; i++) {
+        Pnv9Chip *chip9 = PNV9_CHIP(pnv->chips[i]);
+        XivePresenter *xptr = XIVE_PRESENTER(&chip9->xive);
+        XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
+        int count;
+
+        count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
+                               priority, logic_serv, match);
+
+        if (count < 0) {
+            return count;
+        }
+
+        total_count += count;
+    }
+
+    return total_count;
+}
+
 static void pnv_get_num_chips(Object *obj, Visitor *v, const char *name,
                               void *opaque, Error **errp)
 {
@@ -1509,9 +1538,11 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
 static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
+    XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
 
     mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
     mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power9_v2.0");
+    xfc->match_nvt = pnv_xive_match_nvt;
 
     mc->alias = "powernv";
 }
@@ -1558,6 +1589,10 @@ static const TypeInfo types[] = {
         .name          = MACHINE_TYPE_NAME("powernv9"),
         .parent        = TYPE_PNV_MACHINE,
         .class_init    = pnv_machine_power9_class_init,
+        .interfaces = (InterfaceInfo[]) {
+            { TYPE_XIVE_FABRIC },
+            { },
+        },
     },
     {
         .name          = MACHINE_TYPE_NAME("powernv8"),
Signature · 1 line
2.21.0
Reviewed-by on v5 13/23
#36 lore raw

Re: [PATCH for-5.0 v5 13/23] ppc/pnv: Implement the XiveFabric interface

Quoted 6 lines · On Fri, 15 Nov 2019 17:24:26 +0100 Cédric Le Goater <[email protected]> wrote:

The CAM line matching on the PowerNV machine now scans all chips of the system and all CPUs of a chip to find a dispatched NVT in the thread contexts.

---

Quoted 66 lines
 hw/ppc/pnv.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
hw/ppc/pnv.c +35 -0
index 94c9f536413f..207a5cf2c650 100644
@@ -1446,6 +1446,35 @@ static void pnv_pic_print_info(InterruptStatsProvider *obj,
     }
 }
 
+static int pnv_xive_match_nvt(XiveFabric *xfb, uint8_t format,
+                               uint8_t nvt_blk, uint32_t nvt_idx,
+                               bool cam_ignore, uint8_t priority,
+                               uint32_t logic_serv,
+                               XiveTCTXMatch *match)
+{
+    PnvMachineState *pnv = PNV_MACHINE(xfb);
+    int total_count = 0;
+    int i;
+
+    for (i = 0; i < pnv->num_chips; i++) {
+        Pnv9Chip *chip9 = PNV9_CHIP(pnv->chips[i]);
+        XivePresenter *xptr = XIVE_PRESENTER(&chip9->xive);
+        XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
+        int count;
+
+        count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
+                               priority, logic_serv, match);
+
+        if (count < 0) {
+            return count;
+        }
+
+        total_count += count;
+    }
+
+    return total_count;
+}
+
 static void pnv_get_num_chips(Object *obj, Visitor *v, const char *name,
                               void *opaque, Error **errp)
 {
@@ -1509,9 +1538,11 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
 static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
+    XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
 
     mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
     mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power9_v2.0");
+    xfc->match_nvt = pnv_xive_match_nvt;
 
     mc->alias = "powernv";
 }
@@ -1558,6 +1589,10 @@ static const TypeInfo types[] = {
         .name          = MACHINE_TYPE_NAME("powernv9"),
         .parent        = TYPE_PNV_MACHINE,
         .class_init    = pnv_machine_power9_class_init,
+        .interfaces = (InterfaceInfo[]) {
+            { TYPE_XIVE_FABRIC },
+            { },
+        },
     },
     {
         .name          = MACHINE_TYPE_NAME("powernv8"),
#37 lore raw

PATCH v5 14/23 ppc/spapr: Implement the XiveFabric interface

In reply to Cédric Le Goater

The CAM line matching sequence in the pseries machine does not change much apart from the use of the new QOM interfaces. There is an extra indirection because of the sPAPR IRQ backend of the machine. Only the XIVE backend implements the new 'match_nvt' handler.

---
 hw/ppc/spapr.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
hw/ppc/spapr.c +36 -0
index 94f9d27096af..a8f5850f65bb 100644
@@ -4270,6 +4270,39 @@ static void spapr_pic_print_info(InterruptStatsProvider *obj,
                    kvm_irqchip_in_kernel() ? "in-kernel" : "emulated");
 }
 
+static int spapr_xive_match_nvt(XiveFabric *xfb, uint8_t format,
+                                uint8_t nvt_blk, uint32_t nvt_idx,
+                                bool cam_ignore, uint8_t priority,
+                                uint32_t logic_serv, XiveTCTXMatch *match)
+{
+    SpaprMachineState *spapr = SPAPR_MACHINE(xfb);
+    XivePresenter *xptr = XIVE_PRESENTER(spapr->xive);
+    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
+    int count;
+
+    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
+                           priority, logic_serv, match);
+    if (count < 0) {
+        return count;
+    }
+
+    /*
+     * When we implement the save and restore of the thread interrupt
+     * contexts in the enter/exit CPU handlers of the machine and the
+     * escalations in QEMU, we should be able to handle non dispatched
+     * vCPUs.
+     *
+     * Until this is done, the sPAPR machine should find at least one
+     * matching context always.
+     */
+    if (count == 0) {
+        qemu_log_mask(LOG_GUEST_ERROR, "XIVE: NVT %x/%x is not dispatched\n",
+                      nvt_blk, nvt_idx);
+    }
+
+    return count;
+}
+
 int spapr_get_vcpu_id(PowerPCCPU *cpu)
 {
     return cpu->vcpu_id;
@@ -4366,6 +4399,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     PPCVirtualHypervisorClass *vhc = PPC_VIRTUAL_HYPERVISOR_CLASS(oc);
     XICSFabricClass *xic = XICS_FABRIC_CLASS(oc);
     InterruptStatsProviderClass *ispc = INTERRUPT_STATS_PROVIDER_CLASS(oc);
+    XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
 
     mc->desc = "pSeries Logical Partition (PAPR compliant)";
     mc->ignore_boot_device_suffixes = true;
@@ -4442,6 +4476,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     smc->linux_pci_probe = true;
     smc->smp_threads_vsmt = true;
     smc->nr_xirqs = SPAPR_NR_XIRQS;
+    xfc->match_nvt = spapr_xive_match_nvt;
 }
 
 static const TypeInfo spapr_machine_info = {
@@ -4460,6 +4495,7 @@ static const TypeInfo spapr_machine_info = {
         { TYPE_PPC_VIRTUAL_HYPERVISOR },
         { TYPE_XICS_FABRIC },
         { TYPE_INTERRUPT_STATS_PROVIDER },
+        { TYPE_XIVE_FABRIC },
         { }
     },
 };
Signature · 1 line
2.21.0
Reviewed-by on v5 14/23
#38 lore raw

Re: [PATCH for-5.0 v5 14/23] ppc/spapr: Implement the XiveFabric interface

Quoted 5 lines · On Fri, 15 Nov 2019 17:24:27 +0100 Cédric Le Goater <[email protected]> wrote:

The CAM line matching sequence in the pseries machine does not change much apart from the use of the new QOM interfaces. There is an extra indirection because of the sPAPR IRQ backend of the machine. Only the XIVE backend implements the new 'match_nvt' handler.

The changelog needs an update since you dropped the indirection you had in v4.

Quoted 13 lines
---
 hw/ppc/spapr.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
hw/ppc/spapr.c +0 -0
index 94f9d27096af..a8f5850f65bb 100644
@@ -4270,6 +4270,39 @@ static void spapr_pic_print_info(InterruptStatsProvider *obj,
                    kvm_irqchip_in_kernel() ? "in-kernel" : "emulated");
 }
 

hw/ppc/spapr.c static void spapr_pic_print_info(InterruptStatsProvider *obj, · line 4282

Quoted 10 lines
hw/ppc/spapr.c +10 -0
+static int spapr_xive_match_nvt(XiveFabric *xfb, uint8_t format,
+                                uint8_t nvt_blk, uint32_t nvt_idx,
+                                bool cam_ignore, uint8_t priority,
+                                uint32_t logic_serv, XiveTCTXMatch *match)
+{
+    SpaprMachineState *spapr = SPAPR_MACHINE(xfb);
+    XivePresenter *xptr = XIVE_PRESENTER(spapr->xive);
+    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
+    int count;
+

As suggested by David, you should probably assert() that XIVE is in use for extra paran^Wsafety.

With these fixed,

Quoted 50 lines
hw/ppc/spapr.c +26 -0
+    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
+                           priority, logic_serv, match);
+    if (count < 0) {
+        return count;
+    }
+
+    /*
+     * When we implement the save and restore of the thread interrupt
+     * contexts in the enter/exit CPU handlers of the machine and the
+     * escalations in QEMU, we should be able to handle non dispatched
+     * vCPUs.
+     *
+     * Until this is done, the sPAPR machine should find at least one
+     * matching context always.
+     */
+    if (count == 0) {
+        qemu_log_mask(LOG_GUEST_ERROR, "XIVE: NVT %x/%x is not dispatched\n",
+                      nvt_blk, nvt_idx);
+    }
+
+    return count;
+}
+
 int spapr_get_vcpu_id(PowerPCCPU *cpu)
 {
     return cpu->vcpu_id;
@@ -4366,6 +4399,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     PPCVirtualHypervisorClass *vhc = PPC_VIRTUAL_HYPERVISOR_CLASS(oc);
     XICSFabricClass *xic = XICS_FABRIC_CLASS(oc);
     InterruptStatsProviderClass *ispc = INTERRUPT_STATS_PROVIDER_CLASS(oc);
+    XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
 
     mc->desc = "pSeries Logical Partition (PAPR compliant)";
     mc->ignore_boot_device_suffixes = true;
@@ -4442,6 +4476,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     smc->linux_pci_probe = true;
     smc->smp_threads_vsmt = true;
     smc->nr_xirqs = SPAPR_NR_XIRQS;
+    xfc->match_nvt = spapr_xive_match_nvt;
 }
 
 static const TypeInfo spapr_machine_info = {
@@ -4460,6 +4495,7 @@ static const TypeInfo spapr_machine_info = {
         { TYPE_PPC_VIRTUAL_HYPERVISOR },
         { TYPE_XICS_FABRIC },
         { TYPE_INTERRUPT_STATS_PROVIDER },
+        { TYPE_XIVE_FABRIC },
         { }
     },
 };
#39 lore raw

Re: [PATCH for-5.0 v5 14/23] ppc/spapr: Implement the XiveFabric interface

Quoted 11 lines · Greg Kurz
Quoted 5 lines · On Fri, 15 Nov 2019 17:24:27 +0100 Cédric Le Goater <[email protected]> wrote:

The CAM line matching sequence in the pseries machine does not change much apart from the use of the new QOM interfaces. There is an extra indirection because of the sPAPR IRQ backend of the machine. Only the XIVE backend implements the new 'match_nvt' handler.

The changelog needs an update since you dropped the indirection you had in v4.

Indeed.

Quoted 14 lines
Quoted 14 lines
---
 hw/ppc/spapr.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
hw/ppc/spapr.c +0 -0
index 94f9d27096af..a8f5850f65bb 100644
@@ -4270,6 +4270,39 @@ static void spapr_pic_print_info(InterruptStatsProvider *obj,
                    kvm_irqchip_in_kernel() ? "in-kernel" : "emulated");
 }
 
Quoted 13 lines

hw/ppc/spapr.c static void spapr_pic_print_info(InterruptStatsProvider *obj, · line 4282

Quoted 10 lines
hw/ppc/spapr.c +10 -0
+static int spapr_xive_match_nvt(XiveFabric *xfb, uint8_t format,
+                                uint8_t nvt_blk, uint32_t nvt_idx,
+                                bool cam_ignore, uint8_t priority,
+                                uint32_t logic_serv, XiveTCTXMatch *match)
+{
+    SpaprMachineState *spapr = SPAPR_MACHINE(xfb);
+    XivePresenter *xptr = XIVE_PRESENTER(spapr->xive);
+    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
+    int count;
+

As suggested by David, you should probably assert() that XIVE is in use for extra paran^Wsafety.

I don't see the need. The stack call is clear enough IMO. It can only be reached from the XiveRouter.

Thanks,

C.

Quoted 55 lines

With these fixed,

Quoted 50 lines
hw/ppc/spapr.c +26 -0
+    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
+                           priority, logic_serv, match);
+    if (count < 0) {
+        return count;
+    }
+
+    /*
+     * When we implement the save and restore of the thread interrupt
+     * contexts in the enter/exit CPU handlers of the machine and the
+     * escalations in QEMU, we should be able to handle non dispatched
+     * vCPUs.
+     *
+     * Until this is done, the sPAPR machine should find at least one
+     * matching context always.
+     */
+    if (count == 0) {
+        qemu_log_mask(LOG_GUEST_ERROR, "XIVE: NVT %x/%x is not dispatched\n",
+                      nvt_blk, nvt_idx);
+    }
+
+    return count;
+}
+
 int spapr_get_vcpu_id(PowerPCCPU *cpu)
 {
     return cpu->vcpu_id;
@@ -4366,6 +4399,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     PPCVirtualHypervisorClass *vhc = PPC_VIRTUAL_HYPERVISOR_CLASS(oc);
     XICSFabricClass *xic = XICS_FABRIC_CLASS(oc);
     InterruptStatsProviderClass *ispc = INTERRUPT_STATS_PROVIDER_CLASS(oc);
+    XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
 
     mc->desc = "pSeries Logical Partition (PAPR compliant)";
     mc->ignore_boot_device_suffixes = true;
@@ -4442,6 +4476,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     smc->linux_pci_probe = true;
     smc->smp_threads_vsmt = true;
     smc->nr_xirqs = SPAPR_NR_XIRQS;
+    xfc->match_nvt = spapr_xive_match_nvt;
 }
 
 static const TypeInfo spapr_machine_info = {
@@ -4460,6 +4495,7 @@ static const TypeInfo spapr_machine_info = {
         { TYPE_PPC_VIRTUAL_HYPERVISOR },
         { TYPE_XICS_FABRIC },
         { TYPE_INTERRUPT_STATS_PROVIDER },
+        { TYPE_XIVE_FABRIC },
         { }
     },
 };
#40 lore raw

Re: [PATCH for-5.0 v5 14/23] ppc/spapr: Implement the XiveFabric interface

Quoted 43 lines · On Thu, 21 Nov 2019 07:56:32 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 11 lines · Greg Kurz
Quoted 5 lines · On Fri, 15 Nov 2019 17:24:27 +0100 Cédric Le Goater <[email protected]> wrote:

The CAM line matching sequence in the pseries machine does not change much apart from the use of the new QOM interfaces. There is an extra indirection because of the sPAPR IRQ backend of the machine. Only the XIVE backend implements the new 'match_nvt' handler.

The changelog needs an update since you dropped the indirection you had in v4.

Indeed.

Quoted 14 lines
Quoted 14 lines
---
 hw/ppc/spapr.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
hw/ppc/spapr.c +0 -0
index 94f9d27096af..a8f5850f65bb 100644
@@ -4270,6 +4270,39 @@ static void spapr_pic_print_info(InterruptStatsProvider *obj,
                    kvm_irqchip_in_kernel() ? "in-kernel" : "emulated");
 }
 
Quoted 13 lines

hw/ppc/spapr.c static void spapr_pic_print_info(InterruptStatsProvider *obj, · line 4282

Quoted 10 lines
hw/ppc/spapr.c +10 -0
+static int spapr_xive_match_nvt(XiveFabric *xfb, uint8_t format,
+                                uint8_t nvt_blk, uint32_t nvt_idx,
+                                bool cam_ignore, uint8_t priority,
+                                uint32_t logic_serv, XiveTCTXMatch *match)
+{
+    SpaprMachineState *spapr = SPAPR_MACHINE(xfb);
+    XivePresenter *xptr = XIVE_PRESENTER(spapr->xive);
+    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
+    int count;
+

As suggested by David, you should probably assert() that XIVE is in use for extra paran^Wsafety.

Quoted 3 lines

I don't see the need. The stack call is clear enough IMO. It can only be reached from the XiveRouter.

Hmm... the assert() proposal isn't about this getting called by some other code, it is about ensuring XIVE is the active IC in case the machine was started with ic-mode=dual. But if you're confident enough it can never ever happen, no matter any subsequent change may done to the code, then don't add it :)

Quoted 60 lines

Thanks,

C.

Quoted 55 lines

With these fixed,

Quoted 50 lines
hw/ppc/spapr.c +26 -0
+    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
+                           priority, logic_serv, match);
+    if (count < 0) {
+        return count;
+    }
+
+    /*
+     * When we implement the save and restore of the thread interrupt
+     * contexts in the enter/exit CPU handlers of the machine and the
+     * escalations in QEMU, we should be able to handle non dispatched
+     * vCPUs.
+     *
+     * Until this is done, the sPAPR machine should find at least one
+     * matching context always.
+     */
+    if (count == 0) {
+        qemu_log_mask(LOG_GUEST_ERROR, "XIVE: NVT %x/%x is not dispatched\n",
+                      nvt_blk, nvt_idx);
+    }
+
+    return count;
+}
+
 int spapr_get_vcpu_id(PowerPCCPU *cpu)
 {
     return cpu->vcpu_id;
@@ -4366,6 +4399,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     PPCVirtualHypervisorClass *vhc = PPC_VIRTUAL_HYPERVISOR_CLASS(oc);
     XICSFabricClass *xic = XICS_FABRIC_CLASS(oc);
     InterruptStatsProviderClass *ispc = INTERRUPT_STATS_PROVIDER_CLASS(oc);
+    XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
 
     mc->desc = "pSeries Logical Partition (PAPR compliant)";
     mc->ignore_boot_device_suffixes = true;
@@ -4442,6 +4476,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     smc->linux_pci_probe = true;
     smc->smp_threads_vsmt = true;
     smc->nr_xirqs = SPAPR_NR_XIRQS;
+    xfc->match_nvt = spapr_xive_match_nvt;
 }
 
 static const TypeInfo spapr_machine_info = {
@@ -4460,6 +4495,7 @@ static const TypeInfo spapr_machine_info = {
         { TYPE_PPC_VIRTUAL_HYPERVISOR },
         { TYPE_XICS_FABRIC },
         { TYPE_INTERRUPT_STATS_PROVIDER },
+        { TYPE_XIVE_FABRIC },
         { }
     },
 };
#41 lore raw

Re: [PATCH for-5.0 v5 14/23] ppc/spapr: Implement the XiveFabric interface

Quoted 50 lines · Greg Kurz
Quoted 43 lines · On Thu, 21 Nov 2019 07:56:32 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 11 lines · Greg Kurz
Quoted 5 lines · On Fri, 15 Nov 2019 17:24:27 +0100 Cédric Le Goater <[email protected]> wrote:

The CAM line matching sequence in the pseries machine does not change much apart from the use of the new QOM interfaces. There is an extra indirection because of the sPAPR IRQ backend of the machine. Only the XIVE backend implements the new 'match_nvt' handler.

The changelog needs an update since you dropped the indirection you had in v4.

Indeed.

Quoted 14 lines
Quoted 14 lines
---
 hw/ppc/spapr.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
hw/ppc/spapr.c +0 -0
index 94f9d27096af..a8f5850f65bb 100644
@@ -4270,6 +4270,39 @@ static void spapr_pic_print_info(InterruptStatsProvider *obj,
                    kvm_irqchip_in_kernel() ? "in-kernel" : "emulated");
 }
 
Quoted 13 lines

hw/ppc/spapr.c static void spapr_pic_print_info(InterruptStatsProvider *obj, · line 4282

Quoted 10 lines
hw/ppc/spapr.c +10 -0
+static int spapr_xive_match_nvt(XiveFabric *xfb, uint8_t format,
+                                uint8_t nvt_blk, uint32_t nvt_idx,
+                                bool cam_ignore, uint8_t priority,
+                                uint32_t logic_serv, XiveTCTXMatch *match)
+{
+    SpaprMachineState *spapr = SPAPR_MACHINE(xfb);
+    XivePresenter *xptr = XIVE_PRESENTER(spapr->xive);
+    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
+    int count;
+

As suggested by David, you should probably assert() that XIVE is in use for extra paran^Wsafety.

Quoted 3 lines

I don't see the need. The stack call is clear enough IMO. It can only be reached from the XiveRouter.

Quoted 5 lines

Hmm... the assert() proposal isn't about this getting called by some other code, it is about ensuring XIVE is the active IC in case the machine was started with ic-mode=dual. But if you're confident enough it can never ever happen, no matter any subsequent change may done to the code, then don't add it :)

If XIVE mode is not selected, the XIVE ESB pages are not mapped in the machine address space and you can not reach the Router without them.

C.

Quoted 62 lines
Quoted 61 lines

Thanks,

C.

Quoted 55 lines

With these fixed,

Quoted 50 lines
hw/ppc/spapr.c +26 -0
+    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
+                           priority, logic_serv, match);
+    if (count < 0) {
+        return count;
+    }
+
+    /*
+     * When we implement the save and restore of the thread interrupt
+     * contexts in the enter/exit CPU handlers of the machine and the
+     * escalations in QEMU, we should be able to handle non dispatched
+     * vCPUs.
+     *
+     * Until this is done, the sPAPR machine should find at least one
+     * matching context always.
+     */
+    if (count == 0) {
+        qemu_log_mask(LOG_GUEST_ERROR, "XIVE: NVT %x/%x is not dispatched\n",
+                      nvt_blk, nvt_idx);
+    }
+
+    return count;
+}
+
 int spapr_get_vcpu_id(PowerPCCPU *cpu)
 {
     return cpu->vcpu_id;
@@ -4366,6 +4399,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     PPCVirtualHypervisorClass *vhc = PPC_VIRTUAL_HYPERVISOR_CLASS(oc);
     XICSFabricClass *xic = XICS_FABRIC_CLASS(oc);
     InterruptStatsProviderClass *ispc = INTERRUPT_STATS_PROVIDER_CLASS(oc);
+    XiveFabricClass *xfc = XIVE_FABRIC_CLASS(oc);
 
     mc->desc = "pSeries Logical Partition (PAPR compliant)";
     mc->ignore_boot_device_suffixes = true;
@@ -4442,6 +4476,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     smc->linux_pci_probe = true;
     smc->smp_threads_vsmt = true;
     smc->nr_xirqs = SPAPR_NR_XIRQS;
+    xfc->match_nvt = spapr_xive_match_nvt;
 }
 
 static const TypeInfo spapr_machine_info = {
@@ -4460,6 +4495,7 @@ static const TypeInfo spapr_machine_info = {
         { TYPE_PPC_VIRTUAL_HYPERVISOR },
         { TYPE_XICS_FABRIC },
         { TYPE_INTERRUPT_STATS_PROVIDER },
+        { TYPE_XIVE_FABRIC },
         { }
     },
 };
#42 lore raw

PATCH v5 15/23 ppc/xive: Use the XiveFabric and XivePresenter interfaces

In reply to Cédric Le Goater

Now that the machines have handlers implementing the XiveFabric and XivePresenter interfaces, remove xive_presenter_match() and make use of the 'match_nvt' handler of the machine.

---
 hw/intc/xive.c | 48 +++++++++++++++++-------------------------------
 1 file changed, 17 insertions(+), 31 deletions(-)
hw/intc/xive.c +17 -31
index 1c9e58f8deac..ab62bda85788 100644
@@ -1423,30 +1423,6 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
     return -1;
 }
 
-static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
-                                 uint8_t nvt_blk, uint32_t nvt_idx,
-                                 bool cam_ignore, uint8_t priority,
-                                 uint32_t logic_serv, XiveTCTXMatch *match)
-{
-    XivePresenter *xptr = XIVE_PRESENTER(xrtr);
-    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
-    int count;
-
-    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
-                           priority, logic_serv, match);
-    if (count < 0) {
-        return false;
-    }
-
-    if (!match->tctx) {
-        qemu_log_mask(LOG_UNIMP, "XIVE: NVT %x/%x is not dispatched\n",
-                      nvt_blk, nvt_idx);
-        return false;
-    }
-
-    return true;
-}
-
 /*
  * This is our simple Xive Presenter Engine model. It is merged in the
  * Router as it does not require an extra object.
@@ -1462,22 +1438,32 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
  *
  * The parameters represent what is sent on the PowerBus
  */
-static bool xive_presenter_notify(XiveRouter *xrtr, uint8_t format,
+static bool xive_presenter_notify(uint8_t format,
                                   uint8_t nvt_blk, uint32_t nvt_idx,
                                   bool cam_ignore, uint8_t priority,
                                   uint32_t logic_serv)
 {
+    XiveFabric *xfb = XIVE_FABRIC(qdev_get_machine());
+    XiveFabricClass *xfc = XIVE_FABRIC_GET_CLASS(xfb);
     XiveTCTXMatch match = { .tctx = NULL, .ring = 0 };
-    bool found;
+    int count;
 
-    found = xive_presenter_match(xrtr, format, nvt_blk, nvt_idx, cam_ignore,
-                                 priority, logic_serv, &match);
-    if (found) {
+    /*
+     * Ask the machine to scan the interrupt controllers for a match
+     */
+    count = xfc->match_nvt(xfb, format, nvt_blk, nvt_idx, cam_ignore,
+                           priority, logic_serv, &match);
+    if (count < 0) {
+        return false;
+    }
+
+    /* handle CPU exception delivery */
+    if (count) {
         ipb_update(&match.tctx->regs[match.ring], priority);
         xive_tctx_notify(match.tctx, match.ring);
     }
 
-    return found;
+    return count;
 }
 
 /*
@@ -1590,7 +1576,7 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
         return;
     }
 
-    found = xive_presenter_notify(xrtr, format, nvt_blk, nvt_idx,
+    found = xive_presenter_notify(format, nvt_blk, nvt_idx,
                           xive_get_field32(END_W7_F0_IGNORE, end.w7),
                           priority,
                           xive_get_field32(END_W7_F1_LOG_SERVER_ID, end.w7));
Signature · 1 line
2.21.0
#43 lore raw

Re: [PATCH for-5.0 v5 15/23] ppc/xive: Use the XiveFabric and XivePresenter interfaces

Quoted 9 lines · On Fri, 15 Nov 2019 17:24:28 +0100 Cédric Le Goater <[email protected]> wrote:

Now that the machines have handlers implementing the XiveFabric and XivePresenter interfaces, remove xive_presenter_match() and make use of the 'match_nvt' handler of the machine.

---
 hw/intc/xive.c | 48 +++++++++++++++++-------------------------------
 1 file changed, 17 insertions(+), 31 deletions(-)

Nice diffstat :)

Quoted 16 lines
hw/intc/xive.c +0 -8
index 1c9e58f8deac..ab62bda85788 100644
@@ -1423,30 +1423,6 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
     return -1;
 }
 
-static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
-                                 uint8_t nvt_blk, uint32_t nvt_idx,
-                                 bool cam_ignore, uint8_t priority,
-                                 uint32_t logic_serv, XiveTCTXMatch *match)
-{
-    XivePresenter *xptr = XIVE_PRESENTER(xrtr);
-    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
-    int count;

hw/intc/xive.c int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx, · line 1443

Quoted 10 lines
hw/intc/xive.c +0 -10
-
-    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
-                           priority, logic_serv, match);
-    if (count < 0) {
-        return false;
-    }
-
-    if (!match->tctx) {
-        qemu_log_mask(LOG_UNIMP, "XIVE: NVT %x/%x is not dispatched\n",
-                      nvt_blk, nvt_idx);

Maybe keep this trace...

Quoted 32 lines
hw/intc/xive.c +8 -11
-        return false;
-    }
-
-    return true;
-}
-
 /*
  * This is our simple Xive Presenter Engine model. It is merged in the
  * Router as it does not require an extra object.
@@ -1462,22 +1438,32 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
  *
  * The parameters represent what is sent on the PowerBus
  */
-static bool xive_presenter_notify(XiveRouter *xrtr, uint8_t format,
+static bool xive_presenter_notify(uint8_t format,
                                   uint8_t nvt_blk, uint32_t nvt_idx,
                                   bool cam_ignore, uint8_t priority,
                                   uint32_t logic_serv)
 {
+    XiveFabric *xfb = XIVE_FABRIC(qdev_get_machine());
+    XiveFabricClass *xfc = XIVE_FABRIC_GET_CLASS(xfb);
     XiveTCTXMatch match = { .tctx = NULL, .ring = 0 };
-    bool found;
+    int count;
 
-    found = xive_presenter_match(xrtr, format, nvt_blk, nvt_idx, cam_ignore,
-                                 priority, logic_serv, &match);
-    if (found) {
+    /*
+     * Ask the machine to scan the interrupt controllers for a match
+     */
+    count = xfc->match_nvt(xfb, format, nvt_blk, nvt_idx, cam_ignore,

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1464

Quoted 10 lines
hw/intc/xive.c +7 -0
+                           priority, logic_serv, &match);
+    if (count < 0) {
+        return false;
+    }
+
+    /* handle CPU exception delivery */
+    if (count) {
         ipb_update(&match.tctx->regs[match.ring], priority);
         xive_tctx_notify(match.tctx, match.ring);
     }

... in an else block here ^^ ?

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1466

Quoted 3 lines
hw/intc/xive.c +1 -1
-    return found;
+    return count;

Implicit cast is ok I guess, but !!count would ensure no paranoid compiler ever complains.

Quoted 12 lines
hw/intc/xive.c +1 -1
 }
 
 /*
@@ -1590,7 +1576,7 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
         return;
     }
 
-    found = xive_presenter_notify(xrtr, format, nvt_blk, nvt_idx,
+    found = xive_presenter_notify(format, nvt_blk, nvt_idx,
                           xive_get_field32(END_W7_F0_IGNORE, end.w7),
                           priority,
                           xive_get_field32(END_W7_F1_LOG_SERVER_ID, end.w7));
#44 lore raw

Re: [PATCH for-5.0 v5 15/23] ppc/xive: Use the XiveFabric and XivePresenter interfaces

Quoted 35 lines · Greg Kurz
Quoted 9 lines · On Fri, 15 Nov 2019 17:24:28 +0100 Cédric Le Goater <[email protected]> wrote:

Now that the machines have handlers implementing the XiveFabric and XivePresenter interfaces, remove xive_presenter_match() and make use of the 'match_nvt' handler of the machine.

---
 hw/intc/xive.c | 48 +++++++++++++++++-------------------------------
 1 file changed, 17 insertions(+), 31 deletions(-)

Nice diffstat :)

Quoted 20 lines
hw/intc/xive.c +0 -12
index 1c9e58f8deac..ab62bda85788 100644
@@ -1423,30 +1423,6 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
     return -1;
 }
 
-static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
-                                 uint8_t nvt_blk, uint32_t nvt_idx,
-                                 bool cam_ignore, uint8_t priority,
-                                 uint32_t logic_serv, XiveTCTXMatch *match)
-{
-    XivePresenter *xptr = XIVE_PRESENTER(xrtr);
-    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
-    int count;
-
-    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
-                           priority, logic_serv, match);
-    if (count < 0) {
Quoted 8 lines

hw/intc/xive.c int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx, · line 1443

Quoted 6 lines
hw/intc/xive.c +0 -6
-        return false;
-    }
-
-    if (!match->tctx) {
-        qemu_log_mask(LOG_UNIMP, "XIVE: NVT %x/%x is not dispatched\n",
-                      nvt_blk, nvt_idx);

Maybe keep this trace...

It's in spapr_xive_match_nvt() now.

Quoted 47 lines
Quoted 33 lines
hw/intc/xive.c +8 -11
-        return false;
-    }
-
-    return true;
-}
-
 /*
  * This is our simple Xive Presenter Engine model. It is merged in the
  * Router as it does not require an extra object.
@@ -1462,22 +1438,32 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
  *
  * The parameters represent what is sent on the PowerBus
  */
-static bool xive_presenter_notify(XiveRouter *xrtr, uint8_t format,
+static bool xive_presenter_notify(uint8_t format,
                                   uint8_t nvt_blk, uint32_t nvt_idx,
                                   bool cam_ignore, uint8_t priority,
                                   uint32_t logic_serv)
 {
+    XiveFabric *xfb = XIVE_FABRIC(qdev_get_machine());
+    XiveFabricClass *xfc = XIVE_FABRIC_GET_CLASS(xfb);
     XiveTCTXMatch match = { .tctx = NULL, .ring = 0 };
-    bool found;
+    int count;
 
-    found = xive_presenter_match(xrtr, format, nvt_blk, nvt_idx, cam_ignore,
-                                 priority, logic_serv, &match);
-    if (found) {
+    /*
+     * Ask the machine to scan the interrupt controllers for a match
+     */
+    count = xfc->match_nvt(xfb, format, nvt_blk, nvt_idx, cam_ignore,

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1464

Quoted 10 lines
hw/intc/xive.c +7 -0
+                           priority, logic_serv, &match);
+    if (count < 0) {
+        return false;
+    }
+
+    /* handle CPU exception delivery */
+    if (count) {
         ipb_update(&match.tctx->regs[match.ring], priority);
         xive_tctx_notify(match.tctx, match.ring);
     }

... in an else block here ^^ ?

Quoted 1 line
Quoted 5 lines

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1466

Quoted 2 lines
hw/intc/xive.c +1 -1
-    return found;
+    return count;

Implicit cast is ok I guess, but !!count would ensure no paranoid compiler ever complains.

yes.

Thanks,

C.

Quoted 14 lines
Quoted 13 lines

}

/*

hw/intc/xive.c +1 -1
@@ -1590,7 +1576,7 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
         return;
     }
 
-    found = xive_presenter_notify(xrtr, format, nvt_blk, nvt_idx,
+    found = xive_presenter_notify(format, nvt_blk, nvt_idx,
                           xive_get_field32(END_W7_F0_IGNORE, end.w7),
                           priority,
                           xive_get_field32(END_W7_F1_LOG_SERVER_ID, end.w7));
#45 lore raw

Re: [PATCH for-5.0 v5 15/23] ppc/xive: Use the XiveFabric and XivePresenter interfaces

Quoted 45 lines · On Thu, 21 Nov 2019 08:01:44 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 35 lines · Greg Kurz
Quoted 9 lines · On Fri, 15 Nov 2019 17:24:28 +0100 Cédric Le Goater <[email protected]> wrote:

Now that the machines have handlers implementing the XiveFabric and XivePresenter interfaces, remove xive_presenter_match() and make use of the 'match_nvt' handler of the machine.

---
 hw/intc/xive.c | 48 +++++++++++++++++-------------------------------
 1 file changed, 17 insertions(+), 31 deletions(-)

Nice diffstat :)

Quoted 20 lines
hw/intc/xive.c +0 -12
index 1c9e58f8deac..ab62bda85788 100644
@@ -1423,30 +1423,6 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
     return -1;
 }
 
-static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
-                                 uint8_t nvt_blk, uint32_t nvt_idx,
-                                 bool cam_ignore, uint8_t priority,
-                                 uint32_t logic_serv, XiveTCTXMatch *match)
-{
-    XivePresenter *xptr = XIVE_PRESENTER(xrtr);
-    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
-    int count;
-
-    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
-                           priority, logic_serv, match);
-    if (count < 0) {
Quoted 8 lines

hw/intc/xive.c int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx, · line 1443

Quoted 6 lines
hw/intc/xive.c +0 -6
-        return false;
-    }
-
-    if (!match->tctx) {
-        qemu_log_mask(LOG_UNIMP, "XIVE: NVT %x/%x is not dispatched\n",
-                      nvt_blk, nvt_idx);

Maybe keep this trace...

Quoted 2 lines

It's in spapr_xive_match_nvt() now.

Not really... spapr_xive_match_nvt() has a trace for the opposite case of duplicate matches:

            if (match->tctx) {
                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a thread "
                              "context NVT %x/%x\n", nvt_blk, nvt_idx);
                return -1;
            }
Quoted 75 lines
Quoted 47 lines
Quoted 33 lines
hw/intc/xive.c +8 -11
-        return false;
-    }
-
-    return true;
-}
-
 /*
  * This is our simple Xive Presenter Engine model. It is merged in the
  * Router as it does not require an extra object.
@@ -1462,22 +1438,32 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
  *
  * The parameters represent what is sent on the PowerBus
  */
-static bool xive_presenter_notify(XiveRouter *xrtr, uint8_t format,
+static bool xive_presenter_notify(uint8_t format,
                                   uint8_t nvt_blk, uint32_t nvt_idx,
                                   bool cam_ignore, uint8_t priority,
                                   uint32_t logic_serv)
 {
+    XiveFabric *xfb = XIVE_FABRIC(qdev_get_machine());
+    XiveFabricClass *xfc = XIVE_FABRIC_GET_CLASS(xfb);
     XiveTCTXMatch match = { .tctx = NULL, .ring = 0 };
-    bool found;
+    int count;
 
-    found = xive_presenter_match(xrtr, format, nvt_blk, nvt_idx, cam_ignore,
-                                 priority, logic_serv, &match);
-    if (found) {
+    /*
+     * Ask the machine to scan the interrupt controllers for a match
+     */
+    count = xfc->match_nvt(xfb, format, nvt_blk, nvt_idx, cam_ignore,

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1464

Quoted 10 lines
hw/intc/xive.c +7 -0
+                           priority, logic_serv, &match);
+    if (count < 0) {
+        return false;
+    }
+
+    /* handle CPU exception delivery */
+    if (count) {
         ipb_update(&match.tctx->regs[match.ring], priority);
         xive_tctx_notify(match.tctx, match.ring);
     }

... in an else block here ^^ ?

Quoted 1 line
Quoted 5 lines

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1466

Quoted 2 lines
hw/intc/xive.c +1 -1
-    return found;
+    return count;

Implicit cast is ok I guess, but !!count would ensure no paranoid compiler ever complains.

yes.

Thanks,

C.

Quoted 14 lines
Quoted 13 lines

}

/*

hw/intc/xive.c +1 -1
@@ -1590,7 +1576,7 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
         return;
     }
 
-    found = xive_presenter_notify(xrtr, format, nvt_blk, nvt_idx,
+    found = xive_presenter_notify(format, nvt_blk, nvt_idx,
                           xive_get_field32(END_W7_F0_IGNORE, end.w7),
                           priority,
                           xive_get_field32(END_W7_F1_LOG_SERVER_ID, end.w7));
#46 lore raw

Re: [PATCH for-5.0 v5 15/23] ppc/xive: Use the XiveFabric and XivePresenter interfaces

Quoted 48 lines · Greg Kurz
Quoted 45 lines · On Thu, 21 Nov 2019 08:01:44 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 43 lines · Greg Kurz
Quoted 9 lines · On Fri, 15 Nov 2019 17:24:28 +0100 Cédric Le Goater <[email protected]> wrote:

Now that the machines have handlers implementing the XiveFabric and XivePresenter interfaces, remove xive_presenter_match() and make use of the 'match_nvt' handler of the machine.

---
 hw/intc/xive.c | 48 +++++++++++++++++-------------------------------
 1 file changed, 17 insertions(+), 31 deletions(-)

Nice diffstat :)

Quoted 16 lines
hw/intc/xive.c +0 -8
index 1c9e58f8deac..ab62bda85788 100644
@@ -1423,30 +1423,6 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
     return -1;
 }
 
-static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
-                                 uint8_t nvt_blk, uint32_t nvt_idx,
-                                 bool cam_ignore, uint8_t priority,
-                                 uint32_t logic_serv, XiveTCTXMatch *match)
-{
-    XivePresenter *xptr = XIVE_PRESENTER(xrtr);
-    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
-    int count;

hw/intc/xive.c int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx, · line 1443

Quoted 10 lines
hw/intc/xive.c +0 -10
-
-    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
-                           priority, logic_serv, match);
-    if (count < 0) {
-        return false;
-    }
-
-    if (!match->tctx) {
-        qemu_log_mask(LOG_UNIMP, "XIVE: NVT %x/%x is not dispatched\n",
-                      nvt_blk, nvt_idx);

Maybe keep this trace...

Quoted 5 lines
Quoted 2 lines

It's in spapr_xive_match_nvt() now.

Not really... spapr_xive_match_nvt() has a trace for the opposite case of duplicate matches:

not that one. The one in spapr.c ... Yes I need to change the name.

C.

Quoted 83 lines
            if (match->tctx) {
                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a thread "
                              "context NVT %x/%x\n", nvt_blk, nvt_idx);
                return -1;
            }
Quoted 75 lines
Quoted 47 lines
Quoted 33 lines
hw/intc/xive.c +8 -11
-        return false;
-    }
-
-    return true;
-}
-
 /*
  * This is our simple Xive Presenter Engine model. It is merged in the
  * Router as it does not require an extra object.
@@ -1462,22 +1438,32 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
  *
  * The parameters represent what is sent on the PowerBus
  */
-static bool xive_presenter_notify(XiveRouter *xrtr, uint8_t format,
+static bool xive_presenter_notify(uint8_t format,
                                   uint8_t nvt_blk, uint32_t nvt_idx,
                                   bool cam_ignore, uint8_t priority,
                                   uint32_t logic_serv)
 {
+    XiveFabric *xfb = XIVE_FABRIC(qdev_get_machine());
+    XiveFabricClass *xfc = XIVE_FABRIC_GET_CLASS(xfb);
     XiveTCTXMatch match = { .tctx = NULL, .ring = 0 };
-    bool found;
+    int count;
 
-    found = xive_presenter_match(xrtr, format, nvt_blk, nvt_idx, cam_ignore,
-                                 priority, logic_serv, &match);
-    if (found) {
+    /*
+     * Ask the machine to scan the interrupt controllers for a match
+     */
+    count = xfc->match_nvt(xfb, format, nvt_blk, nvt_idx, cam_ignore,

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1464

Quoted 10 lines
hw/intc/xive.c +7 -0
+                           priority, logic_serv, &match);
+    if (count < 0) {
+        return false;
+    }
+
+    /* handle CPU exception delivery */
+    if (count) {
         ipb_update(&match.tctx->regs[match.ring], priority);
         xive_tctx_notify(match.tctx, match.ring);
     }

... in an else block here ^^ ?

Quoted 1 line
Quoted 5 lines

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1466

Quoted 2 lines
hw/intc/xive.c +1 -1
-    return found;
+    return count;

Implicit cast is ok I guess, but !!count would ensure no paranoid compiler ever complains.

yes.

Thanks,

C.

Quoted 14 lines
Quoted 13 lines

}

/*

hw/intc/xive.c +1 -1
@@ -1590,7 +1576,7 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
         return;
     }
 
-    found = xive_presenter_notify(xrtr, format, nvt_blk, nvt_idx,
+    found = xive_presenter_notify(format, nvt_blk, nvt_idx,
                           xive_get_field32(END_W7_F0_IGNORE, end.w7),
                           priority,
                           xive_get_field32(END_W7_F1_LOG_SERVER_ID, end.w7));
Reviewed-by
#47 lore raw

Re: [PATCH for-5.0 v5 15/23] ppc/xive: Use the XiveFabric and XivePresenter interfaces

Quoted 49 lines · On Thu, 21 Nov 2019 08:40:32 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 48 lines · Greg Kurz
Quoted 45 lines · On Thu, 21 Nov 2019 08:01:44 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 43 lines · Greg Kurz
Quoted 9 lines · On Fri, 15 Nov 2019 17:24:28 +0100 Cédric Le Goater <[email protected]> wrote:

Now that the machines have handlers implementing the XiveFabric and XivePresenter interfaces, remove xive_presenter_match() and make use of the 'match_nvt' handler of the machine.

---
 hw/intc/xive.c | 48 +++++++++++++++++-------------------------------
 1 file changed, 17 insertions(+), 31 deletions(-)

Nice diffstat :)

Quoted 16 lines
hw/intc/xive.c +0 -8
index 1c9e58f8deac..ab62bda85788 100644
@@ -1423,30 +1423,6 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
     return -1;
 }
 
-static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
-                                 uint8_t nvt_blk, uint32_t nvt_idx,
-                                 bool cam_ignore, uint8_t priority,
-                                 uint32_t logic_serv, XiveTCTXMatch *match)
-{
-    XivePresenter *xptr = XIVE_PRESENTER(xrtr);
-    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
-    int count;

hw/intc/xive.c int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx, · line 1443

Quoted 10 lines
hw/intc/xive.c +0 -10
-
-    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
-                           priority, logic_serv, match);
-    if (count < 0) {
-        return false;
-    }
-
-    if (!match->tctx) {
-        qemu_log_mask(LOG_UNIMP, "XIVE: NVT %x/%x is not dispatched\n",
-                      nvt_blk, nvt_idx);

Maybe keep this trace...

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

It's in spapr_xive_match_nvt() now.

Not really... spapr_xive_match_nvt() has a trace for the opposite case of duplicate matches:

not that one. The one in spapr.c ... Yes I need to change the name.

... and it seems I cannot memorize a change that was made by the previous patch :-\ Sorry for the noise.

With or without the !!count change:

Quoted 86 lines

C.

Quoted 83 lines
            if (match->tctx) {
                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a thread "
                              "context NVT %x/%x\n", nvt_blk, nvt_idx);
                return -1;
            }
Quoted 75 lines
Quoted 47 lines
Quoted 33 lines
hw/intc/xive.c +8 -11
-        return false;
-    }
-
-    return true;
-}
-
 /*
  * This is our simple Xive Presenter Engine model. It is merged in the
  * Router as it does not require an extra object.
@@ -1462,22 +1438,32 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
  *
  * The parameters represent what is sent on the PowerBus
  */
-static bool xive_presenter_notify(XiveRouter *xrtr, uint8_t format,
+static bool xive_presenter_notify(uint8_t format,
                                   uint8_t nvt_blk, uint32_t nvt_idx,
                                   bool cam_ignore, uint8_t priority,
                                   uint32_t logic_serv)
 {
+    XiveFabric *xfb = XIVE_FABRIC(qdev_get_machine());
+    XiveFabricClass *xfc = XIVE_FABRIC_GET_CLASS(xfb);
     XiveTCTXMatch match = { .tctx = NULL, .ring = 0 };
-    bool found;
+    int count;
 
-    found = xive_presenter_match(xrtr, format, nvt_blk, nvt_idx, cam_ignore,
-                                 priority, logic_serv, &match);
-    if (found) {
+    /*
+     * Ask the machine to scan the interrupt controllers for a match
+     */
+    count = xfc->match_nvt(xfb, format, nvt_blk, nvt_idx, cam_ignore,

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1464

Quoted 10 lines
hw/intc/xive.c +7 -0
+                           priority, logic_serv, &match);
+    if (count < 0) {
+        return false;
+    }
+
+    /* handle CPU exception delivery */
+    if (count) {
         ipb_update(&match.tctx->regs[match.ring], priority);
         xive_tctx_notify(match.tctx, match.ring);
     }

... in an else block here ^^ ?

Quoted 1 line
Quoted 5 lines

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1466

Quoted 2 lines
hw/intc/xive.c +1 -1
-    return found;
+    return count;

Implicit cast is ok I guess, but !!count would ensure no paranoid compiler ever complains.

yes.

Thanks,

C.

Quoted 14 lines
Quoted 13 lines

}

/*

hw/intc/xive.c +1 -1
@@ -1590,7 +1576,7 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
         return;
     }
 
-    found = xive_presenter_notify(xrtr, format, nvt_blk, nvt_idx,
+    found = xive_presenter_notify(format, nvt_blk, nvt_idx,
                           xive_get_field32(END_W7_F0_IGNORE, end.w7),
                           priority,
                           xive_get_field32(END_W7_F1_LOG_SERVER_ID, end.w7));
#48 lore raw

Re: [PATCH for-5.0 v5 15/23] ppc/xive: Use the XiveFabric and XivePresenter interfaces

Quoted 55 lines · Greg Kurz
Quoted 52 lines · On Thu, 21 Nov 2019 08:40:32 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 51 lines · Greg Kurz
Quoted 47 lines · On Thu, 21 Nov 2019 08:01:44 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 35 lines · Greg Kurz
Quoted 9 lines · On Fri, 15 Nov 2019 17:24:28 +0100 Cédric Le Goater <[email protected]> wrote:

Now that the machines have handlers implementing the XiveFabric and XivePresenter interfaces, remove xive_presenter_match() and make use of the 'match_nvt' handler of the machine.

---
 hw/intc/xive.c | 48 +++++++++++++++++-------------------------------
 1 file changed, 17 insertions(+), 31 deletions(-)

Nice diffstat :)

Quoted 20 lines
hw/intc/xive.c +0 -12
index 1c9e58f8deac..ab62bda85788 100644
@@ -1423,30 +1423,6 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
     return -1;
 }
 
-static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
-                                 uint8_t nvt_blk, uint32_t nvt_idx,
-                                 bool cam_ignore, uint8_t priority,
-                                 uint32_t logic_serv, XiveTCTXMatch *match)
-{
-    XivePresenter *xptr = XIVE_PRESENTER(xrtr);
-    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
-    int count;
-
-    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
-                           priority, logic_serv, match);
-    if (count < 0) {
Quoted 8 lines

hw/intc/xive.c int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx, · line 1443

Quoted 6 lines
hw/intc/xive.c +0 -6
-        return false;
-    }
-
-    if (!match->tctx) {
-        qemu_log_mask(LOG_UNIMP, "XIVE: NVT %x/%x is not dispatched\n",
-                      nvt_blk, nvt_idx);

Maybe keep this trace...

It's in spapr_xive_match_nvt() now.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Not really... spapr_xive_match_nvt() has a trace for the opposite case of duplicate matches:

not that one. The one in spapr.c ... Yes I need to change the name.

... and it seems I cannot memorize a change that was made by the previous patch :-\ Sorry for the noise.

np but this is problem for gdb ! Any suggestion on the name :

spapr_match_nvt()

?

Quoted 1 line

With or without the !!count change:

I will add the !!

C.

Quoted 90 lines
Quoted 86 lines

C.

Quoted 83 lines
            if (match->tctx) {
                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a thread "
                              "context NVT %x/%x\n", nvt_blk, nvt_idx);
                return -1;
            }
Quoted 75 lines
Quoted 47 lines
Quoted 33 lines
hw/intc/xive.c +8 -11
-        return false;
-    }
-
-    return true;
-}
-
 /*
  * This is our simple Xive Presenter Engine model. It is merged in the
  * Router as it does not require an extra object.
@@ -1462,22 +1438,32 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
  *
  * The parameters represent what is sent on the PowerBus
  */
-static bool xive_presenter_notify(XiveRouter *xrtr, uint8_t format,
+static bool xive_presenter_notify(uint8_t format,
                                   uint8_t nvt_blk, uint32_t nvt_idx,
                                   bool cam_ignore, uint8_t priority,
                                   uint32_t logic_serv)
 {
+    XiveFabric *xfb = XIVE_FABRIC(qdev_get_machine());
+    XiveFabricClass *xfc = XIVE_FABRIC_GET_CLASS(xfb);
     XiveTCTXMatch match = { .tctx = NULL, .ring = 0 };
-    bool found;
+    int count;
 
-    found = xive_presenter_match(xrtr, format, nvt_blk, nvt_idx, cam_ignore,
-                                 priority, logic_serv, &match);
-    if (found) {
+    /*
+     * Ask the machine to scan the interrupt controllers for a match
+     */
+    count = xfc->match_nvt(xfb, format, nvt_blk, nvt_idx, cam_ignore,

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1464

Quoted 10 lines
hw/intc/xive.c +7 -0
+                           priority, logic_serv, &match);
+    if (count < 0) {
+        return false;
+    }
+
+    /* handle CPU exception delivery */
+    if (count) {
         ipb_update(&match.tctx->regs[match.ring], priority);
         xive_tctx_notify(match.tctx, match.ring);
     }

... in an else block here ^^ ?

Quoted 1 line
Quoted 5 lines

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1466

Quoted 2 lines
hw/intc/xive.c +1 -1
-    return found;
+    return count;

Implicit cast is ok I guess, but !!count would ensure no paranoid compiler ever complains.

yes.

Thanks,

C.

Quoted 14 lines
Quoted 13 lines

}

/*

hw/intc/xive.c +1 -1
@@ -1590,7 +1576,7 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
         return;
     }
 
-    found = xive_presenter_notify(xrtr, format, nvt_blk, nvt_idx,
+    found = xive_presenter_notify(format, nvt_blk, nvt_idx,
                           xive_get_field32(END_W7_F0_IGNORE, end.w7),
                           priority,
                           xive_get_field32(END_W7_F1_LOG_SERVER_ID, end.w7));
#49 lore raw

Re: [PATCH for-5.0 v5 15/23] ppc/xive: Use the XiveFabric and XivePresenter interfaces

Quoted 62 lines · On Thu, 21 Nov 2019 10:22:38 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 61 lines · Greg Kurz
Quoted 57 lines · On Thu, 21 Nov 2019 08:40:32 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 48 lines · Greg Kurz
Quoted 45 lines · On Thu, 21 Nov 2019 08:01:44 +0100 Cédric Le Goater <[email protected]> wrote:
Quoted 43 lines · Greg Kurz
Quoted 9 lines · On Fri, 15 Nov 2019 17:24:28 +0100 Cédric Le Goater <[email protected]> wrote:

Now that the machines have handlers implementing the XiveFabric and XivePresenter interfaces, remove xive_presenter_match() and make use of the 'match_nvt' handler of the machine.

---
 hw/intc/xive.c | 48 +++++++++++++++++-------------------------------
 1 file changed, 17 insertions(+), 31 deletions(-)

Nice diffstat :)

Quoted 16 lines
hw/intc/xive.c +0 -8
index 1c9e58f8deac..ab62bda85788 100644
@@ -1423,30 +1423,6 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
     return -1;
 }
 
-static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
-                                 uint8_t nvt_blk, uint32_t nvt_idx,
-                                 bool cam_ignore, uint8_t priority,
-                                 uint32_t logic_serv, XiveTCTXMatch *match)
-{
-    XivePresenter *xptr = XIVE_PRESENTER(xrtr);
-    XivePresenterClass *xpc = XIVE_PRESENTER_GET_CLASS(xptr);
-    int count;

hw/intc/xive.c int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx, · line 1443

Quoted 10 lines
hw/intc/xive.c +0 -10
-
-    count = xpc->match_nvt(xptr, format, nvt_blk, nvt_idx, cam_ignore,
-                           priority, logic_serv, match);
-    if (count < 0) {
-        return false;
-    }
-
-    if (!match->tctx) {
-        qemu_log_mask(LOG_UNIMP, "XIVE: NVT %x/%x is not dispatched\n",
-                      nvt_blk, nvt_idx);

Maybe keep this trace...

Quoted 5 lines
Quoted 2 lines

It's in spapr_xive_match_nvt() now.

Not really... spapr_xive_match_nvt() has a trace for the opposite case of duplicate matches:

not that one. The one in spapr.c ... Yes I need to change the name.

Quoted 7 lines
Quoted 2 lines

... and it seems I cannot memorize a change that was made by the previous patch :-\ Sorry for the noise.

np but this is problem for gdb ! Any suggestion on the name :

spapr_match_nvt()

I guess having nvt in the name is enough to identify this as a XIVE related function. It's ok for me.

BTW, the same problem exists in powernv:

$ git grep pnv_xive_match_nvt
hw/intc/pnv_xive.c:static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
hw/intc/pnv_xive.c:    xpc->match_nvt  = pnv_xive_match_nvt;
hw/ppc/pnv.c:static int pnv_xive_match_nvt(XiveFabric *xfb, uint8_t format,
hw/ppc/pnv.c:    xfc->match_nvt = pnv_xive_match_nvt;
Quoted 99 lines

?

Quoted 1 line

With or without the !!count change:

I will add the !!

C.

Quoted 90 lines
Quoted 86 lines

C.

Quoted 83 lines
            if (match->tctx) {
                qemu_log_mask(LOG_GUEST_ERROR, "XIVE: already found a thread "
                              "context NVT %x/%x\n", nvt_blk, nvt_idx);
                return -1;
            }
Quoted 75 lines
Quoted 47 lines
Quoted 33 lines
hw/intc/xive.c +8 -11
-        return false;
-    }
-
-    return true;
-}
-
 /*
  * This is our simple Xive Presenter Engine model. It is merged in the
  * Router as it does not require an extra object.
@@ -1462,22 +1438,32 @@ static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format,
  *
  * The parameters represent what is sent on the PowerBus
  */
-static bool xive_presenter_notify(XiveRouter *xrtr, uint8_t format,
+static bool xive_presenter_notify(uint8_t format,
                                   uint8_t nvt_blk, uint32_t nvt_idx,
                                   bool cam_ignore, uint8_t priority,
                                   uint32_t logic_serv)
 {
+    XiveFabric *xfb = XIVE_FABRIC(qdev_get_machine());
+    XiveFabricClass *xfc = XIVE_FABRIC_GET_CLASS(xfb);
     XiveTCTXMatch match = { .tctx = NULL, .ring = 0 };
-    bool found;
+    int count;
 
-    found = xive_presenter_match(xrtr, format, nvt_blk, nvt_idx, cam_ignore,
-                                 priority, logic_serv, &match);
-    if (found) {
+    /*
+     * Ask the machine to scan the interrupt controllers for a match
+     */
+    count = xfc->match_nvt(xfb, format, nvt_blk, nvt_idx, cam_ignore,

hw/intc/xive.c static bool xive_presenter_match(XiveRouter *xrtr, uint8_t format, · line 1464

Quoted 10 lines
hw/intc/xive.c +7 -0
+                           priority, logic_serv, &match);
+    if (count < 0) {
+        return false;
+    }
+
+    /* handle CPU exception delivery */
+    if (count) {
         ipb_update(&match.tctx->regs[match.ring], priority);
         xive_tctx_notify(match.tctx, match.ring);
     }

... in an else block here ^^ ?

Quoted 1 line
Quoted 5 lines

hw/intc/xive.c

Quoted 2 lines
hw/intc/xive.c +1 -1
-    return found;
+    return count;

Implicit cast is ok I guess, but !!count would ensure no paranoid compiler ever complains.

yes.

Thanks,

C.

Quoted 14 lines
Quoted 13 lines

}

/*

hw/intc/xive.c +1 -1
@@ -1590,7 +1576,7 @@ static void xive_router_end_notify(XiveRouter *xrtr, uint8_t end_blk,
         return;
     }
 
-    found = xive_presenter_notify(xrtr, format, nvt_blk, nvt_idx,
+    found = xive_presenter_notify(format, nvt_blk, nvt_idx,
                           xive_get_field32(END_W7_F0_IGNORE, end.w7),
                           priority,
                           xive_get_field32(END_W7_F1_LOG_SERVER_ID, end.w7));
#50 lore raw

PATCH v5 16/23 ppc/xive: Extend the TIMA operation with a XivePresenter parameter

In reply to Cédric Le Goater

The TIMA operations are performed on behalf of the XIVE IVPE sub-engine (Presenter) on the thread interrupt context registers. The current operations supported by the model are simple and do not require access to the controller but more complex operations will need access to the controller NVT table and to its configuration.

---
 include/hw/ppc/xive.h |  7 +++---
 hw/intc/pnv_xive.c    |  4 +--
 hw/intc/xive.c        | 58 ++++++++++++++++++++++++-------------------
 3 files changed, 38 insertions(+), 31 deletions(-)
include/hw/ppc/xive.h +4 -3
index b00af988779b..97bbcddb381d 100644
@@ -463,9 +463,10 @@ typedef struct XiveENDSource {
 #define XIVE_TM_USER_PAGE       0x3
 
 extern const MemoryRegionOps xive_tm_ops;
-void xive_tctx_tm_write(XiveTCTX *tctx, hwaddr offset, uint64_t value,
-                        unsigned size);
-uint64_t xive_tctx_tm_read(XiveTCTX *tctx, hwaddr offset, unsigned size);
+void xive_tctx_tm_write(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
+                        uint64_t value, unsigned size);
+uint64_t xive_tctx_tm_read(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
+                           unsigned size);
 
 void xive_tctx_pic_print_info(XiveTCTX *tctx, Monitor *mon);
 Object *xive_tctx_create(Object *cpu, XiveRouter *xrtr, Error **errp);
hw/intc/pnv_xive.c +2 -2
index 4c8c6e51c20f..3ee28f00694a 100644
@@ -1436,7 +1436,7 @@ static void xive_tm_indirect_write(void *opaque, hwaddr offset,
 {
     XiveTCTX *tctx = pnv_xive_get_indirect_tctx(PNV_XIVE(opaque));
 
-    xive_tctx_tm_write(tctx, offset, value, size);
+    xive_tctx_tm_write(XIVE_PRESENTER(opaque), tctx, offset, value, size);
 }
 
 static uint64_t xive_tm_indirect_read(void *opaque, hwaddr offset,
@@ -1444,7 +1444,7 @@ static uint64_t xive_tm_indirect_read(void *opaque, hwaddr offset,
 {
     XiveTCTX *tctx = pnv_xive_get_indirect_tctx(PNV_XIVE(opaque));
 
-    return xive_tctx_tm_read(tctx, offset, size);
+    return xive_tctx_tm_read(XIVE_PRESENTER(opaque), tctx, offset, size);
 }
 
 static const MemoryRegionOps xive_tm_indirect_ops = {
hw/intc/xive.c +32 -26

159 lines not on this page · raw message

Signature · 1 line
2.21.0
Reviewed-by on v5 16/23
#51 lore raw

Re: [PATCH for-5.0 v5 16/23] ppc/xive: Extend the TIMA operation with a XivePresenter parameter

Quoted 8 lines · On Fri, 15 Nov 2019 17:24:29 +0100 Cédric Le Goater <[email protected]> wrote:

The TIMA operations are performed on behalf of the XIVE IVPE sub-engine (Presenter) on the thread interrupt context registers. The current operations supported by the model are simple and do not require access to the controller but more complex operations will need access to the controller NVT table and to its configuration.

---

Quoted 221 lines
 include/hw/ppc/xive.h |  7 +++---
 hw/intc/pnv_xive.c    |  4 +--
 hw/intc/xive.c        | 58 ++++++++++++++++++++++++-------------------
 3 files changed, 38 insertions(+), 31 deletions(-)
include/hw/ppc/xive.h +4 -3
index b00af988779b..97bbcddb381d 100644
@@ -463,9 +463,10 @@ typedef struct XiveENDSource {
 #define XIVE_TM_USER_PAGE       0x3
 
 extern const MemoryRegionOps xive_tm_ops;
-void xive_tctx_tm_write(XiveTCTX *tctx, hwaddr offset, uint64_t value,
-                        unsigned size);
-uint64_t xive_tctx_tm_read(XiveTCTX *tctx, hwaddr offset, unsigned size);
+void xive_tctx_tm_write(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
+                        uint64_t value, unsigned size);
+uint64_t xive_tctx_tm_read(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
+                           unsigned size);
 
 void xive_tctx_pic_print_info(XiveTCTX *tctx, Monitor *mon);
 Object *xive_tctx_create(Object *cpu, XiveRouter *xrtr, Error **errp);
hw/intc/pnv_xive.c +2 -2
index 4c8c6e51c20f..3ee28f00694a 100644
@@ -1436,7 +1436,7 @@ static void xive_tm_indirect_write(void *opaque, hwaddr offset,
 {
     XiveTCTX *tctx = pnv_xive_get_indirect_tctx(PNV_XIVE(opaque));
 
-    xive_tctx_tm_write(tctx, offset, value, size);
+    xive_tctx_tm_write(XIVE_PRESENTER(opaque), tctx, offset, value, size);
 }
 
 static uint64_t xive_tm_indirect_read(void *opaque, hwaddr offset,
@@ -1444,7 +1444,7 @@ static uint64_t xive_tm_indirect_read(void *opaque, hwaddr offset,
 {
     XiveTCTX *tctx = pnv_xive_get_indirect_tctx(PNV_XIVE(opaque));
 
-    return xive_tctx_tm_read(tctx, offset, size);
+    return xive_tctx_tm_read(XIVE_PRESENTER(opaque), tctx, offset, size);
 }
 
 static const MemoryRegionOps xive_tm_indirect_ops = {
hw/intc/xive.c +32 -26
index ab62bda85788..a9298783e7d2 100644
@@ -144,19 +144,20 @@ static inline uint32_t xive_tctx_word2(uint8_t *ring)
  * XIVE Thread Interrupt Management Area (TIMA)
  */
 
-static void xive_tm_set_hv_cppr(XiveTCTX *tctx, hwaddr offset,
-                                uint64_t value, unsigned size)
+static void xive_tm_set_hv_cppr(XivePresenter *xptr, XiveTCTX *tctx,
+                                hwaddr offset, uint64_t value, unsigned size)
 {
     xive_tctx_set_cppr(tctx, TM_QW3_HV_PHYS, value & 0xff);
 }
 
-static uint64_t xive_tm_ack_hv_reg(XiveTCTX *tctx, hwaddr offset, unsigned size)
+static uint64_t xive_tm_ack_hv_reg(XivePresenter *xptr, XiveTCTX *tctx,
+                                   hwaddr offset, unsigned size)
 {
     return xive_tctx_accept(tctx, TM_QW3_HV_PHYS);
 }
 
-static uint64_t xive_tm_pull_pool_ctx(XiveTCTX *tctx, hwaddr offset,
-                                      unsigned size)
+static uint64_t xive_tm_pull_pool_ctx(XivePresenter *xptr, XiveTCTX *tctx,
+                                      hwaddr offset, unsigned size)
 {
     uint32_t qw2w2_prev = xive_tctx_word2(&tctx->regs[TM_QW2_HV_POOL]);
     uint32_t qw2w2;
@@ -166,13 +167,14 @@ static uint64_t xive_tm_pull_pool_ctx(XiveTCTX *tctx, hwaddr offset,
     return qw2w2;
 }
 
-static void xive_tm_vt_push(XiveTCTX *tctx, hwaddr offset,
+static void xive_tm_vt_push(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
                             uint64_t value, unsigned size)
 {
     tctx->regs[TM_QW3_HV_PHYS + TM_WORD2] = value & 0xff;
 }
 
-static uint64_t xive_tm_vt_poll(XiveTCTX *tctx, hwaddr offset, unsigned size)
+static uint64_t xive_tm_vt_poll(XivePresenter *xptr, XiveTCTX *tctx,
+                                hwaddr offset, unsigned size)
 {
     return tctx->regs[TM_QW3_HV_PHYS + TM_WORD2] & 0xff;
 }
@@ -315,13 +317,14 @@ static uint64_t xive_tm_raw_read(XiveTCTX *tctx, hwaddr offset, unsigned size)
  * state changes (side effects) in addition to setting/returning the
  * interrupt management area context of the processor thread.
  */
-static uint64_t xive_tm_ack_os_reg(XiveTCTX *tctx, hwaddr offset, unsigned size)
+static uint64_t xive_tm_ack_os_reg(XivePresenter *xptr, XiveTCTX *tctx,
+                                   hwaddr offset, unsigned size)
 {
     return xive_tctx_accept(tctx, TM_QW1_OS);
 }
 
-static void xive_tm_set_os_cppr(XiveTCTX *tctx, hwaddr offset,
-                                uint64_t value, unsigned size)
+static void xive_tm_set_os_cppr(XivePresenter *xptr, XiveTCTX *tctx,
+                                hwaddr offset, uint64_t value, unsigned size)
 {
     xive_tctx_set_cppr(tctx, TM_QW1_OS, value & 0xff);
 }
@@ -330,8 +333,8 @@ static void xive_tm_set_os_cppr(XiveTCTX *tctx, hwaddr offset,
  * Adjust the IPB to allow a CPU to process event queues of other
  * priorities during one physical interrupt cycle.
  */
-static void xive_tm_set_os_pending(XiveTCTX *tctx, hwaddr offset,
-                                   uint64_t value, unsigned size)
+static void xive_tm_set_os_pending(XivePresenter *xptr, XiveTCTX *tctx,
+                                   hwaddr offset, uint64_t value, unsigned size)
 {
     ipb_update(&tctx->regs[TM_QW1_OS], value & 0xff);
     xive_tctx_notify(tctx, TM_QW1_OS);
@@ -366,8 +369,8 @@ static void xive_tctx_set_os_cam(XiveTCTX *tctx, uint32_t qw1w2)
     memcpy(&tctx->regs[TM_QW1_OS + TM_WORD2], &qw1w2, 4);
 }
 
-static uint64_t xive_tm_pull_os_ctx(XiveTCTX *tctx, hwaddr offset,
-                                    unsigned size)
+static uint64_t xive_tm_pull_os_ctx(XivePresenter *xptr, XiveTCTX *tctx,
+                                    hwaddr offset, unsigned size)
 {
     uint32_t qw1w2;
     uint32_t qw1w2_new;
@@ -396,9 +399,11 @@ typedef struct XiveTmOp {
     uint8_t  page_offset;
     uint32_t op_offset;
     unsigned size;
-    void     (*write_handler)(XiveTCTX *tctx, hwaddr offset, uint64_t value,
-                              unsigned size);
-    uint64_t (*read_handler)(XiveTCTX *tctx, hwaddr offset, unsigned size);
+    void     (*write_handler)(XivePresenter *xptr, XiveTCTX *tctx,
+                              hwaddr offset,
+                              uint64_t value, unsigned size);
+    uint64_t (*read_handler)(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
+                             unsigned size);
 } XiveTmOp;
 
 static const XiveTmOp xive_tm_operations[] = {
@@ -444,8 +449,8 @@ static const XiveTmOp *xive_tm_find_op(hwaddr offset, unsigned size, bool write)
 /*
  * TIMA MMIO handlers
  */
-void xive_tctx_tm_write(XiveTCTX *tctx, hwaddr offset, uint64_t value,
-                        unsigned size)
+void xive_tctx_tm_write(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
+                        uint64_t value, unsigned size)
 {
     const XiveTmOp *xto;
 
@@ -462,7 +467,7 @@ void xive_tctx_tm_write(XiveTCTX *tctx, hwaddr offset, uint64_t value,
             qemu_log_mask(LOG_GUEST_ERROR, "XIVE: invalid write access at TIMA "
                           "@%"HWADDR_PRIx"\n", offset);
         } else {
-            xto->write_handler(tctx, offset, value, size);
+            xto->write_handler(xptr, tctx, offset, value, size);
         }
         return;
     }
@@ -472,7 +477,7 @@ void xive_tctx_tm_write(XiveTCTX *tctx, hwaddr offset, uint64_t value,
      */
     xto = xive_tm_find_op(offset, size, true);
     if (xto) {
-        xto->write_handler(tctx, offset, value, size);
+        xto->write_handler(xptr, tctx, offset, value, size);
         return;
     }
 
@@ -482,7 +487,8 @@ void xive_tctx_tm_write(XiveTCTX *tctx, hwaddr offset, uint64_t value,
     xive_tm_raw_write(tctx, offset, value, size);
 }
 
-uint64_t xive_tctx_tm_read(XiveTCTX *tctx, hwaddr offset, unsigned size)
+uint64_t xive_tctx_tm_read(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
+                           unsigned size)
 {
     const XiveTmOp *xto;
 
@@ -500,7 +506,7 @@ uint64_t xive_tctx_tm_read(XiveTCTX *tctx, hwaddr offset, unsigned size)
                           "@%"HWADDR_PRIx"\n", offset);
             return -1;
         }
-        return xto->read_handler(tctx, offset, size);
+        return xto->read_handler(xptr, tctx, offset, size);
     }
 
     /*
@@ -508,7 +514,7 @@ uint64_t xive_tctx_tm_read(XiveTCTX *tctx, hwaddr offset, unsigned size)
      */
     xto = xive_tm_find_op(offset, size, false);
     if (xto) {
-        return xto->read_handler(tctx, offset, size);
+        return xto->read_handler(xptr, tctx, offset, size);
     }
 
     /*
@@ -522,14 +528,14 @@ static void xive_tm_write(void *opaque, hwaddr offset,
 {
     XiveTCTX *tctx = xive_router_get_tctx(XIVE_ROUTER(opaque), current_cpu);
 
-    xive_tctx_tm_write(tctx, offset, value, size);
+    xive_tctx_tm_write(XIVE_PRESENTER(opaque), tctx, offset, value, size);
 }
 
 static uint64_t xive_tm_read(void *opaque, hwaddr offset, unsigned size)
 {
     XiveTCTX *tctx = xive_router_get_tctx(XIVE_ROUTER(opaque), current_cpu);
 
-    return xive_tctx_tm_read(tctx, offset, size);
+    return xive_tctx_tm_read(XIVE_PRESENTER(opaque), tctx, offset, size);
 }
 
 const MemoryRegionOps xive_tm_ops = {
#52 lore raw

PATCH v5 17/23 ppc/pnv: Clarify how the TIMA is accessed on a multichip system

In reply to Cédric Le Goater

The TIMA region gives access to the thread interrupt context registers of a CPU. It is mapped at the same address on all chips and can be accessed by any CPU of the system. To identify the chip from which the access is being done, the PowerBUS uses a 'chip' field in the load/store messages. QEMU does not model these messages, instead, we extract the chip id from the CPU PIR and do a lookup at the machine level to fetch the targeted interrupt controller.

Introduce pnv_get_chip() and pnv_xive_tm_get_xive() helpers to clarify this process in pnv_xive_get_tctx(). The latter will be removed in the subsequent patches but the same principle will be kept.

---
 include/hw/ppc/pnv.h | 13 +++++++++++++
 hw/intc/pnv_xive.c   | 46 ++++++++++++++++++++++++++++----------------
 2 files changed, 42 insertions(+), 17 deletions(-)
include/hw/ppc/pnv.h +13 -0
index 58f4dcc0b71d..9b98b6afa31b 100644
@@ -192,6 +192,19 @@ static inline bool pnv_is_power9(PnvMachineState *pnv)
     return pnv_chip_is_power9(pnv->chips[0]);
 }
 
+static inline PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id)
+{
+    int i;
+
+    for (i = 0; i < pnv->num_chips; i++) {
+        PnvChip *chip = pnv->chips[i];
+        if (chip->chip_id == chip_id) {
+            return chip;
+        }
+    }
+    return NULL;
+}
+
 #define PNV_FDT_ADDR          0x01000000
 #define PNV_TIMEBASE_FREQ     512000000ULL
 
hw/intc/pnv_xive.c +29 -17

66 lines not on this page · raw message

Signature · 1 line
2.21.0
Reviewed-by on v5 17/23
#53 lore raw

Re: [PATCH for-5.0 v5 17/23] ppc/pnv: Clarify how the TIMA is accessed on a multichip system

Quoted 23 lines · On Fri, 15 Nov 2019 17:24:30 +0100 Cédric Le Goater <[email protected]> wrote:

The TIMA region gives access to the thread interrupt context registers of a CPU. It is mapped at the same address on all chips and can be accessed by any CPU of the system. To identify the chip from which the access is being done, the PowerBUS uses a 'chip' field in the load/store messages. QEMU does not model these messages, instead, we extract the chip id from the CPU PIR and do a lookup at the machine level to fetch the targeted interrupt controller.

Introduce pnv_get_chip() and pnv_xive_tm_get_xive() helpers to clarify this process in pnv_xive_get_tctx(). The latter will be removed in the subsequent patches but the same principle will be kept.

---
 include/hw/ppc/pnv.h | 13 +++++++++++++
 hw/intc/pnv_xive.c   | 46 ++++++++++++++++++++++++++++----------------
 2 files changed, 42 insertions(+), 17 deletions(-)
include/hw/ppc/pnv.h +0 -0
index 58f4dcc0b71d..9b98b6afa31b 100644

include/hw/ppc/pnv.h static inline bool pnv_is_power9(PnvMachineState *pnv) · line 195

Quoted 4 lines
include/hw/ppc/pnv.h +1 -0
@@ -192,6 +192,19 @@ static inline bool pnv_is_power9(PnvMachineState *pnv)
     return pnv_chip_is_power9(pnv->chips[0]);
 }
 
+static inline PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id)

I understand this has global scope because it may be used by some other code, eg. PHB4 in your private branch, but I'm not sure it is small enough to deserve the inline qualifier.

Quoted 20 lines
include/hw/ppc/pnv.h +12 -0
+{
+    int i;
+
+    for (i = 0; i < pnv->num_chips; i++) {
+        PnvChip *chip = pnv->chips[i];
+        if (chip->chip_id == chip_id) {
+            return chip;
+        }
+    }
+    return NULL;
+}
+
 #define PNV_FDT_ADDR          0x01000000
 #define PNV_TIMEBASE_FREQ     512000000ULL
 
hw/intc/pnv_xive.c +0 -0
index 3ee28f00694a..d75053d0baad 100644

hw/intc/pnv_xive.c static int cpu_pir(PowerPCCPU *cpu) · line 385

Quoted 8 lines
hw/intc/pnv_xive.c +5 -0
@@ -378,6 +378,12 @@ static int cpu_pir(PowerPCCPU *cpu)
     return env->spr_cb[SPR_PIR].default_value;
 }
 
+static int cpu_chip_id(PowerPCCPU *cpu)
+{
+    int pir = cpu_pir(cpu);
+    return (pir >> 8) & 0x7f;
+}

Just a thought: it would be nice to have all pir<->ids conversion functions grouped by CPU type in pnv.h (inline makes sense in this case) along with the layout comment.

Anyway,

Quoted 59 lines
hw/intc/pnv_xive.c +24 -17
+
 static bool pnv_xive_is_cpu_enabled(PnvXive *xive, PowerPCCPU *cpu)
 {
     int pir = cpu_pir(cpu);
@@ -440,31 +446,37 @@ static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
     return count;
 }
 
+/*
+ * The TIMA MMIO space is shared among the chips and to identify the
+ * chip from which the access is being done, we extract the chip id
+ * from the PIR.
+ */
+static PnvXive *pnv_xive_tm_get_xive(PowerPCCPU *cpu)
+{
+    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
+    PnvChip *chip;
+    PnvXive *xive;
+
+    chip = pnv_get_chip(pnv, cpu_chip_id(cpu));
+    assert(chip);
+    xive = &PNV9_CHIP(chip)->xive;
+
+    if (!pnv_xive_is_cpu_enabled(xive, cpu)) {
+        xive_error(xive, "IC: CPU %x is not enabled", cpu_pir(cpu));
+    }
+    return xive;
+}
+
 static XiveTCTX *pnv_xive_get_tctx(XiveRouter *xrtr, CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
-    XiveTCTX *tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
-    PnvXive *xive = NULL;
-    CPUPPCState *env = &cpu->env;
-    int pir = env->spr_cb[SPR_PIR].default_value;
+    PnvXive *xive = pnv_xive_tm_get_xive(cpu);
 
-    /*
-     * Perform an extra check on the HW thread enablement.
-     *
-     * The TIMA is shared among the chips and to identify the chip
-     * from which the access is being done, we extract the chip id
-     * from the PIR.
-     */
-    xive = pnv_xive_get_ic((pir >> 8) & 0xf);
     if (!xive) {
         return NULL;
     }
 
-    if (!(xive->regs[PC_THREAD_EN_REG0 >> 3] & PPC_BIT(pir & 0x3f))) {
-        xive_error(PNV_XIVE(xrtr), "IC: CPU %x is not enabled", pir);
-    }
-
-    return tctx;
+    return XIVE_TCTX(pnv_cpu_state(cpu)->intc);
 }
 
 /*
#54 lore raw

PATCH v5 18/23 ppc/xive: Move the TIMA operations to the controller model

In reply to Cédric Le Goater

On the P9 Processor, the thread interrupt context registers of a CPU can be accessed "directly" when by load/store from the CPU or "indirectly" by the IC through an indirect TIMA page. This requires to configure first the PC_TCTXT_INDIRx registers.

Today, we rely on the get_tctx() handler to deduce from the CPU PIR the chip from which the TIMA access is being done. By handling the TIMA memory ops under the interrupt controller model of each machine, we can uniformize the TIMA direct and indirect ops under PowerNV. We can also check that the CPUs have been enabled in the XIVE controller.

This prepares ground for the future versions of XIVE.

---
 include/hw/ppc/xive.h |  1 -
 hw/intc/pnv_xive.c    | 35 ++++++++++++++++++++++++++++++++++-
 hw/intc/spapr_xive.c  | 33 +++++++++++++++++++++++++++++++--
 hw/intc/xive.c        | 29 -----------------------------
 4 files changed, 65 insertions(+), 33 deletions(-)
include/hw/ppc/xive.h +0 -1
index 97bbcddb381d..dcf897451589 100644
@@ -462,7 +462,6 @@ typedef struct XiveENDSource {
 #define XIVE_TM_OS_PAGE         0x2
 #define XIVE_TM_USER_PAGE       0x3
 
-extern const MemoryRegionOps xive_tm_ops;
 void xive_tctx_tm_write(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
                         uint64_t value, unsigned size);
 uint64_t xive_tctx_tm_read(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
hw/intc/pnv_xive.c +34 -1
index d75053d0baad..4501c671d8df 100644
@@ -1473,6 +1473,39 @@ static const MemoryRegionOps xive_tm_indirect_ops = {
     },
 };
 
+static void pnv_xive_tm_write(void *opaque, hwaddr offset,
+                              uint64_t value, unsigned size)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(current_cpu);
+    PnvXive *xive = pnv_xive_tm_get_xive(cpu);
+    XiveTCTX *tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
+
+    xive_tctx_tm_write(XIVE_PRESENTER(xive), tctx, offset, value, size);
+}
+
+static uint64_t pnv_xive_tm_read(void *opaque, hwaddr offset, unsigned size)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(current_cpu);
+    PnvXive *xive = pnv_xive_tm_get_xive(cpu);
+    XiveTCTX *tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
+
+    return xive_tctx_tm_read(XIVE_PRESENTER(xive), tctx, offset, size);
+}
+
+const MemoryRegionOps pnv_xive_tm_ops = {
+    .read = pnv_xive_tm_read,
+    .write = pnv_xive_tm_write,
+    .endianness = DEVICE_BIG_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+    .impl = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+};
+
 /*
  * Interrupt controller XSCOM region.
  */
@@ -1845,7 +1878,7 @@ static void pnv_xive_realize(DeviceState *dev, Error **errp)
                           "xive-pc", PNV9_XIVE_PC_SIZE);
 
     /* Thread Interrupt Management Area (Direct) */
-    memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &xive_tm_ops,
+    memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &pnv_xive_tm_ops,
                           xive, "xive-tima", PNV9_XIVE_TM_SIZE);
 
     qemu_register_reset(pnv_xive_reset, dev);
hw/intc/spapr_xive.c +31 -2
index bb3b2dfdb77f..6292da58f62c 100644
@@ -205,6 +205,35 @@ void spapr_xive_mmio_set_enabled(SpaprXive *xive, bool enable)
     memory_region_set_enabled(&xive->end_source.esb_mmio, false);
 }
 
+static void spapr_xive_tm_write(void *opaque, hwaddr offset,
+                          uint64_t value, unsigned size)
+{
+    XiveTCTX *tctx = spapr_cpu_state(POWERPC_CPU(current_cpu))->tctx;
+
+    xive_tctx_tm_write(XIVE_PRESENTER(opaque), tctx, offset, value, size);
+}
+
+static uint64_t spapr_xive_tm_read(void *opaque, hwaddr offset, unsigned size)
+{
+    XiveTCTX *tctx = spapr_cpu_state(POWERPC_CPU(current_cpu))->tctx;
+
+    return xive_tctx_tm_read(XIVE_PRESENTER(opaque), tctx, offset, size);
+}
+
+const MemoryRegionOps spapr_xive_tm_ops = {
+    .read = spapr_xive_tm_read,
+    .write = spapr_xive_tm_write,
+    .endianness = DEVICE_BIG_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+    .impl = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+};
+
 static void spapr_xive_end_reset(XiveEND *end)
 {
     memset(end, 0, sizeof(*end));
@@ -314,8 +343,8 @@ static void spapr_xive_realize(DeviceState *dev, Error **errp)
     qemu_register_reset(spapr_xive_reset, dev);
 
     /* TIMA initialization */
-    memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &xive_tm_ops, xive,
-                          "xive.tima", 4ull << TM_SHIFT);
+    memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &spapr_xive_tm_ops,
+                          xive, "xive.tima", 4ull << TM_SHIFT);
     sysbus_init_mmio(SYS_BUS_DEVICE(xive), &xive->tm_mmio);
 
     /*
hw/intc/xive.c +0 -29
index a9298783e7d2..ab779c4c2a0f 100644
@@ -523,35 +523,6 @@ uint64_t xive_tctx_tm_read(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
     return xive_tm_raw_read(tctx, offset, size);
 }
 
-static void xive_tm_write(void *opaque, hwaddr offset,
-                          uint64_t value, unsigned size)
-{
-    XiveTCTX *tctx = xive_router_get_tctx(XIVE_ROUTER(opaque), current_cpu);
-
-    xive_tctx_tm_write(XIVE_PRESENTER(opaque), tctx, offset, value, size);
-}
-
-static uint64_t xive_tm_read(void *opaque, hwaddr offset, unsigned size)
-{
-    XiveTCTX *tctx = xive_router_get_tctx(XIVE_ROUTER(opaque), current_cpu);
-
-    return xive_tctx_tm_read(XIVE_PRESENTER(opaque), tctx, offset, size);
-}
-
-const MemoryRegionOps xive_tm_ops = {
-    .read = xive_tm_read,
-    .write = xive_tm_write,
-    .endianness = DEVICE_BIG_ENDIAN,
-    .valid = {
-        .min_access_size = 1,
-        .max_access_size = 8,
-    },
-    .impl = {
-        .min_access_size = 1,
-        .max_access_size = 8,
-    },
-};
-
 static char *xive_tctx_ring_print(uint8_t *ring)
 {
     uint32_t w2 = xive_tctx_word2(ring);
Signature · 1 line
2.21.0
Reviewed-by on v5 18/23
#55 lore raw

Re: [PATCH for-5.0 v5 18/23] ppc/xive: Move the TIMA operations to the controller model

Quoted 11 lines · On Fri, 15 Nov 2019 17:24:31 +0100 Cédric Le Goater <[email protected]> wrote:

On the P9 Processor, the thread interrupt context registers of a CPU can be accessed "directly" when by load/store from the CPU or "indirectly" by the IC through an indirect TIMA page. This requires to configure first the PC_TCTXT_INDIRx registers.

Today, we rely on the get_tctx() handler to deduce from the CPU PIR the chip from which the TIMA access is being done. By handling the TIMA memory ops under the interrupt controller model of each machine, we can uniformize the TIMA direct and indirect ops under PowerNV. We can also check that the CPUs have been enabled in the XIVE controller.

Quoted 4 lines

This prepares ground for the future versions of XIVE.

---

Quoted 162 lines
 include/hw/ppc/xive.h |  1 -
 hw/intc/pnv_xive.c    | 35 ++++++++++++++++++++++++++++++++++-
 hw/intc/spapr_xive.c  | 33 +++++++++++++++++++++++++++++++--
 hw/intc/xive.c        | 29 -----------------------------
 4 files changed, 65 insertions(+), 33 deletions(-)
include/hw/ppc/xive.h +0 -1
index 97bbcddb381d..dcf897451589 100644
@@ -462,7 +462,6 @@ typedef struct XiveENDSource {
 #define XIVE_TM_OS_PAGE         0x2
 #define XIVE_TM_USER_PAGE       0x3
 
-extern const MemoryRegionOps xive_tm_ops;
 void xive_tctx_tm_write(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
                         uint64_t value, unsigned size);
 uint64_t xive_tctx_tm_read(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
hw/intc/pnv_xive.c +34 -1
index d75053d0baad..4501c671d8df 100644
@@ -1473,6 +1473,39 @@ static const MemoryRegionOps xive_tm_indirect_ops = {
     },
 };
 
+static void pnv_xive_tm_write(void *opaque, hwaddr offset,
+                              uint64_t value, unsigned size)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(current_cpu);
+    PnvXive *xive = pnv_xive_tm_get_xive(cpu);
+    XiveTCTX *tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
+
+    xive_tctx_tm_write(XIVE_PRESENTER(xive), tctx, offset, value, size);
+}
+
+static uint64_t pnv_xive_tm_read(void *opaque, hwaddr offset, unsigned size)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(current_cpu);
+    PnvXive *xive = pnv_xive_tm_get_xive(cpu);
+    XiveTCTX *tctx = XIVE_TCTX(pnv_cpu_state(cpu)->intc);
+
+    return xive_tctx_tm_read(XIVE_PRESENTER(xive), tctx, offset, size);
+}
+
+const MemoryRegionOps pnv_xive_tm_ops = {
+    .read = pnv_xive_tm_read,
+    .write = pnv_xive_tm_write,
+    .endianness = DEVICE_BIG_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+    .impl = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+};
+
 /*
  * Interrupt controller XSCOM region.
  */
@@ -1845,7 +1878,7 @@ static void pnv_xive_realize(DeviceState *dev, Error **errp)
                           "xive-pc", PNV9_XIVE_PC_SIZE);
 
     /* Thread Interrupt Management Area (Direct) */
-    memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &xive_tm_ops,
+    memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &pnv_xive_tm_ops,
                           xive, "xive-tima", PNV9_XIVE_TM_SIZE);
 
     qemu_register_reset(pnv_xive_reset, dev);
hw/intc/spapr_xive.c +31 -2
index bb3b2dfdb77f..6292da58f62c 100644
@@ -205,6 +205,35 @@ void spapr_xive_mmio_set_enabled(SpaprXive *xive, bool enable)
     memory_region_set_enabled(&xive->end_source.esb_mmio, false);
 }
 
+static void spapr_xive_tm_write(void *opaque, hwaddr offset,
+                          uint64_t value, unsigned size)
+{
+    XiveTCTX *tctx = spapr_cpu_state(POWERPC_CPU(current_cpu))->tctx;
+
+    xive_tctx_tm_write(XIVE_PRESENTER(opaque), tctx, offset, value, size);
+}
+
+static uint64_t spapr_xive_tm_read(void *opaque, hwaddr offset, unsigned size)
+{
+    XiveTCTX *tctx = spapr_cpu_state(POWERPC_CPU(current_cpu))->tctx;
+
+    return xive_tctx_tm_read(XIVE_PRESENTER(opaque), tctx, offset, size);
+}
+
+const MemoryRegionOps spapr_xive_tm_ops = {
+    .read = spapr_xive_tm_read,
+    .write = spapr_xive_tm_write,
+    .endianness = DEVICE_BIG_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+    .impl = {
+        .min_access_size = 1,
+        .max_access_size = 8,
+    },
+};
+
 static void spapr_xive_end_reset(XiveEND *end)
 {
     memset(end, 0, sizeof(*end));
@@ -314,8 +343,8 @@ static void spapr_xive_realize(DeviceState *dev, Error **errp)
     qemu_register_reset(spapr_xive_reset, dev);
 
     /* TIMA initialization */
-    memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &xive_tm_ops, xive,
-                          "xive.tima", 4ull << TM_SHIFT);
+    memory_region_init_io(&xive->tm_mmio, OBJECT(xive), &spapr_xive_tm_ops,
+                          xive, "xive.tima", 4ull << TM_SHIFT);
     sysbus_init_mmio(SYS_BUS_DEVICE(xive), &xive->tm_mmio);
 
     /*
hw/intc/xive.c +0 -29
index a9298783e7d2..ab779c4c2a0f 100644
@@ -523,35 +523,6 @@ uint64_t xive_tctx_tm_read(XivePresenter *xptr, XiveTCTX *tctx, hwaddr offset,
     return xive_tm_raw_read(tctx, offset, size);
 }
 
-static void xive_tm_write(void *opaque, hwaddr offset,
-                          uint64_t value, unsigned size)
-{
-    XiveTCTX *tctx = xive_router_get_tctx(XIVE_ROUTER(opaque), current_cpu);
-
-    xive_tctx_tm_write(XIVE_PRESENTER(opaque), tctx, offset, value, size);
-}
-
-static uint64_t xive_tm_read(void *opaque, hwaddr offset, unsigned size)
-{
-    XiveTCTX *tctx = xive_router_get_tctx(XIVE_ROUTER(opaque), current_cpu);
-
-    return xive_tctx_tm_read(XIVE_PRESENTER(opaque), tctx, offset, size);
-}
-
-const MemoryRegionOps xive_tm_ops = {
-    .read = xive_tm_read,
-    .write = xive_tm_write,
-    .endianness = DEVICE_BIG_ENDIAN,
-    .valid = {
-        .min_access_size = 1,
-        .max_access_size = 8,
-    },
-    .impl = {
-        .min_access_size = 1,
-        .max_access_size = 8,
-    },
-};
-
 static char *xive_tctx_ring_print(uint8_t *ring)
 {
     uint32_t w2 = xive_tctx_word2(ring);
#56 lore raw

PATCH v5 19/23 ppc/xive: Remove the get_tctx() XiveRouter handler

In reply to Cédric Le Goater

It is now unused.

---
 include/hw/ppc/xive.h |  2 --
 hw/intc/pnv_xive.c    | 13 -------------
 hw/intc/spapr_xive.c  |  8 --------
 hw/intc/xive.c        |  7 -------
 4 files changed, 30 deletions(-)
include/hw/ppc/xive.h +0 -2
index dcf897451589..24315480e7c2 100644
@@ -351,7 +351,6 @@ typedef struct XiveRouterClass {
                    XiveNVT *nvt);
     int (*write_nvt)(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
                      XiveNVT *nvt, uint8_t word_number);
-    XiveTCTX *(*get_tctx)(XiveRouter *xrtr, CPUState *cs);
 } XiveRouterClass;
 
 int xive_router_get_eas(XiveRouter *xrtr, uint8_t eas_blk, uint32_t eas_idx,
@@ -364,7 +363,6 @@ int xive_router_get_nvt(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
                         XiveNVT *nvt);
 int xive_router_write_nvt(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
                           XiveNVT *nvt, uint8_t word_number);
-XiveTCTX *xive_router_get_tctx(XiveRouter *xrtr, CPUState *cs);
 void xive_router_notify(XiveNotifier *xn, uint32_t lisn);
 
 /*
hw/intc/pnv_xive.c +0 -13
index 4501c671d8df..2e568721b44e 100644
@@ -467,18 +467,6 @@ static PnvXive *pnv_xive_tm_get_xive(PowerPCCPU *cpu)
     return xive;
 }
 
-static XiveTCTX *pnv_xive_get_tctx(XiveRouter *xrtr, CPUState *cs)
-{
-    PowerPCCPU *cpu = POWERPC_CPU(cs);
-    PnvXive *xive = pnv_xive_tm_get_xive(cpu);
-
-    if (!xive) {
-        return NULL;
-    }
-
-    return XIVE_TCTX(pnv_cpu_state(cpu)->intc);
-}
-
 /*
  * The internal sources (IPIs) of the interrupt controller have no
  * knowledge of the XIVE chip on which they reside. Encode the block
@@ -1936,7 +1924,6 @@ static void pnv_xive_class_init(ObjectClass *klass, void *data)
     xrc->write_end = pnv_xive_write_end;
     xrc->get_nvt = pnv_xive_get_nvt;
     xrc->write_nvt = pnv_xive_write_nvt;
-    xrc->get_tctx = pnv_xive_get_tctx;
 
     xnc->notify = pnv_xive_notify;
     xpc->match_nvt  = pnv_xive_match_nvt;
hw/intc/spapr_xive.c +0 -8
index 6292da58f62c..1542cef91878 100644
@@ -427,13 +427,6 @@ static int spapr_xive_write_nvt(XiveRouter *xrtr, uint8_t nvt_blk,
     g_assert_not_reached();
 }
 
-static XiveTCTX *spapr_xive_get_tctx(XiveRouter *xrtr, CPUState *cs)
-{
-    PowerPCCPU *cpu = POWERPC_CPU(cs);
-
-    return spapr_cpu_state(cpu)->tctx;
-}
-
 static int spapr_xive_match_nvt(XivePresenter *xptr, uint8_t format,
                                 uint8_t nvt_blk, uint32_t nvt_idx,
                                 bool cam_ignore, uint8_t priority,
@@ -771,7 +764,6 @@ static void spapr_xive_class_init(ObjectClass *klass, void *data)
     xrc->write_end = spapr_xive_write_end;
     xrc->get_nvt = spapr_xive_get_nvt;
     xrc->write_nvt = spapr_xive_write_nvt;
-    xrc->get_tctx = spapr_xive_get_tctx;
 
     sicc->activate = spapr_xive_activate;
     sicc->deactivate = spapr_xive_deactivate;
hw/intc/xive.c +0 -7
index ab779c4c2a0f..e576a1e4ba9c 100644
@@ -1317,13 +1317,6 @@ int xive_router_write_nvt(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
    return xrc->write_nvt(xrtr, nvt_blk, nvt_idx, nvt, word_number);
 }
 
-XiveTCTX *xive_router_get_tctx(XiveRouter *xrtr, CPUState *cs)
-{
-    XiveRouterClass *xrc = XIVE_ROUTER_GET_CLASS(xrtr);
-
-    return xrc->get_tctx(xrtr, cs);
-}
-
 /*
  * Encode the HW CAM line in the block group mode format :
  *
Signature · 1 line
2.21.0
Reviewed-by on v5 19/23
#57 lore raw

Re: [PATCH for-5.0 v5 19/23] ppc/xive: Remove the get_tctx() XiveRouter handler

Quoted 4 lines · On Fri, 15 Nov 2019 17:24:32 +0100 Cédric Le Goater <[email protected]> wrote:

It is now unused.

---

Quoted 101 lines
 include/hw/ppc/xive.h |  2 --
 hw/intc/pnv_xive.c    | 13 -------------
 hw/intc/spapr_xive.c  |  8 --------
 hw/intc/xive.c        |  7 -------
 4 files changed, 30 deletions(-)
include/hw/ppc/xive.h +0 -2
index dcf897451589..24315480e7c2 100644
@@ -351,7 +351,6 @@ typedef struct XiveRouterClass {
                    XiveNVT *nvt);
     int (*write_nvt)(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
                      XiveNVT *nvt, uint8_t word_number);
-    XiveTCTX *(*get_tctx)(XiveRouter *xrtr, CPUState *cs);
 } XiveRouterClass;
 
 int xive_router_get_eas(XiveRouter *xrtr, uint8_t eas_blk, uint32_t eas_idx,
@@ -364,7 +363,6 @@ int xive_router_get_nvt(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
                         XiveNVT *nvt);
 int xive_router_write_nvt(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
                           XiveNVT *nvt, uint8_t word_number);
-XiveTCTX *xive_router_get_tctx(XiveRouter *xrtr, CPUState *cs);
 void xive_router_notify(XiveNotifier *xn, uint32_t lisn);
 
 /*
hw/intc/pnv_xive.c +0 -13
index 4501c671d8df..2e568721b44e 100644
@@ -467,18 +467,6 @@ static PnvXive *pnv_xive_tm_get_xive(PowerPCCPU *cpu)
     return xive;
 }
 
-static XiveTCTX *pnv_xive_get_tctx(XiveRouter *xrtr, CPUState *cs)
-{
-    PowerPCCPU *cpu = POWERPC_CPU(cs);
-    PnvXive *xive = pnv_xive_tm_get_xive(cpu);
-
-    if (!xive) {
-        return NULL;
-    }
-
-    return XIVE_TCTX(pnv_cpu_state(cpu)->intc);
-}
-
 /*
  * The internal sources (IPIs) of the interrupt controller have no
  * knowledge of the XIVE chip on which they reside. Encode the block
@@ -1936,7 +1924,6 @@ static void pnv_xive_class_init(ObjectClass *klass, void *data)
     xrc->write_end = pnv_xive_write_end;
     xrc->get_nvt = pnv_xive_get_nvt;
     xrc->write_nvt = pnv_xive_write_nvt;
-    xrc->get_tctx = pnv_xive_get_tctx;
 
     xnc->notify = pnv_xive_notify;
     xpc->match_nvt  = pnv_xive_match_nvt;
hw/intc/spapr_xive.c +0 -8
index 6292da58f62c..1542cef91878 100644
@@ -427,13 +427,6 @@ static int spapr_xive_write_nvt(XiveRouter *xrtr, uint8_t nvt_blk,
     g_assert_not_reached();
 }
 
-static XiveTCTX *spapr_xive_get_tctx(XiveRouter *xrtr, CPUState *cs)
-{
-    PowerPCCPU *cpu = POWERPC_CPU(cs);
-
-    return spapr_cpu_state(cpu)->tctx;
-}
-
 static int spapr_xive_match_nvt(XivePresenter *xptr, uint8_t format,
                                 uint8_t nvt_blk, uint32_t nvt_idx,
                                 bool cam_ignore, uint8_t priority,
@@ -771,7 +764,6 @@ static void spapr_xive_class_init(ObjectClass *klass, void *data)
     xrc->write_end = spapr_xive_write_end;
     xrc->get_nvt = spapr_xive_get_nvt;
     xrc->write_nvt = spapr_xive_write_nvt;
-    xrc->get_tctx = spapr_xive_get_tctx;
 
     sicc->activate = spapr_xive_activate;
     sicc->deactivate = spapr_xive_deactivate;
hw/intc/xive.c +0 -7
index ab779c4c2a0f..e576a1e4ba9c 100644
@@ -1317,13 +1317,6 @@ int xive_router_write_nvt(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
    return xrc->write_nvt(xrtr, nvt_blk, nvt_idx, nvt, word_number);
 }
 
-XiveTCTX *xive_router_get_tctx(XiveRouter *xrtr, CPUState *cs)
-{
-    XiveRouterClass *xrc = XIVE_ROUTER_GET_CLASS(xrtr);
-
-    return xrc->get_tctx(xrtr, cs);
-}
-
 /*
  * Encode the HW CAM line in the block group mode format :
  *
#58 lore raw

PATCH v5 20/23 ppc/xive: Introduce a xive_tctx_ipb_update() helper

In reply to Cédric Le Goater

We will use it to resend missed interrupts when a vCPU context is pushed on a HW thread.

---
 include/hw/ppc/xive.h |  1 +
 hw/intc/xive.c        | 21 +++++++++++----------
 2 files changed, 12 insertions(+), 10 deletions(-)
include/hw/ppc/xive.h +1 -0
index 24315480e7c2..9c0bf2c301e2 100644
@@ -469,6 +469,7 @@ void xive_tctx_pic_print_info(XiveTCTX *tctx, Monitor *mon);
 Object *xive_tctx_create(Object *cpu, XiveRouter *xrtr, Error **errp);
 void xive_tctx_reset(XiveTCTX *tctx);
 void xive_tctx_destroy(XiveTCTX *tctx);
+void xive_tctx_ipb_update(XiveTCTX *tctx, uint8_t ring, uint8_t ipb);
 
 /*
  * KVM XIVE device helpers
hw/intc/xive.c +11 -10
index e576a1e4ba9c..840ab2a555e1 100644
@@ -47,12 +47,6 @@ static uint8_t ipb_to_pipr(uint8_t ibp)
     return ibp ? clz32((uint32_t)ibp << 24) : 0xff;
 }
 
-static void ipb_update(uint8_t *regs, uint8_t priority)
-{
-    regs[TM_IPB] |= priority_to_ipb(priority);
-    regs[TM_PIPR] = ipb_to_pipr(regs[TM_IPB]);
-}
-
 static uint8_t exception_mask(uint8_t ring)
 {
     switch (ring) {
@@ -135,6 +129,15 @@ static void xive_tctx_set_cppr(XiveTCTX *tctx, uint8_t ring, uint8_t cppr)
     xive_tctx_notify(tctx, ring);
 }
 
+void xive_tctx_ipb_update(XiveTCTX *tctx, uint8_t ring, uint8_t ipb)
+{
+    uint8_t *regs = &tctx->regs[ring];
+
+    regs[TM_IPB] |= ipb;
+    regs[TM_PIPR] = ipb_to_pipr(regs[TM_IPB]);
+    xive_tctx_notify(tctx, ring);
+}
+
 static inline uint32_t xive_tctx_word2(uint8_t *ring)
 {
     return *((uint32_t *) &ring[TM_WORD2]);
@@ -336,8 +339,7 @@ static void xive_tm_set_os_cppr(XivePresenter *xptr, XiveTCTX *tctx,
 static void xive_tm_set_os_pending(XivePresenter *xptr, XiveTCTX *tctx,
                                    hwaddr offset, uint64_t value, unsigned size)
 {
-    ipb_update(&tctx->regs[TM_QW1_OS], value & 0xff);
-    xive_tctx_notify(tctx, TM_QW1_OS);
+    xive_tctx_ipb_update(tctx, TM_QW1_OS, priority_to_ipb(value & 0xff));
 }
 
 static void xive_os_cam_decode(uint32_t cam, uint8_t *nvt_blk,
@@ -1429,8 +1431,7 @@ static bool xive_presenter_notify(uint8_t format,
 
     /* handle CPU exception delivery */
     if (count) {
-        ipb_update(&match.tctx->regs[match.ring], priority);
-        xive_tctx_notify(match.tctx, match.ring);
+        xive_tctx_ipb_update(match.tctx, match.ring, priority_to_ipb(priority));
     }
 
     return count;
Signature · 1 line
2.21.0
#59 lore raw

PATCH v5 21/23 ppc/xive: Synthesize interrupt from the saved IPB in the NVT

In reply to Cédric Le Goater

When a vCPU is dispatched on a HW thread, its context is pushed in the thread registers and it is activated by setting the VO bit in the CAM line word2. The HW grabs the associated NVT, pulls the IPB bits and merges them with the IPB of the new context. If interrupts were missed while the vCPU was not dispatched, these are synthesized in this sequence.

---
 hw/intc/xive.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
hw/intc/xive.c +52 -0
index 840ab2a555e1..ce904b0b5ab4 100644
@@ -393,6 +393,57 @@ static uint64_t xive_tm_pull_os_ctx(XivePresenter *xptr, XiveTCTX *tctx,
     return qw1w2;
 }
 
+static void xive_tctx_need_resend(XiveRouter *xrtr, XiveTCTX *tctx,
+                                  uint8_t nvt_blk, uint32_t nvt_idx)
+{
+    XiveNVT nvt;
+    uint8_t ipb;
+
+    /*
+     * Grab the associated NVT to pull the pending bits, and merge
+     * them with the IPB of the thread interrupt context registers
+     */
+    if (xive_router_get_nvt(xrtr, nvt_blk, nvt_idx, &nvt)) {
+        qemu_log_mask(LOG_GUEST_ERROR, "XIVE: invalid NVT %x/%x\n",
+                          nvt_blk, nvt_idx);
+        return;
+    }
+
+    ipb = xive_get_field32(NVT_W4_IPB, nvt.w4);
+
+    if (ipb) {
+        /* Reset the NVT value */
+        nvt.w4 = xive_set_field32(NVT_W4_IPB, nvt.w4, 0);
+        xive_router_write_nvt(xrtr, nvt_blk, nvt_idx, &nvt, 4);
+
+        /* Merge in current context */
+        xive_tctx_ipb_update(tctx, TM_QW1_OS, ipb);
+    }
+}
+
+/*
+ * Updating the OS CAM line can trigger a resend of interrupt
+ */
+static void xive_tm_push_os_ctx(XivePresenter *xptr, XiveTCTX *tctx,
+                                hwaddr offset, uint64_t value, unsigned size)
+{
+    uint32_t cam = value;
+    uint32_t qw1w2 = cpu_to_be32(cam);
+    uint8_t nvt_blk;
+    uint32_t nvt_idx;
+    bool vo;
+
+    xive_os_cam_decode(cam, &nvt_blk, &nvt_idx, &vo);
+
+    /* First update the registers */
+    xive_tctx_set_os_cam(tctx, qw1w2);
+
+    /* Check the interrupt pending bits */
+    if (vo) {
+        xive_tctx_need_resend(XIVE_ROUTER(xptr), tctx, nvt_blk, nvt_idx);
+    }
+}
+
 /*
  * Define a mapping of "special" operations depending on the TIMA page
  * offset and the size of the operation.
@@ -414,6 +465,7 @@ static const XiveTmOp xive_tm_operations[] = {
      * effects
      */
     { XIVE_TM_OS_PAGE, TM_QW1_OS + TM_CPPR,   1, xive_tm_set_os_cppr, NULL },
+    { XIVE_TM_HV_PAGE, TM_QW1_OS + TM_WORD2,     4, xive_tm_push_os_ctx, NULL },
     { XIVE_TM_HV_PAGE, TM_QW3_HV_PHYS + TM_CPPR, 1, xive_tm_set_hv_cppr, NULL },
     { XIVE_TM_HV_PAGE, TM_QW3_HV_PHYS + TM_WORD2, 1, xive_tm_vt_push, NULL },
     { XIVE_TM_HV_PAGE, TM_QW3_HV_PHYS + TM_WORD2, 1, NULL, xive_tm_vt_poll },
Signature · 1 line
2.21.0
#60 lore raw

PATCH v5 22/23 ppc/pnv: Introduce a pnv_xive_block_id() helper

In reply to Cédric Le Goater

When PC_TCTXT_CHIPID_OVERRIDE is configured, the PC_TCTXT_CHIPID field overrides the hardwired chip ID in the Powerbus operations and for CAM compares. This is typically used in the one block-per-chip configuration to associate a unique block id number to each IC of the system.

Simplify the model with a pnv_xive_block_id() helper and remove 'tctx_chipid' which becomes useless.

---
 include/hw/ppc/pnv_xive.h |  3 --
 hw/intc/pnv_xive.c        | 68 ++++++++++++++++++++-------------------
 2 files changed, 35 insertions(+), 36 deletions(-)
include/hw/ppc/pnv_xive.h +0 -3
index 4fdaa9247d65..f4c7caad40ee 100644
@@ -72,9 +72,6 @@ typedef struct PnvXive {
     /* Interrupt controller registers */
     uint64_t      regs[0x300];
 
-    /* Can be configured by FW */
-    uint32_t      tctx_chipid;
-
     /*
      * Virtual Structure Descriptor tables : EAT, SBE, ENDT, NVTT, IRQ
      * These are in a SRAM protected by ECC.
hw/intc/pnv_xive.c +35 -33

147 lines not on this page · raw message

Signature · 1 line
2.21.0
#61 lore raw

PATCH v5 23/23 ppc/pnv: Extend XiveRouter with a get_block_id() handler

In reply to Cédric Le Goater

When doing CAM line compares, fetch the block id from the interrupt controller which can have set the PC_TCTXT_CHIPID field.

---
 include/hw/ppc/xive.h |  2 +-
 hw/intc/pnv_xive.c    |  6 ++++++
 hw/intc/spapr_xive.c  |  6 ++++++
 hw/intc/xive.c        | 21 ++++++++++++++++-----
 4 files changed, 29 insertions(+), 6 deletions(-)
include/hw/ppc/xive.h +1 -1
index 9c0bf2c301e2..1b7b89098f71 100644
@@ -351,6 +351,7 @@ typedef struct XiveRouterClass {
                    XiveNVT *nvt);
     int (*write_nvt)(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
                      XiveNVT *nvt, uint8_t word_number);
+    uint8_t (*get_block_id)(XiveRouter *xrtr);
 } XiveRouterClass;
 
 int xive_router_get_eas(XiveRouter *xrtr, uint8_t eas_blk, uint32_t eas_idx,
@@ -431,7 +432,6 @@ typedef struct XiveENDSource {
     DeviceState parent;
 
     uint32_t        nr_ends;
-    uint8_t         block_id;
 
     /* ESB memory region */
     uint32_t        esb_shift;
hw/intc/pnv_xive.c +6 -0
index 93c27cce568b..c7c2fe3a7b5f 100644
@@ -466,6 +466,11 @@ static int pnv_xive_match_nvt(XivePresenter *xptr, uint8_t format,
     return count;
 }
 
+static uint8_t pnv_xive_get_block_id(XiveRouter *xrtr)
+{
+    return pnv_xive_block_id(PNV_XIVE(xrtr));
+}
+
 /*
  * The TIMA MMIO space is shared among the chips and to identify the
  * chip from which the access is being done, we extract the chip id
@@ -1926,6 +1931,7 @@ static void pnv_xive_class_init(ObjectClass *klass, void *data)
     xrc->write_end = pnv_xive_write_end;
     xrc->get_nvt = pnv_xive_get_nvt;
     xrc->write_nvt = pnv_xive_write_nvt;
+    xrc->get_block_id = pnv_xive_get_block_id;
 
     xnc->notify = pnv_xive_notify;
     xpc->match_nvt  = pnv_xive_match_nvt;
hw/intc/spapr_xive.c +6 -0
index 1542cef91878..daa0656859a3 100644
@@ -473,6 +473,11 @@ static int spapr_xive_match_nvt(XivePresenter *xptr, uint8_t format,
     return count;
 }
 
+static uint8_t spapr_xive_get_block_id(XiveRouter *xrtr)
+{
+    return SPAPR_XIVE_BLOCK_ID;
+}
+
 static const VMStateDescription vmstate_spapr_xive_end = {
     .name = TYPE_SPAPR_XIVE "/end",
     .version_id = 1,
@@ -764,6 +769,7 @@ static void spapr_xive_class_init(ObjectClass *klass, void *data)
     xrc->write_end = spapr_xive_write_end;
     xrc->get_nvt = spapr_xive_get_nvt;
     xrc->write_nvt = spapr_xive_write_nvt;
+    xrc->get_block_id = spapr_xive_get_block_id;
 
     sicc->activate = spapr_xive_activate;
     sicc->deactivate = spapr_xive_deactivate;
hw/intc/xive.c +16 -5
index ce904b0b5ab4..e5159466f088 100644
@@ -1371,17 +1371,25 @@ int xive_router_write_nvt(XiveRouter *xrtr, uint8_t nvt_blk, uint32_t nvt_idx,
    return xrc->write_nvt(xrtr, nvt_blk, nvt_idx, nvt, word_number);
 }
 
+static int xive_router_get_block_id(XiveRouter *xrtr)
+{
+   XiveRouterClass *xrc = XIVE_ROUTER_GET_CLASS(xrtr);
+
+   return xrc->get_block_id(xrtr);
+}
+
 /*
  * Encode the HW CAM line in the block group mode format :
  *
  *   chip << 19 | 0000000 0 0001 thread (7Bit)
  */
-static uint32_t xive_tctx_hw_cam_line(XiveTCTX *tctx)
+static uint32_t xive_tctx_hw_cam_line(XivePresenter *xptr, XiveTCTX *tctx)
 {
     CPUPPCState *env = &POWERPC_CPU(tctx->cs)->env;
     uint32_t pir = env->spr_cb[SPR_PIR].default_value;
+    uint8_t blk = xive_router_get_block_id(XIVE_ROUTER(xptr));
 
-    return xive_nvt_cam_line((pir >> 8) & 0xf, 1 << 7 | (pir & 0x7f));
+    return xive_nvt_cam_line(blk, 1 << 7 | (pir & 0x7f));
 }
 
 /*
@@ -1418,7 +1426,7 @@ int xive_presenter_tctx_match(XivePresenter *xptr, XiveTCTX *tctx,
 
         /* PHYS ring */
         if ((be32_to_cpu(qw3w2) & TM_QW3W2_VT) &&
-            cam == xive_tctx_hw_cam_line(tctx)) {
+            cam == xive_tctx_hw_cam_line(xptr, tctx)) {
             return TM_QW3_HV_PHYS;
         }
 
@@ -1755,7 +1763,11 @@ static uint64_t xive_end_source_read(void *opaque, hwaddr addr, unsigned size)
     uint8_t pq;
     uint64_t ret = -1;
 
-    end_blk = xsrc->block_id;
+    /*
+     * The block id should be deduced from the load address on the END
+     * ESB MMIO but our model only supports a single block per XIVE chip.
+     */
+    end_blk = xive_router_get_block_id(xsrc->xrtr);
     end_idx = addr >> (xsrc->esb_shift + 1);
 
     if (xive_router_get_end(xsrc->xrtr, end_blk, end_idx, &end)) {
@@ -1855,7 +1867,6 @@ static void xive_end_source_realize(DeviceState *dev, Error **errp)
 }
 
 static Property xive_end_source_properties[] = {
-    DEFINE_PROP_UINT8("block-id", XiveENDSource, block_id, 0),
     DEFINE_PROP_UINT32("nr-ends", XiveENDSource, nr_ends, 0),
     DEFINE_PROP_UINT32("shift", XiveENDSource, esb_shift, XIVE_ESB_64K),
     DEFINE_PROP_LINK("xive", XiveENDSource, xrtr, TYPE_XIVE_ROUTER,
Signature · 1 line
2.21.0
#62 lore raw
In reply to Cédric Le Goater
Quoted 27 lines · Cédric Le Goater

Hello,

The QEMU PowerNV machine emulates a baremetal OpenPOWER system and acts as an hypervisor (L0). Supporting emulation of KVM to run guests (L1) requires a few more extensions, among which guest support for the XIVE interrupt controller on POWER9 processor.

The following changes extend the XIVE models with the new XiveFabric and XivePresenter interfaces to provide support for XIVE escalations and interrupt resend. This mechanism is used by XIVE to notify the hypervisor that a vCPU is not dispatched on a HW thread. Tested on a QEMU PowerNV machine and a simple QEMU pseries guest doing network on a local bridge.

The XIVE interrupt controller offers a way to increase the XIVE resources per chip by configuring multiple XIVE blocks on a chip. This is not currently supported by the model. However, some configurations, such as OPAL/skiboot, use one block-per-chip configuration with some optimizations. One of them is to override the hardwired chip ID by the block id in the PowerBUS operations and for CAM line compares. This patchset improves the support for this setup. Tested with 4 chips.

A series from Suraj adding guest support in the Radix MMU model of the QEMU PowerNV machine is still required and will be send later. The whole patchset can be found under :

https://github.com/legoater/qemu/tree/powernv-4.2

[ ... ]

Quoted 10 lines

Cédric Le Goater (23):
ppc/xive: Record the IPB in the associated NVT
ppc/xive: Introduce helpers for the NVT id
ppc/pnv: Remove pnv_xive_vst_size() routine
ppc/pnv: Dump the XIVE NVT table
ppc/pnv: Quiesce some XIVE errors
ppc/xive: Introduce OS CAM line helpers
ppc/xive: Check V bit in TM_PULL_POOL_CTX
ppc/xive: Introduce a XivePresenter interface
ppc/xive: Implement the XivePresenter interface

David,

I have reworked the following patches to address Greg's comments and your comment on "ppc/pnv: Dump the XIVE NVT table".

Shall I wait for some feedback from you or just resend ?

Thanks,

C.

Quoted 26 lines

ppc/pnv: Loop on the threads of the chip to find a matching NVT ppc/pnv: Introduce a pnv_xive_is_cpu_enabled() helper ppc/xive: Introduce a XiveFabric interface
ppc/pnv: Implement the XiveFabric interface
ppc/spapr: Implement the XiveFabric interface
ppc/xive: Use the XiveFabric and XivePresenter interfaces ppc/xive: Extend the TIMA operation with a XivePresenter parameter ppc/pnv: Clarify how the TIMA is accessed on a multichip system ppc/xive: Move the TIMA operations to the controller model ppc/xive: Remove the get_tctx() XiveRouter handler ppc/xive: Introduce a xive_tctx_ipb_update() helper ppc/xive: Synthesize interrupt from the saved IPB in the NVT ppc/pnv: Introduce a pnv_xive_block_id() helper ppc/pnv: Extend XiveRouter with a get_block_id() handler

 include/hw/ppc/pnv.h       |  15 ++
 include/hw/ppc/pnv_xive.h  |   3 -
 include/hw/ppc/xive.h      |  72 ++++++--
 include/hw/ppc/xive_regs.h |  24 +++
 hw/intc/pnv_xive.c         | 360 ++++++++++++++++++++++++-------------
 hw/intc/spapr_xive.c       |  88 ++++++++-
 hw/intc/xive.c             | 350 ++++++++++++++++++++----------------
 hw/ppc/pnv.c               |  37 +++-
 hw/ppc/spapr.c             |  36 ++++
 9 files changed, 691 insertions(+), 294 deletions(-)
#63 lore raw
Quoted 43 lines · Cédric Le Goater
Quoted 27 lines · Cédric Le Goater

Hello,

The QEMU PowerNV machine emulates a baremetal OpenPOWER system and acts as an hypervisor (L0). Supporting emulation of KVM to run guests (L1) requires a few more extensions, among which guest support for the XIVE interrupt controller on POWER9 processor.

The following changes extend the XIVE models with the new XiveFabric and XivePresenter interfaces to provide support for XIVE escalations and interrupt resend. This mechanism is used by XIVE to notify the hypervisor that a vCPU is not dispatched on a HW thread. Tested on a QEMU PowerNV machine and a simple QEMU pseries guest doing network on a local bridge.

The XIVE interrupt controller offers a way to increase the XIVE resources per chip by configuring multiple XIVE blocks on a chip. This is not currently supported by the model. However, some configurations, such as OPAL/skiboot, use one block-per-chip configuration with some optimizations. One of them is to override the hardwired chip ID by the block id in the PowerBUS operations and for CAM line compares. This patchset improves the support for this setup. Tested with 4 chips.

A series from Suraj adding guest support in the Radix MMU model of the QEMU PowerNV machine is still required and will be send later. The whole patchset can be found under :

https://github.com/legoater/qemu/tree/powernv-4.2

[ ... ]

Quoted 10 lines

Cédric Le Goater (23):
ppc/xive: Record the IPB in the associated NVT
ppc/xive: Introduce helpers for the NVT id
ppc/pnv: Remove pnv_xive_vst_size() routine
ppc/pnv: Dump the XIVE NVT table
ppc/pnv: Quiesce some XIVE errors
ppc/xive: Introduce OS CAM line helpers
ppc/xive: Check V bit in TM_PULL_POOL_CTX
ppc/xive: Introduce a XivePresenter interface
ppc/xive: Implement the XivePresenter interface

Quoted 6 lines

David,

I have reworked the following patches to address Greg's comments and your comment on "ppc/pnv: Dump the XIVE NVT table".

Shall I wait for some feedback from you or just resend ?

Just resend, thanks.

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson