Skip to contentSkip to conversation

Generic PCIe host bridge INTx determination for INTx routing

PATCH Superseded Eric Auger 1 participant 5 messages Jun 13, 2017 – Jul 3, 2017

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Eric Auger

author
#0 lore raw

v3 cover Generic PCIe host bridge INTx determination for INTx routing

This series implements INTx to gsi routing for ARM VIRT/GPEX. This is a respin of [1] which was lost in limbo.

ARM virt uses GPEX PCIe bridge. This latter does not implement INTx to GSI routing. PCIe/INTx assignment works but the consequence is irqfd is not used along with INTx interrupts and VFIO INTx handlers are executed on userspace leading to an important performance degradation.

This issue is witnessed by the following messages;

qemu-system-aarch64: -device vfio-pci,host=0006:90:00.0: PCI: Bug - unimplemented PCI INTx routing (gpex-pcihost)
qemu-system-aarch64: PCI: Bug - unimplemented PCI INTx routing (gpex-pcihost)
qemu-system-aarch64: PCI: Bug - unimplemented PCI INTx routing (gpex-pcihost)

So with this series, irqfd is set up for PCIe/INTx passthrough and we get the optimal performance. Also we get rid of the above messages.

This series can be found at:
https://github.com/eauger/qemu/tree/v2.9-gpex-intx-v3

References:
[1] Generic PCIe host bridge INTx determination for INTx routing
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg04975.html

Pranavkumar Sawargaonkar (3):
hw/pci-host/gpex: Set INTx index/gsi mapping
hw/arm/virt: Set INTx/gsi mapping
hw/pci-host/gpex: Implement PCI INTx routing

 hw/arm/virt.c              |  4 ++++
 hw/pci-host/gpex.c         | 22 ++++++++++++++++++++++
 include/hw/pci-host/gpex.h |  3 +++
 3 files changed, 29 insertions(+)
Signature · 1 line
2.5.5
author
#1 lore raw

PATCH v3 1/3 hw/pci-host/gpex: Set INTx index/gsi mapping

From: Pranavkumar Sawargaonkar <[email protected]>

To implement INTx to gsi routing we need to pass the gpex host bridge the gsi associated to each INTx index. Let's introduce irq_num array and gpex_set_irq_num setter function.

---
 hw/pci-host/gpex.c         | 10 ++++++++++
 include/hw/pci-host/gpex.h |  3 +++
 2 files changed, 13 insertions(+)
hw/pci-host/gpex.c +10 -0
index e2629ce..22367aa4 100644
@@ -43,6 +43,16 @@ static void gpex_set_irq(void *opaque, int irq_num, int level)
     qemu_set_irq(s->irq[irq_num], level);
 }
 
+int gpex_set_irq_num(GPEXHost *s, int index, uint32_t gsi)
+{
+    if (index >=  GPEX_NUM_IRQS) {
+        return -EINVAL;
+    }
+
+    s->irq_num[index] = gsi;
+    return 0;
+}
+
 static void gpex_host_realize(DeviceState *dev, Error **errp)
 {
     PCIHostState *pci = PCI_HOST_BRIDGE(dev);
include/hw/pci-host/gpex.h +3 -0
index 68c9348..db26b3e 100644
@@ -51,6 +51,9 @@ typedef struct GPEXHost {
     MemoryRegion io_ioport;
     MemoryRegion io_mmio;
     qemu_irq irq[GPEX_NUM_IRQS];
+    uint32_t irq_num[GPEX_NUM_IRQS];
 } GPEXHost;
 
+int gpex_set_irq_num(GPEXHost *s, int index, uint32_t gsi);
+
 #endif /* HW_GPEX_H */
Signature · 1 line
2.5.5
author
#2 lore raw

PATCH v3 2/3 hw/arm/virt: Set INTx/gsi mapping

In reply to Eric Auger

From: Pranavkumar Sawargaonkar <[email protected]>

Let's provide the GPEX host bridge with the INTx/gsi mapping. This is needed for INTx/gsi routing.

---
 hw/arm/virt.c | 4 ++++
 1 file changed, 4 insertions(+)
hw/arm/virt.c +4 -0
index 010f724..f86c229 100644
@@ -1012,6 +1012,7 @@ static void create_pcie(const VirtMachineState *vms, qemu_irq *pic)
     char *nodename;
     int i;
     PCIHostState *pci;
+    GPEXHost *s;
 
     dev = qdev_create(NULL, TYPE_GPEX_HOST);
     qdev_init_nofail(dev);
@@ -1047,8 +1048,11 @@ static void create_pcie(const VirtMachineState *vms, qemu_irq *pic)
     /* Map IO port space */
     sysbus_mmio_map(SYS_BUS_DEVICE(dev), 2, base_pio);
 
+    s = GPEX_HOST(dev);
+
     for (i = 0; i < GPEX_NUM_IRQS; i++) {
         sysbus_connect_irq(SYS_BUS_DEVICE(dev), i, pic[irq + i]);
+        gpex_set_irq_num(s, i, irq + i);
     }
 
     pci = PCI_HOST_BRIDGE(dev);
Signature · 1 line
2.5.5
author
#3 lore raw

PATCH v3 3/3 hw/pci-host/gpex: Implement PCI INTx routing

In reply to Eric Auger

From: Pranavkumar Sawargaonkar <[email protected]>

Now we are able to retrieve the gsi from the INTx pin, let's enable intx_to_irq routing. From that point on, irqfd becomes usable along with INTx when assigning a PCIe device.

---
 hw/pci-host/gpex.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
hw/pci-host/gpex.c +12 -0
index 22367aa4..2ec918b 100644
@@ -53,6 +53,17 @@ int gpex_set_irq_num(GPEXHost *s, int index, uint32_t gsi)
     return 0;
 }
 
+static PCIINTxRoute gpex_route_intx_pin_to_irq(void *opaque, int pin)
+{
+    PCIINTxRoute route;
+    GPEXHost *s = opaque;
+
+    route.mode = PCI_INTX_ENABLED;
+    route.irq = (int)s->irq_num[pin];
+
+    return route;
+}
+
 static void gpex_host_realize(DeviceState *dev, Error **errp)
 {
     PCIHostState *pci = PCI_HOST_BRIDGE(dev);
@@ -77,6 +88,7 @@ static void gpex_host_realize(DeviceState *dev, Error **errp)
                                 &s->io_ioport, 0, 4, TYPE_PCIE_BUS);
 
     qdev_set_parent_bus(DEVICE(&s->gpex_root), BUS(pci->bus));
+    pci_bus_set_route_irq_fn(pci->bus, gpex_route_intx_pin_to_irq);
     qdev_init_nofail(DEVICE(&s->gpex_root));
 }
 
Signature · 1 line
2.5.5
author
#4 lore raw
In reply to Eric Auger

Hi,

Quoted 10 lines · Eric Auger

This series implements INTx to gsi routing for ARM VIRT/GPEX. This is a respin of [1] which was lost in limbo.

ARM virt uses GPEX PCIe bridge. This latter does not implement INTx to GSI routing. PCIe/INTx assignment works but the consequence is irqfd is not used along with INTx interrupts and VFIO INTx handlers are executed on userspace leading to an important performance degradation.

This issue is witnessed by the following messages;

Quoted 7 lines

qemu-system-aarch64: -device vfio-pci,host=0006:90:00.0: PCI: Bug - unimplemented PCI INTx routing (gpex-pcihost)
qemu-system-aarch64: PCI: Bug - unimplemented PCI INTx routing (gpex-pcihost)
qemu-system-aarch64: PCI: Bug - unimplemented PCI INTx routing (gpex-pcihost)

So with this series, irqfd is set up for PCIe/INTx passthrough and we get the optimal performance. Also we get rid of the above messages.

Ping? any feedback on this series that fixes irqfd setup for PCIe passthrough/ARM with legacy interrupts?

Thanks

Eric

Quoted 18 lines

This series can be found at:
https://github.com/eauger/qemu/tree/v2.9-gpex-intx-v3

References:
[1] Generic PCIe host bridge INTx determination for INTx routing
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg04975.html

Pranavkumar Sawargaonkar (3):
hw/pci-host/gpex: Set INTx index/gsi mapping
hw/arm/virt: Set INTx/gsi mapping
hw/pci-host/gpex: Implement PCI INTx routing

 hw/arm/virt.c              |  4 ++++
 hw/pci-host/gpex.c         | 22 ++++++++++++++++++++++
 include/hw/pci-host/gpex.h |  3 +++
 3 files changed, 29 insertions(+)