Remove PCIE root bridge LSI on powernv
cover Remove PCIE root bridge LSI on powernv
The powernv8/powernv9/powernv10 machines allocate a LSI for their root port bridge, which is not the case on real hardware. The default root port implementation in qemu requests a LSI. Since the powernv implementation derives from it, that's where the LSI is coming from. This series fixes it, so that the model matches the hardware.
However, the code in hw/pci to handle AER and hotplug events assume a LSI is defined. It tends to assert/deassert a LSI if MSI or MSIX is not enabled. Since we have hardware where that is not true, this patch also fixes a few code paths to check if a LSI is configured before trying to trigger it.
Frederic Barrat (2):
pcie: Don't try triggering a LSI when not defined ppc/pnv: Remove LSI on the PCIE host bridge
hw/pci-host/pnv_phb3.c | 1 + hw/pci-host/pnv_phb4.c | 1 + hw/pci/pcie.c | 8 ++++++-- hw/pci/pcie_aer.c | 4 +++- 4 files changed, 11 insertions(+), 3 deletions(-)
Signature · 1 line
2.35.1
PATCH 1/2 pcie: Don't try triggering a LSI when not defined
This patch skips [de]asserting a LSI interrupt if the device doesn't have any LSI defined. Doing so would trigger an assert in pci_irq_handler().
The PCIE root port implementation in qemu requests a LSI (INTA), but a subclass may want to change that behavior since it's a valid configuration. For example on the POWER8/POWER9/POWER10 systems, the root bridge doesn't request any LSI.
--- hw/pci/pcie.c | 8 ++++++-- hw/pci/pcie_aer.c | 4 +++- 2 files changed, 9 insertions(+), 3 deletions(-)
hw/pci/pcie.c +6 -2
@@ -354,7 +354,9 @@ static void hotplug_event_notify(PCIDevice *dev) }else if (msi_enabled (dev)) {msi_notify (dev,pcie_cap_flags_get_vector (dev)); }else { -pci_set_irq (dev, dev->exp.hpev_notified); +if (pci_intx (dev) !=-1 ) { +pci_set_irq (dev, dev->exp.hpev_notified); + } } }
@@ -362,7 +364,9 @@ static void hotplug_event_clear(PCIDevice *dev) {hotplug_event_update_event_status (dev);if (!msix_enabled (dev) && !msi_enabled (dev) && !dev->exp.hpev_notified) { -pci_irq_deassert (dev); +if (pci_intx (dev) !=-1 ) { +pci_irq_deassert (dev); + } } }
hw/pci/pcie_aer.c +3 -1
@@ -291,7 +291,9 @@ static void pcie_aer_root_notify(PCIDevice *dev) }else if (msi_enabled (dev)) {msi_notify (dev,pcie_aer_root_get_vector (dev)); }else { -pci_irq_assert (dev); +if (pci_intx (dev) !=-1 ) { +pci_irq_assert (dev); + } } }
Signature · 1 line
2.35.1
Re: [PATCH 1/2] pcie: Don't try triggering a LSI when not defined
Quoted 11 lines · Frederic Barrat
This patch skips [de]asserting a LSI interrupt if the device doesn't have any LSI defined. Doing so would trigger an assert in
pci_irq_handler().The PCIE root port implementation in qemu requests a LSI (INTA), but a subclass may want to change that behavior since it's a valid configuration. For example on the POWER8/POWER9/POWER10 systems, the root bridge doesn't request any LSI.
Signed-off-by Frederic Barrat <[email protected]>---
I assume that it's easier to handle just the codepaths that powernv PHBs uses rather than handling all instances where pci_irq_handler() would be asserting without LSIs.
Patch LGTM. Small nits below:
Quoted 9 lines
hw/pci/pcie.c | 8 ++++++-- hw/pci/pcie_aer.c | 4 +++- 2 files changed, 9 insertions(+), 3 deletions(-)hw/pci/pcie.c +0
-0
hw/pci/pcie.c static void hotplug_event_notify(PCIDevice *dev) · line 359
Quoted 7 lines
hw/pci/pcie.c +3
-1@@ -354,7 +354,9 @@ static void hotplug_event_notify(PCIDevice *dev) }else if (msi_enabled (dev)) {msi_notify (dev,pcie_cap_flags_get_vector (dev)); }else { -pci_set_irq (dev, dev->exp.hpev_notified); +if (pci_intx (dev) !=-1 ) { +pci_set_irq (dev, dev->exp.hpev_notified); + }
Since you're not doing anything unless the condition is met, you can use 'else if' like it's done in the other conditionals:
if (msix_enabled(dev)) {
msix_notify(dev, pcie_cap_flags_get_vector(dev));
} else if (msi_enabled(dev)) {
msi_notify(dev, pcie_cap_flags_get_vector(dev));
} else if (pci_intx(dev) != -1) {
pci_set_irq(dev, dev->exp.hpev_notified);
}hw/pci/pcie.c static void hotplug_event_clear(PCIDevice *dev) · line 370
Quoted 12 lines
hw/pci/pcie.c +3
-1} }@@ -362,7 +364,9 @@ static void hotplug_event_clear(PCIDevice *dev) {hotplug_event_update_event_status (dev);if (!msix_enabled (dev) && !msi_enabled (dev) && !dev->exp.hpev_notified) { -pci_irq_deassert (dev); +if (pci_intx (dev) !=-1 ) { +pci_irq_deassert (dev); + } }
Similar comment here:
if (!msix_enabled(dev) && !msi_enabled(dev) && !dev->exp.hpev_notified &&
pci_intx(dev) != -1) {
pci_irq_deassert(dev);
}Quoted 7 lines
hw/pci/pcie.c +0
-0}hw/pci/pcie_aer.c +0
-0
hw/pci/pcie_aer.c static void pcie_aer_root_notify(PCIDevice *dev) · line 296
Quoted 7 lines
hw/pci/pcie_aer.c +3
-1@@ -291,7 +291,9 @@ static void pcie_aer_root_notify(PCIDevice *dev) }else if (msi_enabled (dev)) {msi_notify (dev,pcie_aer_root_get_vector (dev)); }else { -pci_irq_assert (dev); +if (pci_intx (dev) !=-1 ) { +pci_irq_assert (dev); + }
And here:
if (msix_enabled(dev)) {
msix_notify(dev, pcie_aer_root_get_vector(dev));
} else if (msi_enabled(dev)) {
msi_notify(dev, pcie_aer_root_get_vector(dev));
} else if (pci_intx(dev) != -1) {
pci_irq_assert(dev);
}Thanks,
Daniel
Quoted 3 lines
hw/pci/pcie_aer.c +0
-0} }
Re: [PATCH 1/2] pcie: Don't try triggering a LSI when not defined
Quoted 12 lines · Daniel Henrique Barboza
Quoted 9 lines · Frederic Barrat
This patch skips [de]asserting a LSI interrupt if the device doesn't have any LSI defined. Doing so would trigger an assert in
pci_irq_handler().The PCIE root port implementation in qemu requests a LSI (INTA), but a subclass may want to change that behavior since it's a valid configuration. For example on the POWER8/POWER9/POWER10 systems, the root bridge doesn't request any LSI.
Quoted 6 lines
Quoted 2 lines
Signed-off-by Frederic Barrat <[email protected]>---
I assume that it's easier to handle just the codepaths that powernv PHBs uses rather than handling all instances where
pci_irq_handler()would be asserting without LSIs.
The real reason is that the LSI is added when we realize the TYPE_PCIE_ROOT_PORT object. See rp_realize(). So I'm only trying to fix the code paths that can be called from a subclass of TYPE_PCIE_ROOT_PORT which would choose to override the "interrupt pin" setting in the config space. I believe they are all covered by this patch. The assert() in pci_irq_handler() is there for a reason and I don't want to mess with that.
Fred
Quoted 92 lines
Patch LGTM. Small nits below:
Quoted 9 lines
hw/pci/pcie.c | 8 ++++++--
hw/pci/pcie_aer.c| 4 +++-2 files changed, 9 insertions(+), 3 deletions(-)hw/pci/pcie.c +0
-0hw/pci/pcie.c static void hotplug_event_notify(PCIDevice *dev) · line 359
Quoted 7 lines
hw/pci/pcie.c +3
-1@@ -354,7 +354,9 @@ static void hotplug_event_notify(PCIDevice *dev) } else if (msi_enabled(dev)) { msi_notify(dev, pcie_cap_flags_get_vector(dev)); } else { - pci_set_irq(dev, dev->exp.hpev_notified); + if (pci_intx(dev) != -1) { + pci_set_irq(dev, dev->exp.hpev_notified); + }Since you're not doing anything unless the condition is met, you can use 'else if' like it's done in the other conditionals:
if (
msix_enabled(dev)) {
msix_notify(dev, pcie_cap_flags_get_vector(dev)); } else if (msi_enabled(dev)) {msi_notify(dev, pcie_cap_flags_get_vector(dev)); } else if (pci_intx(dev)!= -1) {pci_set_irq(dev, dev->exp.hpev_notified); }hw/pci/pcie.c static void hotplug_event_clear(PCIDevice *dev) · line 370
Quoted 11 lines
hw/pci/pcie.c +3
-1} }@@ -362,7 +364,9 @@ static void hotplug_event_clear(PCIDevice *dev) { hotplug_event_update_event_status(dev); if (!msix_enabled(dev) && !msi_enabled(dev) && !dev->exp.hpev_notified) { - pci_irq_deassert(dev); + if (pci_intx(dev) != -1) { + pci_irq_deassert(dev); + } }Similar comment here:
if (!msix_enabled(dev) && !msi_enabled(dev) &&
!dev->exp.hpev_notified &&
pci_intx(dev)!= -1) {pci_irq_deassert(dev);
}hw/pci/pcie_aer.c static void pcie_aer_root_notify(PCIDevice *dev) · line 296
Quoted 13 lines
hw/pci/pcie.c +0
-0}hw/pci/pcie_aer.c +3
-1@@ -291,7 +291,9 @@ static void pcie_aer_root_notify(PCIDevice *dev) } else if (msi_enabled(dev)) { msi_notify(dev, pcie_aer_root_get_vector(dev)); } else { - pci_irq_assert(dev); + if (pci_intx(dev) != -1) { + pci_irq_assert(dev); + }And here:
if (
msix_enabled(dev)) {
msix_notify(dev, pcie_aer_root_get_vector(dev)); } else if (msi_enabled(dev)) {msi_notify(dev, pcie_aer_root_get_vector(dev)); } else if (pci_intx(dev)!= -1) {pci_irq_assert(dev);
}Thanks,
Daniel
Quoted 2 lines
hw/pci/pcie_aer.c +0
-0} }
Re: [PATCH 1/2] pcie: Don't try triggering a LSI when not defined
Quoted 18 lines · Frederic Barrat
Quoted 15 lines · Daniel Henrique Barboza
Quoted 11 lines · Frederic Barrat
This patch skips [de]asserting a LSI interrupt if the device doesn't have any LSI defined. Doing so would trigger an assert in
pci_irq_handler().The PCIE root port implementation in qemu requests a LSI (INTA), but a subclass may want to change that behavior since it's a valid configuration. For example on the POWER8/POWER9/POWER10 systems, the root bridge doesn't request any LSI.
Signed-off-by Frederic Barrat <[email protected]>---
Quoted 7 lines
Quoted 3 lines
I assume that it's easier to handle just the codepaths that powernv PHBs uses rather than handling all instances where
pci_irq_handler()would be asserting without LSIs.The real reason is that the LSI is added when we realize the
TYPE_PCIE_ROOT_PORTobject. Seerp_realize(). So I'm only trying to fix the code paths that can be called from a subclass ofTYPE_PCIE_ROOT_PORTwhich would choose to override the "interrupt pin" setting in the config space. I believe they are all covered by this patch. Theassert()inpci_irq_handler()is there for a reason and I don't want to mess with that.
Yes, handling this situation inside pci_irq_handler() would require changing the "assert(0 <= irq_num && irq_num < PCI_NUM_PINS)" assert or doing some sanity before it to avoid the trigger.
Since this is a common code used everywhere we're better of doing minimalist changes as you're doing. We can reevaluate this design if more machines/devices start to get in the same situation we have now with powernv PHBs.
Daniel
Quoted 95 lines
Fred
Quoted 91 lines
Patch LGTM. Small nits below:
Quoted 9 lines
hw/pci/pcie.c | 8 ++++++--
hw/pci/pcie_aer.c| 4 +++-2 files changed, 9 insertions(+), 3 deletions(-)hw/pci/pcie.c +0
-0hw/pci/pcie.c static void hotplug_event_notify(PCIDevice *dev) · line 359
Quoted 7 lines
hw/pci/pcie.c +3
-1@@ -354,7 +354,9 @@ static void hotplug_event_notify(PCIDevice *dev) } else if (msi_enabled(dev)) { msi_notify(dev, pcie_cap_flags_get_vector(dev)); } else { - pci_set_irq(dev, dev->exp.hpev_notified); + if (pci_intx(dev) != -1) { + pci_set_irq(dev, dev->exp.hpev_notified); + }Since you're not doing anything unless the condition is met, you can use 'else if' like it's done in the other conditionals:
if (
msix_enabled(dev)) {
msix_notify(dev, pcie_cap_flags_get_vector(dev)); } else if (msi_enabled(dev)) {msi_notify(dev, pcie_cap_flags_get_vector(dev)); } else if (pci_intx(dev)!= -1) {pci_set_irq(dev, dev->exp.hpev_notified); }hw/pci/pcie.c static void hotplug_event_clear(PCIDevice *dev) · line 370
Quoted 11 lines
hw/pci/pcie.c +3
-1} }@@ -362,7 +364,9 @@ static void hotplug_event_clear(PCIDevice *dev) { hotplug_event_update_event_status(dev); if (!msix_enabled(dev) && !msi_enabled(dev) && !dev->exp.hpev_notified) { - pci_irq_deassert(dev); + if (pci_intx(dev) != -1) { + pci_irq_deassert(dev); + } }Similar comment here:
if (!msix_enabled(dev) && !msi_enabled(dev) && !dev->exp.hpev_notified &&
pci_intx(dev)!= -1) {
pci_irq_deassert(dev);
}hw/pci/pcie_aer.c static void pcie_aer_root_notify(PCIDevice *dev) · line 296
Quoted 13 lines
hw/pci/pcie.c +0
-0}hw/pci/pcie_aer.c +3
-1@@ -291,7 +291,9 @@ static void pcie_aer_root_notify(PCIDevice *dev) } else if (msi_enabled(dev)) { msi_notify(dev, pcie_aer_root_get_vector(dev)); } else { - pci_irq_assert(dev); + if (pci_intx(dev) != -1) { + pci_irq_assert(dev); + }And here:
if (
msix_enabled(dev)) {
msix_notify(dev, pcie_aer_root_get_vector(dev)); } else if (msi_enabled(dev)) {msi_notify(dev, pcie_aer_root_get_vector(dev)); } else if (pci_intx(dev)!= -1) {pci_irq_assert(dev);
}Thanks,
Daniel
Quoted 2 lines
hw/pci/pcie_aer.c +0
-0} }
PATCH 2/2 ppc/pnv: Remove LSI on the PCIE host bridge
In reply to Frederic BarratThe phb3/phb4/phb5 root ports inherit from the default PCIE root port implementation, which requests a LSI interrupt (#INTA). On real hardware (POWER8/POWER9/POWER10), there is no such LSI. This patch corrects it so that it matches the hardware.
As a consequence, the device tree previously generated was bogus, as the root bridge LSI was not properly mapped. On some implementation (powernv9), it was leading to inconsistent interrupt controller (xive) data. With this patch, it is now clean.
--- hw/pci-host/pnv_phb3.c | 1 + hw/pci-host/pnv_phb4.c | 1 + 2 files changed, 2 insertions(+)
hw/pci-host/pnv_phb3.c +1 -0
@@ -1162,6 +1162,7 @@ static void pnv_phb3_root_port_realize(DeviceState *dev, Error **errp)error_propagate (errp, local_err);return ; } +pci_config_set_interrupt_pin (pci->config,0 ); }static void pnv_phb3_root_port_class_init (ObjectClass *klass,void *data)
hw/pci-host/pnv_phb4.c +1 -0
@@ -1772,6 +1772,7 @@ static void pnv_phb4_root_port_reset(DeviceState *dev)pci_set_word (conf +PCI_PREF_MEMORY_LIMIT ,0xfff1 );pci_set_long (conf +PCI_PREF_BASE_UPPER32 ,0x1 );/* Hack */ pci_set_long (conf +PCI_PREF_LIMIT_UPPER32 ,0xffffffff ); +pci_config_set_interrupt_pin (conf,0 ); }static void pnv_phb4_root_port_realize (DeviceState *dev,Error **errp)
Signature · 1 line
2.35.1
Re: [PATCH 2/2] ppc/pnv: Remove LSI on the PCIE host bridge
Quoted 5 lines · Frederic Barrat
The phb3/phb4/phb5 root ports inherit from the default PCIE root port implementation, which requests a LSI interrupt (#INTA). On real hardware (POWER8/POWER9/POWER10), there is no such LSI. This patch corrects it so that it matches the hardware.
Quoted 7 lines
As a consequence, the device tree previously generated was bogus, as the root bridge LSI was not properly mapped. On some implementation (powernv9), it was leading to inconsistent interrupt controller (xive) data. With this patch, it is now clean.
Signed-off-by Frederic Barrat <[email protected]>---
Quoted 28 lines
hw/pci-host/pnv_phb3.c | 1 + hw/pci-host/pnv_phb4.c | 1 + 2 files changed, 2 insertions(+)hw/pci-host/pnv_phb3.c +1
-0@@ -1162,6 +1162,7 @@ static void pnv_phb3_root_port_realize(DeviceState *dev, Error **errp)error_propagate (errp, local_err);return ; } +pci_config_set_interrupt_pin (pci->config,0 ); }static void pnv_phb3_root_port_class_init (ObjectClass *klass,void *data)hw/pci-host/pnv_phb4.c +1
-0@@ -1772,6 +1772,7 @@ static void pnv_phb4_root_port_reset(DeviceState *dev)pci_set_word (conf +PCI_PREF_MEMORY_LIMIT ,0xfff1 );pci_set_long (conf +PCI_PREF_BASE_UPPER32 ,0x1 );/* Hack */ pci_set_long (conf +PCI_PREF_LIMIT_UPPER32 ,0xffffffff ); +pci_config_set_interrupt_pin (conf,0 ); }static void pnv_phb4_root_port_realize (DeviceState *dev,Error **errp)
Quoted 11 lines · Frederic Barrat
The powernv8/powernv9/powernv10 machines allocate a LSI for their root port bridge, which is not the case on real hardware. The default root port implementation in qemu requests a LSI. Since the powernv implementation derives from it, that's where the LSI is coming from. This series fixes it, so that the model matches the hardware.
However, the code in hw/pci to handle AER and hotplug events assume a LSI is defined. It tends to assert/deassert a LSI if MSI or MSIX is not enabled. Since we have hardware where that is not true, this patch also fixes a few code paths to check if a LSI is configured before trying to trigger it.
Hi Frederic, thanks for the patch!
I assume you will address Daniel's comments and post a new version, right?
Quoted 13 lines
Frederic Barrat (2):
pcie: Don't try triggering a LSI when not defined ppc/pnv: Remove LSI on the PCIE host bridgehw/pci-host/pnv_phb3.c | 1 + hw/pci-host/pnv_phb4.c | 1 + hw/pci/pcie.c | 8 ++++++-- hw/pci/pcie_aer.c | 4 +++- 4 files changed, 11 insertions(+), 3 deletions(-)Signature · 1 line
2.35.1
Quoted 14 lines · Michael S. Tsirkin
Quoted 11 lines · Frederic Barrat
The powernv8/powernv9/powernv10 machines allocate a LSI for their root port bridge, which is not the case on real hardware. The default root port implementation in qemu requests a LSI. Since the powernv implementation derives from it, that's where the LSI is coming from. This series fixes it, so that the model matches the hardware.
However, the code in hw/pci to handle AER and hotplug events assume a LSI is defined. It tends to assert/deassert a LSI if MSI or MSIX is not enabled. Since we have hardware where that is not true, this patch also fixes a few code paths to check if a LSI is configured before trying to trigger it.
Quoted 3 lines
Hi Frederic, thanks for the patch!
I assume you will address Daniel's comments and post a new version, right?
Yes, I will. I'm obviously only targeting 7.1
Fred
Quoted 15 lines
Quoted 14 lines
Frederic Barrat (2):
pcie: Don't try triggering a LSI when not defined ppc/pnv: Remove LSI on the PCIE host bridgehw/pci-host/pnv_phb3.c | 1 + hw/pci-host/pnv_phb4.c | 1 + hw/pci/pcie.c | 8 ++++++-- hw/pci/pcie_aer.c | 4 +++- 4 files changed, 11 insertions(+), 3 deletions(-)Signature · 1 line
2.35.1