Skip to contentSkip to conversation

irqchip/gic-v3-its: Correctly honor the RID remapping

PATCH Applied Marc Zyngier 3 participants 3 messages Jul 17, 2024 – Jul 18, 2024

Subsystems (2)
lore
Share & tools
lore

Marc Zyngier

author maintainer
#0 lore raw

PATCH 1/1 irqchip/gic-v3-its: Correctly honor the RID remapping

Since 6adb35ff43a16 ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]"), the primary domain a PCI device allocates its interrupts from is the one that is directly attached to the device itself.

By virtue of being a PCI device, it has no OF node.

This domain is (through more layer than it is worth describing) passed to its_pci_msi_prepare(), which tries to compute the full RID that is presented to the ITS by the device. This is ultimately done by calling pci_msi_domain_get_msi_rid(), passing both the domain and the PCI device as arguments.

The baked-in assumption is that either the domain that is passed to pci_msi_domain_get_msi_rid() describes an interrupt controller with either an OF node or an entry in an ACPI IORT table. In our case, it is *neither*. This domain is does not represent anything firmware-based, but just an allocation unit for the device.

As a result, we fail to provide the full RID (which requires inspecting the msi-map/msi-mask properties in the DT), and stick to the BDF, which isn't very useful.

Tragedy follows with a litany of devices that randomly die as they fail to see any MSI (because the RID is wrong) or fail to get an allocation (because they try to steal LPIs from their neighbour's pool).

This will happen on any system where a single ITS is shared by multiple root ports and end-points with overlapping BDF numbers, and has the topology described in the device-tree. Simpler DT topologies will luckily work, and so will ACPI-based systems.

Solve it by pointing pci_msi_domain_get_msi_rid() at the *parent* domain, which is the ITS, resulting in a correct mapping and a restored happiness in my personal zoo.

Fixes 6adb35ff43a16 ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")
Cc Thomas Gleixner <[email protected]>
---
 drivers/irqchip/irq-gic-v3-its-msi-parent.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its-msi-parent.c +1 -1
index 780e1bc9df549..2f3fc597331ba 100644
@@ -65,7 +65,7 @@ static int its_pci_msi_prepare(struct irq_domain *domain, struct device *dev,
 	}
 
 	/* ITS specific DeviceID, as the core ITS ignores dev. */
-	info->scratchpad[0].ul = pci_msi_domain_get_msi_rid(domain, pdev);
+	info->scratchpad[0].ul = pci_msi_domain_get_msi_rid(domain->parent, pdev);
 
 	/*
 	 * @domain->msi_domain_info->hwsize contains the size of the
Signature · 1 line
2.39.2
Tested-by on v1 1/1
#1 lore raw
Quoted 12 lines · Marc Zyngier

Since 6adb35ff43a16 ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]"), the primary domain a PCI device allocates its interrupts from is the one that is directly attached to the device itself.

By virtue of being a PCI device, it has no OF node.

This domain is (through more layer than it is worth describing) passed to its_pci_msi_prepare(), which tries to compute the full RID that is presented to the ITS by the device. This is ultimately done by calling pci_msi_domain_get_msi_rid(), passing both the domain and the PCI device as arguments.

Quoted 4 lines

The baked-in assumption is that either the domain that is passed to pci_msi_domain_get_msi_rid() describes an interrupt controller with either an OF node or an entry in an ACPI IORT table. In our case, it is *neither*. This domain is does not represent

nit: stray "is"

Quoted 16 lines

anything firmware-based, but just an allocation unit for the device.

As a result, we fail to provide the full RID (which requires inspecting the msi-map/msi-mask properties in the DT), and stick to the BDF, which isn't very useful.

Tragedy follows with a litany of devices that randomly die as they fail to see any MSI (because the RID is wrong) or fail to get an allocation (because they try to steal LPIs from their neighbour's pool).

This will happen on any system where a single ITS is shared by multiple root ports and end-points with overlapping BDF numbers, and has the topology described in the device-tree. Simpler DT topologies will luckily work, and so will ACPI-based systems.

Quoted 8 lines

Solve it by pointing pci_msi_domain_get_msi_rid() at the *parent* domain, which is the ITS, resulting in a correct mapping and a restored happiness in my personal zoo.

Fixes 6adb35ff43a16 ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")
Cc Thomas Gleixner <[email protected]>

This fixes the regression for the Qualcomm machines I ran into this with:

Thanks!

Johan

merge bot Applied
#2 lore raw
In reply to Marc Zyngier
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     99d7fbf8f813eb77dd4ec148c4596455910b2fa1
Gitweb:        https://git.kernel.org/tip/99d7fbf8f813eb77dd4ec148c4596455910b2fa1
Author:        Marc Zyngier <[email protected]>
AuthorDate:    Wed, 17 Jul 2024 20:59:37 +01:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:34:46 +02:00

irqchip/gic-v3-its: Correctly honor the RID remapping

Since 6adb35ff43a16 ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]"), the primary domain a PCI device allocates its interrupts from is the one that is directly attached to the device itself.

By virtue of being a PCI device, it has no OF node.

This domain is (through more layer than it is worth describing) passed to its_pci_msi_prepare(), which tries to compute the full RID that is presented to the ITS by the device. This is ultimately done by calling pci_msi_domain_get_msi_rid(), passing both the domain and the PCI device as arguments.

The baked-in assumption is that either the domain that is passed to pci_msi_domain_get_msi_rid() describes an interrupt controller with either an OF node or an entry in an ACPI IORT table. In this case, it is *neither*. This domain is does not represent anything firmware-based, but just an allocation unit for the device.

As a result, it fails to provide the full RID (which requires inspecting the msi-map/msi-mask properties in the DT), and stick to the BDF, which isn't very useful.

Tragedy follows with a litany of devices that randomly die as they fail to see any MSI (because the RID is wrong) or fail to get an allocation (because they try to steal LPIs from their neighbour's pool).

This will happen on any system where a single ITS is shared by multiple root ports and end-points with overlapping BDF numbers, and has the topology described in the device-tree. Simpler DT topologies will luckily work, and so will ACPI-based systems.

Solve it by pointing pci_msi_domain_get_msi_rid() at the *parent* domain, which is the ITS, resulting in a correct mapping and a restored happiness in my personal zoo.

Fixes 6adb35ff43a16 ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")
---
 drivers/irqchip/irq-gic-v3-its-msi-parent.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its-msi-parent.c +1 -1
index 780e1bc..2f3fc59 100644
@@ -65,7 +65,7 @@ static int its_pci_msi_prepare(struct irq_domain *domain, struct device *dev,
 	}
 
 	/* ITS specific DeviceID, as the core ITS ignores dev. */
-	info->scratchpad[0].ul = pci_msi_domain_get_msi_rid(domain, pdev);
+	info->scratchpad[0].ul = pci_msi_domain_get_msi_rid(domain->parent, pdev);
 
 	/*
 	 * @domain->msi_domain_info->hwsize contains the size of the