PCI: vmd: IRQ domain assignment to sub devices
Subsystems (3)
Nirmal Patel
maintainerv2 cover PCI: vmd: IRQ domain assignment to sub devices
Host OS fails to boot and DMAR errors were observed when interrupt remapping is enabled by intel_iommu because of the fact that VMD child devices are on different IRQ domain than all other PCI devices. Make sure VMD assigns proper IRQ domain to the child devices during device enumeration.
Nirmal Patel (2):
PCI: vmd: Assign VMD IRQ domain before enumeration
PCI: vmd: Revert 2565e5b69c44 ("PCI: vmd: Do not disable MSI-X
remapping if interrupt remapping is enabled by IOMMU.")
drivers/pci/controller/vmd.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
Signature · 1 line
2.26.2
Nirmal Patel
maintainerPATCH v2 1/2 PCI: vmd: Assign VMD IRQ domain before enumeration
During the boot process all the PCI devices are assigned default PCI-MSI IRQ domain including VMD endpoint devices. If interrupt-remapping is enabled by IOMMU, the PCI devices except VMD get new INTEL-IR-MSI IRQ domain. And VMD is supposed to create and assign a separate VMD-MSI IRQ domain for its child devices in order to support MSI-X remapping capabilities.
Now when MSI-X remapping in VMD is disabled in order to improve performance, VMD skips VMD-MSI IRQ domain assignment process to its child devices. Thus the devices behind VMD get default PCI-MSI IRQ domain instead of INTEL-IR-MSI IRQ domain when VMD creates root bus and configures child devices.
As a result host OS fails to boot and DMAR errors were observed when interrupt remapping was enabled on Intel Icelake CPUs. For instance:
DMAR: DRHD: handling fault status reg 2
DMAR: [INTR-REMAP] Request device [0xe2:0x00.0] fault index 0xa00 [fault reason 0x25] Blocked a compatibility format interrupt request
To fix this issue, dev_msi_info struct in dev struct maintains correct value of IRQ domain. VMD will use this information to assign proper IRQ domain to its child devices when it doesn't create a separate IRQ domain.
---v1->v2: Adding more information to commit log.
--- drivers/pci/controller/vmd.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/pci/controller/vmd.c +3 -0
@@ -853,6 +853,9 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)vmd_attach_resources (vmd);if (vmd->irq_domain)dev_set_msi_domain (&vmd->bus->dev, vmd->irq_domain); +else +dev_set_msi_domain (&vmd->bus->dev, +dev_get_msi_domain (&vmd->dev->dev));vmd_acpi_begin ();
Signature · 1 line
2.26.2
Nirmal Patel
maintainerPATCH v2 2/2 PCI: vmd: Revert 2565e5b69c44 ("PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU.")
In reply to Nirmal PatelRevert 2565e5b69c44 ("PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU.")
The commit 2565e5b69c44 was added as a workaround to keep MSI-X remapping enabled if IOMMU enables interrupt remapping. VMD would keep running in low performance mode. There is no dependency between MSI-X remapping by VMD and interrupt remapping by IOMMU.
---v1->v2: Add more information to the commit log.
--- drivers/pci/controller/vmd.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
drivers/pci/controller/vmd.c +1 -3
@@ -6,7 +6,6 @@#include <linux/device.h> #include <linux/interrupt.h> -#include <linux/iommu.h> #include <linux/irq.h> #include <linux/kernel.h> #include <linux/module.h>
@@ -813,8 +812,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)* acceptable because the guest is usually CPU-limited and MSI * remapping doesn't become a performance bottleneck. */ -if (iommu_capable (vmd->dev->dev.bus,IOMMU_CAP_INTR_REMAP ) || - !(features &VMD_FEAT_CAN_BYPASS_MSI_REMAP ) || +if (!(features &VMD_FEAT_CAN_BYPASS_MSI_REMAP ) || offset[0 ] || offset[1 ]) { ret =vmd_alloc_irqs (vmd);if (ret)
Signature · 1 line
2.26.2
Lorenzo Pieralisi
AppliedQuoted 6 lines · Nirmal Patel
Host OS fails to boot and DMAR errors were observed when interrupt remapping is enabled by
intel_iommubecause of the fact that VMD child devices are on different IRQ domain than all other PCI devices. Make sure VMD assigns proper IRQ domain to the child devices during device enumeration.
Quoted 6 lines
Nirmal Patel (2):
PCI: vmd: Assign VMD IRQ domain before enumeration
PCI: vmd: Revert2565e5b69c44("PCI: vmd: Do not disable MSI-X
remapping if interrupt remapping is enabled by IOMMU.")[...]
Applied to pci/vmd, thanks!
[1/2] PCI: vmd: Assign VMD IRQ domain before enumeration
https://git.kernel.org/lpieralisi/pci/c/886e67100b [2/2] PCI: vmd: Revert 2565e5b69c44 ("PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU.") https://git.kernel.org/lpieralisi/pci/c/c94f732e80
Thanks,
Lorenzo