PCI: Disable ATS via quirk before notifying IOMMU drivers
Subsystems (3)
PATCH v2 1/1 PCI: Disable ATS via quirk before notifying IOMMU drivers
Ensure that PCI devices with ATS disabled via quirk have it disabled before IOMMU drivers are notified about the device rather than after. Fix this by converting the existing quirks from final to early fixups and changing the quirk logic to set a new no_ats bit in struct pci_dev that prevents pci_dev.ats_cap from ever gettting set.
This change ensures that pci_ats_supported() takes quirks into account during iommu_ops.probe_device(), when IOMMU drivers are first notified about devices. It also ensures that pci_ats_supported() returns the same value when the device is released in iommu_ops.release_device().
Notably, the Intel IOMMU driver uses pci_ats_supported() in probe/release to determine whether to add/remove a device from a data structure, which easily leads to a use-after-free without this fix.
This change also makes disabling ATS via quirk behave the same way as the pci=noats command line option, in that pci_ats_init() bails immediately and never intializes pci_dev.ats_cap.
---
v2:
- Update the commit message with reasons why this is being fixed in the PCI core, rather than applying a point fix to the Intel IOMMU driver (Andy)
- Condense the pci_ats_disabled() and dev->no_ats checks into a single line in pci_ats_init()
- Reorder the no_ats bitfield to be after ats_stu since there is likely u8-sized gap there for alignment purposes
v1: https://lore.kernel.org/linux-pci/[email protected]/
drivers/pci/ats.c | 2 +- drivers/pci/quirks.c | 50 ++++++++++++++++++++++---------------------- include/linux/pci.h | 1 + 3 files changed, 27 insertions(+), 26 deletions(-)
drivers/pci/ats.c +1 -1
@@ -21,7 +21,7 @@ void pci_ats_init(struct pci_dev *dev) {int pos; -if (pci_ats_disabled ()) +if (pci_ats_disabled () || dev->no_ats)return ; pos =pci_find_ext_capability (dev,PCI_EXT_CAP_ID_ATS );
drivers/pci/quirks.c +25 -25
72 lines not on this page · raw message
include/linux/pci.h +1 -0
@@ -539,6 +539,7 @@ struct pci_dev { };u16 ats_cap;/* ATS Capability offset */ u8 ats_stu;/* ATS Smallest Translation Unit */ + unsigned int no_ats:1; /* ATS disabled via quirk */#endif #ifdef CONFIG_PCI_PRI u16 pri_cap;/* PRI Capability offset */
base-commit: 7df48e36313029e4c0907b2023905dd7213fd678
Signature · 1 line
2.53.0.1018.g2bb0e51243-goog
Bjorn Helgaas
AppliedQuoted 15 lines · David Matlack
Ensure that PCI devices with ATS disabled via quirk have it disabled before IOMMU drivers are notified about the device rather than after. Fix this by converting the existing quirks from final to early fixups and changing the quirk logic to set a new
no_atsbit in structpci_devthat prevents pci_dev.ats_cap from ever gettting set.This change ensures that
pci_ats_supported()takes quirks into account duringiommu_ops.probe_device(), when IOMMU drivers are first notified about devices. It also ensures thatpci_ats_supported()returns the same value when the device is released iniommu_ops.release_device().Notably, the Intel IOMMU driver uses
pci_ats_supported()in probe/release to determine whether to add/remove a device from a data structure, which easily leads to a use-after-free without this fix.
Quoted 7 lines
This change also makes disabling ATS via quirk behave the same way as the pci=noats command line option, in that
pci_ats_init()bails immediately and never intializes pci_dev.ats_cap.Fixes a18615b1cfc0 ("PCI: Disable ATS for specific Intel IPU E2000 devices")Signed-off-by David Matlack <[email protected]>
Applied to pci/enumeration for v7.1, thank you!
Quoted 131 lines
---
v2:
- Update the commit message with reasons why this is being fixed in the PCI core, rather than applying a point fix to the Intel IOMMU driver (Andy)
- Condense thepci_ats_disabled()anddev->no_atschecks into a single line inpci_ats_init()
- Reorder theno_atsbitfield to be afterats_stusince there is likely u8-sized gap there for alignment purposesv1: https://lore.kernel.org/linux-pci/[email protected]/
Cc Raghavendra Rao Ananta <[email protected]>Cc David Woodhouse <[email protected]>Cc Lu Baolu <[email protected]>Cc Andy Shevchenko <[email protected]>drivers/pci/ats.c | 2 +- drivers/pci/quirks.c | 50 ++++++++++++++++++++++---------------------- include/linux/pci.h | 1 + 3 files changed, 27 insertions(+), 26 deletions(-)drivers/pci/ats.c +1
-1@@ -21,7 +21,7 @@ void pci_ats_init(struct pci_dev *dev) {int pos; -if (pci_ats_disabled ()) +if (pci_ats_disabled () || dev->no_ats)return ; pos =pci_find_ext_capability (dev,PCI_EXT_CAP_ID_ATS );drivers/pci/quirks.c +25
-25@@ -5653,7 +5653,7 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_SERVERWORKS, 0x0422, quirk_no_ext_tags);static void quirk_no_ats (struct pci_dev *pdev) {pci_info (pdev,"disabling ATS\n" ); - pdev->ats_cap =0 ; + pdev->no_ats =1 ; }/* @@ -5676,25 +5676,25 @@ static void quirk_amd_harvest_no_ats(struct pci_dev *pdev) }/* AMD Stoney platform GPU */ -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x98e4 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x98e4 , quirk_amd_harvest_no_ats);/* AMD Iceland dGPU */ -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x6900 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x6900 , quirk_amd_harvest_no_ats);/* AMD Navi10 dGPU */ -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x7310 , quirk_amd_harvest_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x7312 , quirk_amd_harvest_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x7318 , quirk_amd_harvest_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x7319 , quirk_amd_harvest_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x731a , quirk_amd_harvest_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x731b , quirk_amd_harvest_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x731e , quirk_amd_harvest_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x731f , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x7310 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x7312 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x7318 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x7319 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x731a , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x731b , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x731e , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x731f , quirk_amd_harvest_no_ats);/* AMD Navi14 dGPU */ -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x7340 , quirk_amd_harvest_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x7341 , quirk_amd_harvest_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x7347 , quirk_amd_harvest_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x734f , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x7340 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x7341 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x7347 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x734f , quirk_amd_harvest_no_ats);/* AMD Raven platform iGPU */ -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x15d8 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x15d8 , quirk_amd_harvest_no_ats);/* * Intel IPU E2000 revisions before C0 implement incorrect endianness @@ -5705,15 +5705,15 @@ static void quirk_intel_e2000_no_ats(struct pci_dev *pdev)if (pdev->revision <0x20 ) quirk_no_ats(pdev); } -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x1451 , quirk_intel_e2000_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x1452 , quirk_intel_e2000_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x1453 , quirk_intel_e2000_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x1454 , quirk_intel_e2000_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x1455 , quirk_intel_e2000_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x1457 , quirk_intel_e2000_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x1459 , quirk_intel_e2000_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x145a , quirk_intel_e2000_no_ats); -DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x145c , quirk_intel_e2000_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_INTEL ,0x1451 , quirk_intel_e2000_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_INTEL ,0x1452 , quirk_intel_e2000_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_INTEL ,0x1453 , quirk_intel_e2000_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_INTEL ,0x1454 , quirk_intel_e2000_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_INTEL ,0x1455 , quirk_intel_e2000_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_INTEL ,0x1457 , quirk_intel_e2000_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_INTEL ,0x1459 , quirk_intel_e2000_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_INTEL ,0x145a , quirk_intel_e2000_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_INTEL ,0x145c , quirk_intel_e2000_no_ats);#endif /* CONFIG_PCI_ATS */ /* Freescale PCIe doesn't support MSI in RC mode */ include/linux/pci.h +1
-0@@ -539,6 +539,7 @@ struct pci_dev { };u16 ats_cap;/* ATS Capability offset */ u8 ats_stu;/* ATS Smallest Translation Unit */ + unsigned int no_ats:1; /* ATS disabled via quirk */#endif #ifdef CONFIG_PCI_PRI u16 pri_cap;/* PRI Capability offset */ base-commit:
7df48e36313029e4c0907b2023905dd7213fd678Signature · 1 line
2.53.0.1018.g2bb0e51243-goog
Baolu Lu
maintainerQuoted 6 lines · David Matlack
Ensure that PCI devices with ATS disabled via quirk have it disabled before IOMMU drivers are notified about the device rather than after. Fix this by converting the existing quirks from final to early fixups and changing the quirk logic to set a new
no_atsbit in structpci_devthat prevents pci_dev.ats_cap from ever gettting set.
Quoted 8 lines
This change ensures that
pci_ats_supported()takes quirks into account duringiommu_ops.probe_device(), when IOMMU drivers are first notified about devices. It also ensures thatpci_ats_supported()returns the same value when the device is released iniommu_ops.release_device().Notably, the Intel IOMMU driver uses
pci_ats_supported()in probe/release to determine whether to add/remove a device from a data structure, which easily leads to a use-after-free without this fix.
Can you please shed more light on the above issue? In my investigation, iommu_ops.probe_device() is always called after the no_ats quirk, regardless of whether this patch is applied.
The diff of the changes I made for testing is as follows:
drivers/iommu/intel/iommu.c +2 -0
@@ -3271,6 +3271,8 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev) info->pfsid =pci_dev_id (pci_physfn (pdev)); info->ats_qdep =pci_ats_queue_depth (pdev); } +pci_info (pdev,"ATS %s\n" , info->ats_supported ? +"supported" :"not supported" );if (sm_supported (iommu)) {if (pasid_supported (iommu)) {int features =pci_pasid_features (pdev);
drivers/pci/quirks.c +0 -0
@@ -5714,6 +5714,8 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1457, quirk_intel_e2000_no_ats); DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1459,
quirk_intel_e2000_no_ats);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145a,quirk_intel_e2000_no_ats);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145c,quirk_intel_e2000_no_ats);
+
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x0b25, quirk_no_ats);
#endif /* CONFIG_PCI_ATS */
/* Freescale PCIe doesn't support MSI in RC mode */
The related kernel messages are shown below:
# dmesg | grep "0000:00:01.0" [ 15.834944] pci 0000:00:01.0: [8086:0b25] type 00 class 0x088000 PCIe Root Complex Integrated Endpoint [ 15.836382] pci 0000:00:01.0: BAR 0 [mem 0x1e0fff980000-0x1e0fff99ffff 64bit pref] [ 15.836655] pci 0000:00:01.0: BAR 2 [mem 0x1e0fff900000-0x1e0fff93ffff 64bit pref] [ 15.837904] pci 0000:00:01.0: calling quirk_igfx_skip_te_disable+0x0/0xe0 @ 1 [ 15.838614] pci 0000:00:01.0: quirk_igfx_skip_te_disable+0x0/0xe0 took 0 usecs [ 21.205177] pci 0000:00:01.0: calling quirk_no_ats+0x0/0x40 @ 1 [ 21.206767] pci 0000:00:01.0: disabling ATS [ 21.207916] pci 0000:00:01.0: quirk_no_ats+0x0/0x40 took 1122 usecs [ 21.305357] pci 0000:00:01.0: DMAR: ATS not supported [ 21.306925] pci 0000:00:01.0: Adding to iommu group 4 [ 42.564912] idxd 0000:00:01.0: Intel(R) Accelerator Device (v200) [ 42.568653] probe of 0000:00:01.0 returned 0 after 87413 usecs
Anything I missed?
Thanks,
baolu
Quoted 29 lines
This change also makes disabling ATS via quirk behave the same way as the pci=noats command line option, in that
pci_ats_init()bails immediately and never intializes pci_dev.ats_cap.Fixes a18615b1cfc0 ("PCI: Disable ATS for specific Intel IPU E2000 devices")Signed-off-by David Matlack<[email protected]>---
v2:
- Update the commit message with reasons why this is being fixed in the PCI core, rather than applying a point fix to the Intel IOMMU driver (Andy)
- Condense thepci_ats_disabled()anddev->no_atschecks into a single line inpci_ats_init()
- Reorder theno_atsbitfield to be afterats_stusince there is likely u8-sized gap there for alignment purposesv1:https://lore.kernel.org/linux-pci/20260223184017.688212-1- [email protected]/
Cc Raghavendra Rao Ananta<[email protected]>Cc David Woodhouse<[email protected]>Cc Lu Baolu<[email protected]>Cc Andy Shevchenko<[email protected]>drivers/pci/ats.c | 2 +- drivers/pci/quirks.c | 50 ++++++++++++++++++++++---------------------- include/linux/pci.h | 1 + 3 files changed, 27 insertions(+), 26 deletions(-)
Quoted 78 lines · Baolu Lu
Quoted 6 lines · David Matlack
Ensure that PCI devices with ATS disabled via quirk have it disabled before IOMMU drivers are notified about the device rather than after. Fix this by converting the existing quirks from final to early fixups and changing the quirk logic to set a new
no_atsbit in structpci_devthat prevents pci_dev.ats_cap from ever gettting set.Quoted 8 lines
This change ensures that
pci_ats_supported()takes quirks into account duringiommu_ops.probe_device(), when IOMMU drivers are first notified about devices. It also ensures thatpci_ats_supported()returns the same value when the device is released iniommu_ops.release_device().Notably, the Intel IOMMU driver uses
pci_ats_supported()in probe/release to determine whether to add/remove a device from a data structure, which easily leads to a use-after-free without this fix.Can you please shed more light on the above issue? In my investigation,
iommu_ops.probe_device()is always called after theno_atsquirk, regardless of whether this patch is applied.The diff of the changes I made for testing is as follows:
drivers/iommu/intel/iommu.c +2
-0@@ -3271,6 +3271,8 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev) info->pfsid = pci_dev_id(pci_physfn(pdev)); info->ats_qdep =pci_ats_queue_depth (pdev); } +pci_info (pdev,"ATS %s\n" , info->ats_supported ? +"supported" :"not supported" );if (sm_supported (iommu)) {if (pasid_supported (iommu)) {int features = pci_pasid_features(pdev);drivers/pci/quirks.c +0
-0@@ -5714,6 +5714,8 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1457, quirk_intel_e2000_no_ats);DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x1459 , quirk_intel_e2000_no_ats);DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x145a , quirk_intel_e2000_no_ats); DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145c,
quirk_intel_e2000_no_ats);drivers/pci/quirks.c +2
-0+ +DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x0b25 , quirk_no_ats);#endif /* CONFIG_PCI_ATS */ /* Freescale PCIe doesn't support MSI in RC mode */ The related kernel messages are shown below:
# dmesg | grep "0000:00:01.0" [ 15.834944] pci 0000:00:01.0: [8086:0b25] type 00 class 0x088000 PCIe Root Complex Integrated Endpoint [ 15.836382] pci 0000:00:01.0: BAR 0 [mem 0x1e0fff980000-0x1e0fff99ffff 64bit pref] [ 15.836655] pci 0000:00:01.0: BAR 2 [mem 0x1e0fff900000-0x1e0fff93ffff 64bit pref] [ 15.837904] pci 0000:00:01.0: calling quirk_igfx_skip_te_disable+0x0/0xe0 @ 1 [ 15.838614] pci 0000:00:01.0: quirk_igfx_skip_te_disable+0x0/0xe0 took 0 usecs [ 21.205177] pci 0000:00:01.0: calling quirk_no_ats+0x0/0x40 @ 1 [ 21.206767] pci 0000:00:01.0: disabling ATS [ 21.207916] pci 0000:00:01.0: quirk_no_ats+0x0/0x40 took 1122 usecs [ 21.305357] pci 0000:00:01.0: DMAR: ATS not supported [ 21.306925] pci 0000:00:01.0: Adding to iommu group 4 [ 42.564912] idxd 0000:00:01.0: Intel(R) Accelerator Device (v200) [ 42.568653] probe of 0000:00:01.0 returned 0 after 87413 usecs
Quoted 1 line
Anything I missed?
The Closes link above has the details of my investigation:
https://lore.kernel.org/linux-iommu/[email protected]/
Can you check if you see the same ordering for VFs? That was the scenario where I saw the IOMMU drivers notified before final fixups were applied.
drivers/pci/quirks.c static void quirk_amd_harvest_no_ats(struct pci_dev *pdev) · line 5697
Quoted 3 lines · David Matlack
/* AMD Raven platform iGPU */
drivers/pci/quirks.c +1
-1-DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x15d8 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x15d8 , quirk_amd_harvest_no_ats);
Sashiko found a bug here that would leave ATS enabled on this device:
https://sashiko.dev/#/patchset/20260327211649.3816010-1-dmatlack%40google.com?patch=12646
Quoted 9 lines · David Matlack
drivers/pci/quirks.c static void quirk_amd_harvest_no_ats(struct pci_dev *pdev) · line 5697
Quoted 3 lines · David Matlack
/* AMD Raven platform iGPU */
drivers/pci/quirks.c +1
-1-DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x15d8 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x15d8 , quirk_amd_harvest_no_ats);Sashiko found a bug here that would leave ATS enabled on this device:
https://sashiko.dev/#/patchset/20260327211649.3816010-1-dmatlack%40google.com?patch=12646
Do you want to update the patch to use DECLARE_PCI_FIXUP_HEADER instead of DECLARE_PCI_FIXUP_EARLY, as Sashiko suggested?
I don't know the IOMMU init ordering with respect to PCI enumeration, but header fixups are run in pci_device_add() before it calls device_add().
Quoted 10 lines · Bjorn Helgaas
Quoted 8 lines · David Matlack
drivers/pci/quirks.c static void quirk_amd_harvest_no_ats(struct pci_dev *pdev) · line 5697
Quoted 3 lines · David Matlack
/* AMD Raven platform iGPU */
drivers/pci/quirks.c +1
-1-DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x15d8 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x15d8 , quirk_amd_harvest_no_ats);Sashiko found a bug here that would leave ATS enabled on this device:
Quoted 8 lines
Quoted 1 line
https://sashiko.dev/#/patchset/20260327211649.3816010-1-dmatlack%40google.com?patch=12646
Do you want to update the patch to use
DECLARE_PCI_FIXUP_HEADERinstead ofDECLARE_PCI_FIXUP_EARLY, as Sashiko suggested?I don't know the IOMMU init ordering with respect to PCI enumeration, but header fixups are run in
pci_device_add()before it callsdevice_add().
Yes I would like to work on another version. I have 3 things I want to investigate/fix:
* Fix the bug Sashiko found with early vs. header fixups.
* Investigate whether no_ats be a u8 instead of unsigned int to avoid
increasing sizeof(struct pci_dev).
* Investigate Boalu's finding. Maybe this issue is specific to VF
creation, and changing the order of calls there would be a better solution.
Quoted 3 lines · David Matlack
Quoted 2 lines · Bjorn Helgaas
Quoted 1 line · David Matlack
On Fri, Mar 27, 2026 at 2:17 PM David Matlack <[email protected]> wrote:
...
Quoted 7 lines
Yes I would like to work on another version. I have 3 things I want to investigate/fix:
* Fix the bug Sashiko found with early vs. header fixups.
* Investigate whether
no_atsbe a u8 instead of unsigned int to avoid
increasingsizeof(struct pci_dev).
Use pahole, it will check the layout and suggests correctly what's going on there. It's not a fuzzy AI :-)
Quoted 3 lines
* Investigate Boalu's finding. Maybe this issue is specific to VF
creation, and changing the order of calls there would be a better solution.
Signature · 2 lines
With Best Regards, Andy Shevchenko
Quoted 22 lines · David Matlack
Quoted 9 lines · Bjorn Helgaas
Quoted 8 lines · David Matlack
drivers/pci/quirks.c static void quirk_amd_harvest_no_ats(struct pci_dev *pdev) · line 5697
Quoted 3 lines · David Matlack
/* AMD Raven platform iGPU */
drivers/pci/quirks.c +1
-1-DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_ATI ,0x15d8 , quirk_amd_harvest_no_ats); +DECLARE_PCI_FIXUP_EARLY (PCI_VENDOR_ID_ATI ,0x15d8 , quirk_amd_harvest_no_ats);Sashiko found a bug here that would leave ATS enabled on this device:
Quoted 8 lines
Quoted 1 line
https://sashiko.dev/#/patchset/20260327211649.3816010-1-dmatlack%40google.com?patch=12646
Do you want to update the patch to use
DECLARE_PCI_FIXUP_HEADERinstead ofDECLARE_PCI_FIXUP_EARLY, as Sashiko suggested?I don't know the IOMMU init ordering with respect to PCI enumeration, but header fixups are run in
pci_device_add()before it callsdevice_add().Yes I would like to work on another version. I have 3 things I want to investigate/fix:
Quoted 6 lines
* Fix the bug Sashiko found with early vs. header fixups.
* Investigate whetherno_atsbe a u8 instead of unsigned int to avoid
increasingsizeof(struct pci_dev).
* Investigate Boalu's finding. Maybe this issue is specific to VF
creation, and changing the order of calls there would be a better solution.
OK, I'll drop this one for now and watch for an update.
Quoted 83 lines · David Matlack
Quoted 78 lines · Baolu Lu
Quoted 6 lines · David Matlack
Ensure that PCI devices with ATS disabled via quirk have it disabled before IOMMU drivers are notified about the device rather than after. Fix this by converting the existing quirks from final to early fixups and changing the quirk logic to set a new
no_atsbit in structpci_devthat prevents pci_dev.ats_cap from ever gettting set.Quoted 8 lines
This change ensures that
pci_ats_supported()takes quirks into account duringiommu_ops.probe_device(), when IOMMU drivers are first notified about devices. It also ensures thatpci_ats_supported()returns the same value when the device is released iniommu_ops.release_device().Notably, the Intel IOMMU driver uses
pci_ats_supported()in probe/release to determine whether to add/remove a device from a data structure, which easily leads to a use-after-free without this fix.Can you please shed more light on the above issue? In my investigation,
iommu_ops.probe_device()is always called after theno_atsquirk, regardless of whether this patch is applied.The diff of the changes I made for testing is as follows:
drivers/iommu/intel/iommu.c +2
-0@@ -3271,6 +3271,8 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev) info->pfsid = pci_dev_id(pci_physfn(pdev)); info->ats_qdep =pci_ats_queue_depth (pdev); } +pci_info (pdev,"ATS %s\n" , info->ats_supported ? +"supported" :"not supported" );if (sm_supported (iommu)) {if (pasid_supported (iommu)) {int features = pci_pasid_features(pdev);drivers/pci/quirks.c +0
-0@@ -5714,6 +5714,8 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1457, quirk_intel_e2000_no_ats);DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x1459 , quirk_intel_e2000_no_ats);DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x145a , quirk_intel_e2000_no_ats); DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145c,
quirk_intel_e2000_no_ats);drivers/pci/quirks.c +2
-0+ +DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_INTEL ,0x0b25 , quirk_no_ats);#endif /* CONFIG_PCI_ATS */ /* Freescale PCIe doesn't support MSI in RC mode */ The related kernel messages are shown below:
# dmesg | grep "0000:00:01.0" [ 15.834944] pci 0000:00:01.0: [8086:0b25] type 00 class 0x088000 PCIe Root Complex Integrated Endpoint [ 15.836382] pci 0000:00:01.0: BAR 0 [mem 0x1e0fff980000-0x1e0fff99ffff 64bit pref] [ 15.836655] pci 0000:00:01.0: BAR 2 [mem 0x1e0fff900000-0x1e0fff93ffff 64bit pref] [ 15.837904] pci 0000:00:01.0: calling quirk_igfx_skip_te_disable+0x0/0xe0 @ 1 [ 15.838614] pci 0000:00:01.0: quirk_igfx_skip_te_disable+0x0/0xe0 took 0 usecs [ 21.205177] pci 0000:00:01.0: calling quirk_no_ats+0x0/0x40 @ 1 [ 21.206767] pci 0000:00:01.0: disabling ATS [ 21.207916] pci 0000:00:01.0: quirk_no_ats+0x0/0x40 took 1122 usecs [ 21.305357] pci 0000:00:01.0: DMAR: ATS not supported [ 21.306925] pci 0000:00:01.0: Adding to iommu group 4 [ 42.564912] idxd 0000:00:01.0: Intel(R) Accelerator Device (v200) [ 42.568653] probe of 0000:00:01.0 returned 0 after 87413 usecsQuoted 1 line
Anything I missed?
The Closes link above has the details of my investigation:
Quoted 5 lines
https://lore.kernel.org/linux-iommu/[email protected]/
Can you check if you see the same ordering for VFs? That was the scenario where I saw the IOMMU drivers notified before final fixups were applied.
During boot I observe the same order as you. PCI final fixups get applied twice actually, and both times is before iommu probe_device().
First time:
[ 1.687987][ T1] pci 0000:00:04.0: pci_fixup_device [ 1.690987][ T1] WARNING: drivers/pci/quirks.c:227 at pci_fixup_device+0xd5/0x230, CPU#5: swapper/0/1 [ 1.708986][ T1] RIP: 0010:pci_fixup_device+0xf9/0x230 [ 1.755989][ T1] pci_bus_add_device+0x31/0x90 [ 1.758986][ T1] pci_bus_add_devices+0x39/0x70 [ 1.760986][ T1] acpi_pci_root_add+0x815/0xa60 [ 1.767986][ T1] acpi_bus_attach+0x260/0x2f0 [ 1.773988][ T1] device_for_each_child+0x6a/0xa0 [ 1.775986][ T1] acpi_dev_for_each_child+0x39/0x60 [ 1.780986][ T1] acpi_bus_attach+0xe8/0x2f0 [ 1.785987][ T1] device_for_each_child+0x6a/0xa0 [ 1.787989][ T1] acpi_dev_for_each_child+0x39/0x60 [ 1.792985][ T1] acpi_bus_attach+0xe8/0x2f0 [ 1.795986][ T1] acpi_bus_scan+0x6a/0x200 [ 1.799986][ T1] acpi_scan_init+0xb7/0x1c0 [ 1.803988][ T1] acpi_init+0xad/0xd0 [ 1.808986][ T1] do_one_initcall+0xba/0x2a0 [ 1.825986][ T1] do_initcall_level+0x7c/0xf0 [ 1.827988][ T1] do_initcalls+0x43/0x70 [ 1.829986][ T1] kernel_init_freeable+0xd7/0x140 [ 1.834985][ T1] kernel_init+0x1a/0x130 [ 1.836986][ T1] ret_from_fork+0xf4/0x280 [ 1.840985][ T1] ret_from_fork_asm+0x1a/0x30
Second time:
[ 4.156625][ T1] pci 0000:00:04.0: pci_fixup_device [ 4.159282][ T1] WARNING: drivers/pci/quirks.c:227 at pci_fixup_device+0xd5/0x230, CPU#1: swapper/0/1 [ 4.178115][ T1] RIP: 0010:pci_fixup_device+0xf9/0x230 [ 4.228013][ T1] pci_apply_final_quirks+0x6a/0x170 [ 4.233485][ T1] do_one_initcall+0xba/0x2a0 [ 4.251171][ T1] do_initcall_level+0x7c/0xf0 [ 4.253521][ T1] do_initcalls+0x43/0x70 [ 4.255653][ T1] kernel_init_freeable+0xd7/0x140 [ 4.260602][ T1] kernel_init+0x1a/0x130 [ 4.262745][ T1] ret_from_fork+0xf4/0x280 [ 4.267345][ T1] ret_from_fork_asm+0x1a/0x30
Then iommu probe_device() happens:
[ 5.744756][ T1] pci 0000:00:04.0: intel_iommu_probe_device [ 5.747688][ T1] WARNING: drivers/iommu/intel/iommu.c:3257 at intel_iommu_probe_device+0x3e9/0x840, CPU#1: swapper/0/1 [ 5.767127][ T1] RIP: 0010:intel_iommu_probe_device+0x403/0x840 [ 5.816998][ T1] __iommu_probe_device+0xe7/0x4d0 [ 5.819522][ T1] probe_iommu_group+0x29/0x50 [ 5.824593][ T1] bus_for_each_dev+0x111/0x130 [ 5.826965][ T1] iommu_device_register+0xfb/0x250 [ 5.829492][ T1] intel_iommu_init+0x36f/0x450 [ 5.834374][ T1] pci_iommu_init+0x16/0x40 [ 5.836614][ T1] do_one_initcall+0xba/0x2a0 [ 5.854121][ T1] do_initcall_level+0x7c/0xf0 [ 5.856450][ T1] do_initcalls+0x43/0x70 [ 5.858563][ T1] kernel_init_freeable+0xd7/0x140 [ 5.863427][ T1] kernel_init+0x1a/0x130 [ 5.865522][ T1] ret_from_fork+0xf4/0x280 [ 5.870169][ T1] ret_from_fork_asm+0x1a/0x30
But when VFs are created the order is reversed:
[ 545.718507][ T242] pci 0000:01:00.1: intel_iommu_probe_device [ 545.718627][ T242] WARNING: drivers/iommu/intel/iommu.c:3257 at intel_iommu_probe_device+0x3e9/0x840, CPU#0: bash/242 [ 545.719468][ T242] RIP: 0010:intel_iommu_probe_device+0x403/0x840 [ 545.721181][ T242] __iommu_probe_device+0xe7/0x4d0 [ 545.721278][ T242] iommu_bus_notifier+0x37/0x100 [ 545.721373][ T242] blocking_notifier_call_chain+0x62/0x100 [ 545.721485][ T242] bus_notify+0x97/0xc0 [ 545.721567][ T242] device_add+0x222/0x400 [ 545.721646][ T242] pci_device_add+0x488/0x6e0 [ 545.721746][ T242] pci_iov_add_virtfn+0x2f2/0x3e0 [ 545.721935][ T242] sriov_add_vfs+0x2c/0x60 [ 545.722064][ T242] sriov_enable+0x306/0x4a0 [ 545.722157][ T242] virtio_pci_sriov_configure+0x6a/0x90 [ 545.722257][ T242] sriov_numvfs_store+0xd7/0x1c0 [ 545.722349][ T242] kernfs_fop_write_iter+0xfe/0x180 [ 545.722453][ T242] vfs_write+0x2e4/0x460 [ 545.722533][ T242] ksys_write+0x70/0x100 [ 545.722613][ T242] do_syscall_64+0xd1/0x8b0 [ 545.722832][ T242] entry_SYSCALL_64_after_hwframe+0x76/0x7e
[ 545.725241][ T242] pci 0000:01:00.1: pci_fixup_device [ 545.725339][ T242] WARNING: drivers/pci/quirks.c:227 at pci_fixup_device+0xd5/0x230, CPU#0: bash/242 [ 545.727806][ T242] pci_bus_add_device+0x31/0x90 [ 545.727902][ T242] pci_iov_add_virtfn+0x3c9/0x3e0 [ 545.728118][ T242] sriov_add_vfs+0x2c/0x60 [ 545.728226][ T242] sriov_enable+0x306/0x4a0 [ 545.728320][ T242] virtio_pci_sriov_configure+0x6a/0x90 [ 545.728416][ T242] sriov_numvfs_store+0xd7/0x1c0 [ 545.728512][ T242] kernfs_fop_write_iter+0xfe/0x180 [ 545.728609][ T242] vfs_write+0x2e4/0x460 [ 545.728690][ T242] ksys_write+0x70/0x100 [ 545.728771][ T242] do_syscall_64+0xd1/0x8b0 [ 545.728866][ T242] ? clear_bhb_loop+0x30/0x80 [ 545.728961][ T242] entry_SYSCALL_64_after_hwframe+0x76/0x7e
My naive reading of the PCI hotplug code makes me think it has the same ordering as VF creation. More generally, any PCI device that is added after the IOMMU bus notifiers are set up will probe the device before final fixups are applied. I can clarify that in the commit message in v3.
Here is the patch I used to collect the traces above:
drivers/iommu/intel/iommu.c +1 -0
@@ -3254,6 +3254,7 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev) info->dev = dev; info->iommu = iommu;if (dev_is_pci (dev)) { +pci_WARN (to_pci_dev (dev),1 ,"intel_iommu_probe_device\n" );if (ecap_dev_iotlb_support (iommu->ecap) &&pci_ats_supported (pdev) &&dmar_ats_supported (pdev, iommu)) {
drivers/pci/quirks.c +1 -0
@@ -224,6 +224,7 @@ void pci_fixup_device(enum pci_fixup_pass pass, struct pci_dev *dev)break ;case pci_fixup_final: +pci_WARN (dev,1 ,"pci_fixup_device\n" );if (!pci_apply_fixup_final_quirks)return ; start = __start_pci_fixups_final;