Skip to contentSkip to conversation

PCI: Disable ATS via quirk before notifying IOMMU drivers

PATCH Applied David Matlack 4 participants 10 messages Mar 27 – Apr 2

Subsystems (3)

v2 of 5 Applied Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 PCI: Disable ATS via quirk before notifying IOMMU drivers

Ensure that PCI devices with ATS disabled via quirk have it disabled before IOMMU drivers are notified about the device rather than after. Fix this by converting the existing quirks from final to early fixups and changing the quirk logic to set a new no_ats bit in struct pci_dev that prevents pci_dev.ats_cap from ever gettting set.

This change ensures that pci_ats_supported() takes quirks into account during iommu_ops.probe_device(), when IOMMU drivers are first notified about devices. It also ensures that pci_ats_supported() returns the same value when the device is released in iommu_ops.release_device().

Notably, the Intel IOMMU driver uses pci_ats_supported() in probe/release to determine whether to add/remove a device from a data structure, which easily leads to a use-after-free without this fix.

This change also makes disabling ATS via quirk behave the same way as the pci=noats command line option, in that pci_ats_init() bails immediately and never intializes pci_dev.ats_cap.

Fixes a18615b1cfc0 ("PCI: Disable ATS for specific Intel IPU E2000 devices")

---
v2:
- Update the commit message with reasons why this is being fixed in the PCI core, rather than applying a point fix to the Intel IOMMU driver (Andy)
- Condense the pci_ats_disabled() and dev->no_ats checks into a single line in pci_ats_init()
- Reorder the no_ats bitfield to be after ats_stu since there is likely u8-sized gap there for alignment purposes

v1: https://lore.kernel.org/linux-pci/[email protected]/

Cc Raghavendra Rao Ananta <[email protected]>
Cc David Woodhouse <[email protected]>
Cc Lu Baolu <[email protected]>
Cc Andy Shevchenko <[email protected]>
 drivers/pci/ats.c    |  2 +-
 drivers/pci/quirks.c | 50 ++++++++++++++++++++++----------------------
 include/linux/pci.h  |  1 +
 3 files changed, 27 insertions(+), 26 deletions(-)
drivers/pci/ats.c +1 -1
index ec6c8dbdc5e9..ceb6f5d3cb10 100644
@@ -21,7 +21,7 @@ void pci_ats_init(struct pci_dev *dev)
 {
 	int pos;
 
-	if (pci_ats_disabled())
+	if (pci_ats_disabled() || dev->no_ats)
 		return;
 
 	pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ATS);
drivers/pci/quirks.c +25 -25

72 lines not on this page · raw message

include/linux/pci.h +1 -0
index 1c270f1d5123..850100f209e3 100644
@@ -539,6 +539,7 @@ struct pci_dev {
 	};
 	u16		ats_cap;	/* ATS Capability offset */
 	u8		ats_stu;	/* ATS Smallest Translation Unit */
+	unsigned int	no_ats:1;	/* ATS disabled via quirk */
 #endif
 #ifdef CONFIG_PCI_PRI
 	u16		pri_cap;	/* PRI Capability offset */

base-commit: 7df48e36313029e4c0907b2023905dd7213fd678

Signature · 1 line
2.53.0.1018.g2bb0e51243-goog
#1 lore raw
Quoted 15 lines · David Matlack

Ensure that PCI devices with ATS disabled via quirk have it disabled before IOMMU drivers are notified about the device rather than after. Fix this by converting the existing quirks from final to early fixups and changing the quirk logic to set a new no_ats bit in struct pci_dev that prevents pci_dev.ats_cap from ever gettting set.

This change ensures that pci_ats_supported() takes quirks into account during iommu_ops.probe_device(), when IOMMU drivers are first notified about devices. It also ensures that pci_ats_supported() returns the same value when the device is released in iommu_ops.release_device().

Notably, the Intel IOMMU driver uses pci_ats_supported() in probe/release to determine whether to add/remove a device from a data structure, which easily leads to a use-after-free without this fix.

Quoted 7 lines

This change also makes disabling ATS via quirk behave the same way as the pci=noats command line option, in that pci_ats_init() bails immediately and never intializes pci_dev.ats_cap.

Fixes a18615b1cfc0 ("PCI: Disable ATS for specific Intel IPU E2000 devices")

Applied to pci/enumeration for v7.1, thank you!

Quoted 131 lines

---
v2:
- Update the commit message with reasons why this is being fixed in the PCI core, rather than applying a point fix to the Intel IOMMU driver (Andy)
- Condense the pci_ats_disabled() and dev->no_ats checks into a single line in pci_ats_init()
- Reorder the no_ats bitfield to be after ats_stu since there is likely u8-sized gap there for alignment purposes

v1: https://lore.kernel.org/linux-pci/[email protected]/

Cc Raghavendra Rao Ananta <[email protected]>
Cc David Woodhouse <[email protected]>
Cc Lu Baolu <[email protected]>
Cc Andy Shevchenko <[email protected]>
 drivers/pci/ats.c    |  2 +-
 drivers/pci/quirks.c | 50 ++++++++++++++++++++++----------------------
 include/linux/pci.h  |  1 +
 3 files changed, 27 insertions(+), 26 deletions(-)
drivers/pci/ats.c +1 -1
index ec6c8dbdc5e9..ceb6f5d3cb10 100644
@@ -21,7 +21,7 @@ void pci_ats_init(struct pci_dev *dev)
 {
 	int pos;
 
-	if (pci_ats_disabled())
+	if (pci_ats_disabled() || dev->no_ats)
 		return;
 
 	pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ATS);
drivers/pci/quirks.c +25 -25
index 48946cca4be7..2c7e11830e45 100644
@@ -5653,7 +5653,7 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_SERVERWORKS, 0x0422, quirk_no_ext_tags);
 static void quirk_no_ats(struct pci_dev *pdev)
 {
 	pci_info(pdev, "disabling ATS\n");
-	pdev->ats_cap = 0;
+	pdev->no_ats = 1;
 }
 
 /*
@@ -5676,25 +5676,25 @@ static void quirk_amd_harvest_no_ats(struct pci_dev *pdev)
 }
 
 /* AMD Stoney platform GPU */
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x98e4, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x98e4, quirk_amd_harvest_no_ats);
 /* AMD Iceland dGPU */
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x6900, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x6900, quirk_amd_harvest_no_ats);
 /* AMD Navi10 dGPU */
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x7310, quirk_amd_harvest_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x7312, quirk_amd_harvest_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x7318, quirk_amd_harvest_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x7319, quirk_amd_harvest_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x731a, quirk_amd_harvest_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x731b, quirk_amd_harvest_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x731e, quirk_amd_harvest_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x731f, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x7310, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x7312, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x7318, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x7319, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x731a, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x731b, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x731e, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x731f, quirk_amd_harvest_no_ats);
 /* AMD Navi14 dGPU */
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x7340, quirk_amd_harvest_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x7341, quirk_amd_harvest_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x7347, quirk_amd_harvest_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x734f, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x7340, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x7341, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x7347, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x734f, quirk_amd_harvest_no_ats);
 /* AMD Raven platform iGPU */
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x15d8, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x15d8, quirk_amd_harvest_no_ats);
 
 /*
  * Intel IPU E2000 revisions before C0 implement incorrect endianness
@@ -5705,15 +5705,15 @@ static void quirk_intel_e2000_no_ats(struct pci_dev *pdev)
 	if (pdev->revision < 0x20)
 		quirk_no_ats(pdev);
 }
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1451, quirk_intel_e2000_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1452, quirk_intel_e2000_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1453, quirk_intel_e2000_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1454, quirk_intel_e2000_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1455, quirk_intel_e2000_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1457, quirk_intel_e2000_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1459, quirk_intel_e2000_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145a, quirk_intel_e2000_no_ats);
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145c, quirk_intel_e2000_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1451, quirk_intel_e2000_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1452, quirk_intel_e2000_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1453, quirk_intel_e2000_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1454, quirk_intel_e2000_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1455, quirk_intel_e2000_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1457, quirk_intel_e2000_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1459, quirk_intel_e2000_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x145a, quirk_intel_e2000_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x145c, quirk_intel_e2000_no_ats);
 #endif /* CONFIG_PCI_ATS */
 
 /* Freescale PCIe doesn't support MSI in RC mode */
include/linux/pci.h +1 -0
index 1c270f1d5123..850100f209e3 100644
@@ -539,6 +539,7 @@ struct pci_dev {
 	};
 	u16		ats_cap;	/* ATS Capability offset */
 	u8		ats_stu;	/* ATS Smallest Translation Unit */
+	unsigned int	no_ats:1;	/* ATS disabled via quirk */
 #endif
 #ifdef CONFIG_PCI_PRI
 	u16		pri_cap;	/* PRI Capability offset */

base-commit: 7df48e36313029e4c0907b2023905dd7213fd678

Signature · 1 line
2.53.0.1018.g2bb0e51243-goog
maintainer
#2 lore raw
In reply to David Matlack
Quoted 6 lines · David Matlack

Ensure that PCI devices with ATS disabled via quirk have it disabled before IOMMU drivers are notified about the device rather than after. Fix this by converting the existing quirks from final to early fixups and changing the quirk logic to set a new no_ats bit in struct pci_dev that prevents pci_dev.ats_cap from ever gettting set.

Quoted 8 lines

This change ensures that pci_ats_supported() takes quirks into account during iommu_ops.probe_device(), when IOMMU drivers are first notified about devices. It also ensures that pci_ats_supported() returns the same value when the device is released in iommu_ops.release_device().

Notably, the Intel IOMMU driver uses pci_ats_supported() in probe/release to determine whether to add/remove a device from a data structure, which easily leads to a use-after-free without this fix.

Can you please shed more light on the above issue? In my investigation, iommu_ops.probe_device() is always called after the no_ats quirk, regardless of whether this patch is applied.

The diff of the changes I made for testing is as follows:

drivers/iommu/intel/iommu.c +2 -0
index 442271a1b92a..c024964ac53b 100644
@@ -3271,6 +3271,8 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
                                info->pfsid = pci_dev_id(pci_physfn(pdev));
                        info->ats_qdep = pci_ats_queue_depth(pdev);
                }
+               pci_info(pdev, "ATS %s\n", info->ats_supported ?
+                        "supported" : "not supported");
                if (sm_supported(iommu)) {
                        if (pasid_supported(iommu)) {
                                int features = pci_pasid_features(pdev);
drivers/pci/quirks.c +0 -0
index 48946cca4be7..c63616d108b7 100644
@@ -5714,6 +5714,8 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1457, quirk_intel_e2000_no_ats);
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1459, 

quirk_intel_e2000_no_ats);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145a,
quirk_intel_e2000_no_ats);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145c,
quirk_intel_e2000_no_ats);
+
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x0b25, quirk_no_ats);
#endif /* CONFIG_PCI_ATS */

/* Freescale PCIe doesn't support MSI in RC mode */

The related kernel messages are shown below:

# dmesg | grep "0000:00:01.0"
[   15.834944] pci 0000:00:01.0: [8086:0b25] type 00 class 0x088000 PCIe Root Complex Integrated Endpoint
[   15.836382] pci 0000:00:01.0: BAR 0 [mem 0x1e0fff980000-0x1e0fff99ffff 64bit pref]
[   15.836655] pci 0000:00:01.0: BAR 2 [mem 0x1e0fff900000-0x1e0fff93ffff 64bit pref]
[   15.837904] pci 0000:00:01.0: calling quirk_igfx_skip_te_disable+0x0/0xe0 @ 1
[   15.838614] pci 0000:00:01.0: quirk_igfx_skip_te_disable+0x0/0xe0 took 0 usecs
[   21.205177] pci 0000:00:01.0: calling  quirk_no_ats+0x0/0x40 @ 1
[   21.206767] pci 0000:00:01.0: disabling ATS
[   21.207916] pci 0000:00:01.0: quirk_no_ats+0x0/0x40 took 1122 usecs
[   21.305357] pci 0000:00:01.0: DMAR: ATS not supported
[   21.306925] pci 0000:00:01.0: Adding to iommu group 4
[   42.564912] idxd 0000:00:01.0: Intel(R) Accelerator Device (v200)
[   42.568653] probe of 0000:00:01.0 returned 0 after 87413 usecs

Anything I missed?

Thanks,
baolu

Quoted 29 lines

This change also makes disabling ATS via quirk behave the same way as the pci=noats command line option, in that pci_ats_init() bails immediately and never intializes pci_dev.ats_cap.

Fixes a18615b1cfc0 ("PCI: Disable ATS for specific Intel IPU E2000 devices")

---
v2:
- Update the commit message with reasons why this is being fixed in the PCI core, rather than applying a point fix to the Intel IOMMU driver (Andy)
- Condense the pci_ats_disabled() and dev->no_ats checks into a single line in pci_ats_init()
- Reorder the no_ats bitfield to be after ats_stu since there is likely u8-sized gap there for alignment purposes

v1:https://lore.kernel.org/linux-pci/20260223184017.688212-1- [email protected]/

Cc Raghavendra Rao Ananta<[email protected]>
Cc David Woodhouse<[email protected]>
Cc Lu Baolu<[email protected]>
Cc Andy Shevchenko<[email protected]>
  drivers/pci/ats.c    |  2 +-
  drivers/pci/quirks.c | 50 ++++++++++++++++++++++----------------------
  include/linux/pci.h  |  1 +
  3 files changed, 27 insertions(+), 26 deletions(-)
#3 lore raw
Quoted 78 lines · Baolu Lu
Quoted 6 lines · David Matlack

Ensure that PCI devices with ATS disabled via quirk have it disabled before IOMMU drivers are notified about the device rather than after. Fix this by converting the existing quirks from final to early fixups and changing the quirk logic to set a new no_ats bit in struct pci_dev that prevents pci_dev.ats_cap from ever gettting set.

Quoted 8 lines

This change ensures that pci_ats_supported() takes quirks into account during iommu_ops.probe_device(), when IOMMU drivers are first notified about devices. It also ensures that pci_ats_supported() returns the same value when the device is released in iommu_ops.release_device().

Notably, the Intel IOMMU driver uses pci_ats_supported() in probe/release to determine whether to add/remove a device from a data structure, which easily leads to a use-after-free without this fix.

Can you please shed more light on the above issue? In my investigation, iommu_ops.probe_device() is always called after the no_ats quirk, regardless of whether this patch is applied.

The diff of the changes I made for testing is as follows:

drivers/iommu/intel/iommu.c +2 -0
index 442271a1b92a..c024964ac53b 100644
@@ -3271,6 +3271,8 @@ static struct iommu_device
 *intel_iommu_probe_device(struct device *dev)
                                 info->pfsid = pci_dev_id(pci_physfn(pdev));
                         info->ats_qdep = pci_ats_queue_depth(pdev);
                 }
+               pci_info(pdev, "ATS %s\n", info->ats_supported ?
+                        "supported" : "not supported");
                 if (sm_supported(iommu)) {
                         if (pasid_supported(iommu)) {
                                 int features = pci_pasid_features(pdev);
drivers/pci/quirks.c +0 -0
index 48946cca4be7..c63616d108b7 100644
@@ -5714,6 +5714,8 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,
 0x1457, quirk_intel_e2000_no_ats);
  DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1459,
 quirk_intel_e2000_no_ats);
  DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145a,
 quirk_intel_e2000_no_ats);
  DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145c,

quirk_intel_e2000_no_ats);

drivers/pci/quirks.c +2 -0
+
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x0b25, quirk_no_ats);
  #endif /* CONFIG_PCI_ATS */
 
  /* Freescale PCIe doesn't support MSI in RC mode */

The related kernel messages are shown below:

# dmesg | grep "0000:00:01.0"
[   15.834944] pci 0000:00:01.0: [8086:0b25] type 00 class 0x088000 PCIe
Root Complex Integrated Endpoint
[   15.836382] pci 0000:00:01.0: BAR 0 [mem
0x1e0fff980000-0x1e0fff99ffff 64bit pref]
[   15.836655] pci 0000:00:01.0: BAR 2 [mem
0x1e0fff900000-0x1e0fff93ffff 64bit pref]
[   15.837904] pci 0000:00:01.0: calling
quirk_igfx_skip_te_disable+0x0/0xe0 @ 1
[   15.838614] pci 0000:00:01.0: quirk_igfx_skip_te_disable+0x0/0xe0
took 0 usecs
[   21.205177] pci 0000:00:01.0: calling  quirk_no_ats+0x0/0x40 @ 1
[   21.206767] pci 0000:00:01.0: disabling ATS
[   21.207916] pci 0000:00:01.0: quirk_no_ats+0x0/0x40 took 1122 usecs
[   21.305357] pci 0000:00:01.0: DMAR: ATS not supported
[   21.306925] pci 0000:00:01.0: Adding to iommu group 4
[   42.564912] idxd 0000:00:01.0: Intel(R) Accelerator Device (v200)
[   42.568653] probe of 0000:00:01.0 returned 0 after 87413 usecs
Quoted 1 line

Anything I missed?

The Closes link above has the details of my investigation:

https://lore.kernel.org/linux-iommu/[email protected]/

Can you check if you see the same ordering for VFs? That was the scenario where I saw the IOMMU drivers notified before final fixups were applied.

#5 lore raw
In reply to David Matlack

drivers/pci/quirks.c static void quirk_amd_harvest_no_ats(struct pci_dev *pdev) · line 5697

Quoted 3 lines · David Matlack

/* AMD Raven platform iGPU */

drivers/pci/quirks.c +1 -1
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x15d8, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x15d8, quirk_amd_harvest_no_ats);

Sashiko found a bug here that would leave ATS enabled on this device:

https://sashiko.dev/#/patchset/20260327211649.3816010-1-dmatlack%40google.com?patch=12646

#6 lore raw
Quoted 9 lines · David Matlack

drivers/pci/quirks.c static void quirk_amd_harvest_no_ats(struct pci_dev *pdev) · line 5697

Quoted 3 lines · David Matlack

/* AMD Raven platform iGPU */

drivers/pci/quirks.c +1 -1
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x15d8, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x15d8, quirk_amd_harvest_no_ats);

Sashiko found a bug here that would leave ATS enabled on this device:

https://sashiko.dev/#/patchset/20260327211649.3816010-1-dmatlack%40google.com?patch=12646

Do you want to update the patch to use DECLARE_PCI_FIXUP_HEADER instead of DECLARE_PCI_FIXUP_EARLY, as Sashiko suggested?

I don't know the IOMMU init ordering with respect to PCI enumeration, but header fixups are run in pci_device_add() before it calls device_add().

#7 lore raw
Quoted 10 lines · Bjorn Helgaas
Quoted 8 lines · David Matlack

drivers/pci/quirks.c static void quirk_amd_harvest_no_ats(struct pci_dev *pdev) · line 5697

Quoted 3 lines · David Matlack

/* AMD Raven platform iGPU */

drivers/pci/quirks.c +1 -1
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x15d8, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x15d8, quirk_amd_harvest_no_ats);

Sashiko found a bug here that would leave ATS enabled on this device:

Quoted 8 lines
Quoted 1 line

https://sashiko.dev/#/patchset/20260327211649.3816010-1-dmatlack%40google.com?patch=12646

Do you want to update the patch to use DECLARE_PCI_FIXUP_HEADER instead of DECLARE_PCI_FIXUP_EARLY, as Sashiko suggested?

I don't know the IOMMU init ordering with respect to PCI enumeration, but header fixups are run in pci_device_add() before it calls device_add().

Yes I would like to work on another version. I have 3 things I want to investigate/fix:

* Fix the bug Sashiko found with early vs. header fixups.
* Investigate whether no_ats be a u8 instead of unsigned int to avoid
increasing sizeof(struct pci_dev).
* Investigate Boalu's finding. Maybe this issue is specific to VF
creation, and changing the order of calls there would be a better solution.

#8 lore raw
Quoted 3 lines · David Matlack
Quoted 2 lines · Bjorn Helgaas
Quoted 1 line · David Matlack

On Fri, Mar 27, 2026 at 2:17 PM David Matlack <[email protected]> wrote:

...

Quoted 7 lines

Yes I would like to work on another version. I have 3 things I want to investigate/fix:

* Fix the bug Sashiko found with early vs. header fixups.

* Investigate whether no_ats be a u8 instead of unsigned int to avoid
increasing sizeof(struct pci_dev).

Use pahole, it will check the layout and suggests correctly what's going on there. It's not a fuzzy AI :-)

Quoted 3 lines

* Investigate Boalu's finding. Maybe this issue is specific to VF
creation, and changing the order of calls there would be a better solution.

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#9 lore raw
In reply to David Matlack
Quoted 22 lines · David Matlack
Quoted 9 lines · Bjorn Helgaas
Quoted 8 lines · David Matlack

drivers/pci/quirks.c static void quirk_amd_harvest_no_ats(struct pci_dev *pdev) · line 5697

Quoted 3 lines · David Matlack

/* AMD Raven platform iGPU */

drivers/pci/quirks.c +1 -1
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x15d8, quirk_amd_harvest_no_ats);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_ATI, 0x15d8, quirk_amd_harvest_no_ats);

Sashiko found a bug here that would leave ATS enabled on this device:

Quoted 8 lines
Quoted 1 line

https://sashiko.dev/#/patchset/20260327211649.3816010-1-dmatlack%40google.com?patch=12646

Do you want to update the patch to use DECLARE_PCI_FIXUP_HEADER instead of DECLARE_PCI_FIXUP_EARLY, as Sashiko suggested?

I don't know the IOMMU init ordering with respect to PCI enumeration, but header fixups are run in pci_device_add() before it calls device_add().

Yes I would like to work on another version. I have 3 things I want to investigate/fix:

Quoted 6 lines

* Fix the bug Sashiko found with early vs. header fixups.
* Investigate whether no_ats be a u8 instead of unsigned int to avoid
increasing sizeof(struct pci_dev).
* Investigate Boalu's finding. Maybe this issue is specific to VF
creation, and changing the order of calls there would be a better solution.

OK, I'll drop this one for now and watch for an update.

#4 lore raw
In reply to David Matlack
Quoted 83 lines · David Matlack
Quoted 78 lines · Baolu Lu
Quoted 6 lines · David Matlack

Ensure that PCI devices with ATS disabled via quirk have it disabled before IOMMU drivers are notified about the device rather than after. Fix this by converting the existing quirks from final to early fixups and changing the quirk logic to set a new no_ats bit in struct pci_dev that prevents pci_dev.ats_cap from ever gettting set.

Quoted 8 lines

This change ensures that pci_ats_supported() takes quirks into account during iommu_ops.probe_device(), when IOMMU drivers are first notified about devices. It also ensures that pci_ats_supported() returns the same value when the device is released in iommu_ops.release_device().

Notably, the Intel IOMMU driver uses pci_ats_supported() in probe/release to determine whether to add/remove a device from a data structure, which easily leads to a use-after-free without this fix.

Can you please shed more light on the above issue? In my investigation, iommu_ops.probe_device() is always called after the no_ats quirk, regardless of whether this patch is applied.

The diff of the changes I made for testing is as follows:

drivers/iommu/intel/iommu.c +2 -0
index 442271a1b92a..c024964ac53b 100644
@@ -3271,6 +3271,8 @@ static struct iommu_device
 *intel_iommu_probe_device(struct device *dev)
                                 info->pfsid = pci_dev_id(pci_physfn(pdev));
                         info->ats_qdep = pci_ats_queue_depth(pdev);
                 }
+               pci_info(pdev, "ATS %s\n", info->ats_supported ?
+                        "supported" : "not supported");
                 if (sm_supported(iommu)) {
                         if (pasid_supported(iommu)) {
                                 int features = pci_pasid_features(pdev);
drivers/pci/quirks.c +0 -0
index 48946cca4be7..c63616d108b7 100644
@@ -5714,6 +5714,8 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL,
 0x1457, quirk_intel_e2000_no_ats);
  DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x1459,
 quirk_intel_e2000_no_ats);
  DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145a,
 quirk_intel_e2000_no_ats);
  DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x145c,

quirk_intel_e2000_no_ats);

drivers/pci/quirks.c +2 -0
+
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, 0x0b25, quirk_no_ats);
  #endif /* CONFIG_PCI_ATS */
 
  /* Freescale PCIe doesn't support MSI in RC mode */

The related kernel messages are shown below:

# dmesg | grep "0000:00:01.0"
[   15.834944] pci 0000:00:01.0: [8086:0b25] type 00 class 0x088000 PCIe
Root Complex Integrated Endpoint
[   15.836382] pci 0000:00:01.0: BAR 0 [mem
0x1e0fff980000-0x1e0fff99ffff 64bit pref]
[   15.836655] pci 0000:00:01.0: BAR 2 [mem
0x1e0fff900000-0x1e0fff93ffff 64bit pref]
[   15.837904] pci 0000:00:01.0: calling
quirk_igfx_skip_te_disable+0x0/0xe0 @ 1
[   15.838614] pci 0000:00:01.0: quirk_igfx_skip_te_disable+0x0/0xe0
took 0 usecs
[   21.205177] pci 0000:00:01.0: calling  quirk_no_ats+0x0/0x40 @ 1
[   21.206767] pci 0000:00:01.0: disabling ATS
[   21.207916] pci 0000:00:01.0: quirk_no_ats+0x0/0x40 took 1122 usecs
[   21.305357] pci 0000:00:01.0: DMAR: ATS not supported
[   21.306925] pci 0000:00:01.0: Adding to iommu group 4
[   42.564912] idxd 0000:00:01.0: Intel(R) Accelerator Device (v200)
[   42.568653] probe of 0000:00:01.0 returned 0 after 87413 usecs
Quoted 1 line

Anything I missed?

The Closes link above has the details of my investigation:

Quoted 5 lines

https://lore.kernel.org/linux-iommu/[email protected]/

Can you check if you see the same ordering for VFs? That was the scenario where I saw the IOMMU drivers notified before final fixups were applied.

During boot I observe the same order as you. PCI final fixups get applied twice actually, and both times is before iommu probe_device().

First time:

[    1.687987][    T1] pci 0000:00:04.0: pci_fixup_device
[    1.690987][    T1] WARNING: drivers/pci/quirks.c:227 at pci_fixup_device+0xd5/0x230, CPU#5: swapper/0/1
[    1.708986][    T1] RIP: 0010:pci_fixup_device+0xf9/0x230
[    1.755989][    T1]  pci_bus_add_device+0x31/0x90
[    1.758986][    T1]  pci_bus_add_devices+0x39/0x70
[    1.760986][    T1]  acpi_pci_root_add+0x815/0xa60
[    1.767986][    T1]  acpi_bus_attach+0x260/0x2f0
[    1.773988][    T1]  device_for_each_child+0x6a/0xa0
[    1.775986][    T1]  acpi_dev_for_each_child+0x39/0x60
[    1.780986][    T1]  acpi_bus_attach+0xe8/0x2f0
[    1.785987][    T1]  device_for_each_child+0x6a/0xa0
[    1.787989][    T1]  acpi_dev_for_each_child+0x39/0x60
[    1.792985][    T1]  acpi_bus_attach+0xe8/0x2f0
[    1.795986][    T1]  acpi_bus_scan+0x6a/0x200
[    1.799986][    T1]  acpi_scan_init+0xb7/0x1c0
[    1.803988][    T1]  acpi_init+0xad/0xd0
[    1.808986][    T1]  do_one_initcall+0xba/0x2a0
[    1.825986][    T1]  do_initcall_level+0x7c/0xf0
[    1.827988][    T1]  do_initcalls+0x43/0x70
[    1.829986][    T1]  kernel_init_freeable+0xd7/0x140
[    1.834985][    T1]  kernel_init+0x1a/0x130
[    1.836986][    T1]  ret_from_fork+0xf4/0x280
[    1.840985][    T1]  ret_from_fork_asm+0x1a/0x30

Second time:

[    4.156625][    T1] pci 0000:00:04.0: pci_fixup_device
[    4.159282][    T1] WARNING: drivers/pci/quirks.c:227 at pci_fixup_device+0xd5/0x230, CPU#1: swapper/0/1
[    4.178115][    T1] RIP: 0010:pci_fixup_device+0xf9/0x230
[    4.228013][    T1]  pci_apply_final_quirks+0x6a/0x170
[    4.233485][    T1]  do_one_initcall+0xba/0x2a0
[    4.251171][    T1]  do_initcall_level+0x7c/0xf0
[    4.253521][    T1]  do_initcalls+0x43/0x70
[    4.255653][    T1]  kernel_init_freeable+0xd7/0x140
[    4.260602][    T1]  kernel_init+0x1a/0x130
[    4.262745][    T1]  ret_from_fork+0xf4/0x280
[    4.267345][    T1]  ret_from_fork_asm+0x1a/0x30

Then iommu probe_device() happens:

[    5.744756][    T1] pci 0000:00:04.0: intel_iommu_probe_device
[    5.747688][    T1] WARNING: drivers/iommu/intel/iommu.c:3257 at intel_iommu_probe_device+0x3e9/0x840, CPU#1: swapper/0/1
[    5.767127][    T1] RIP: 0010:intel_iommu_probe_device+0x403/0x840
[    5.816998][    T1]  __iommu_probe_device+0xe7/0x4d0
[    5.819522][    T1]  probe_iommu_group+0x29/0x50
[    5.824593][    T1]  bus_for_each_dev+0x111/0x130
[    5.826965][    T1]  iommu_device_register+0xfb/0x250
[    5.829492][    T1]  intel_iommu_init+0x36f/0x450
[    5.834374][    T1]  pci_iommu_init+0x16/0x40
[    5.836614][    T1]  do_one_initcall+0xba/0x2a0
[    5.854121][    T1]  do_initcall_level+0x7c/0xf0
[    5.856450][    T1]  do_initcalls+0x43/0x70
[    5.858563][    T1]  kernel_init_freeable+0xd7/0x140
[    5.863427][    T1]  kernel_init+0x1a/0x130
[    5.865522][    T1]  ret_from_fork+0xf4/0x280
[    5.870169][    T1]  ret_from_fork_asm+0x1a/0x30

But when VFs are created the order is reversed:

[  545.718507][  T242] pci 0000:01:00.1: intel_iommu_probe_device
[  545.718627][  T242] WARNING: drivers/iommu/intel/iommu.c:3257 at intel_iommu_probe_device+0x3e9/0x840, CPU#0: bash/242
[  545.719468][  T242] RIP: 0010:intel_iommu_probe_device+0x403/0x840
[  545.721181][  T242]  __iommu_probe_device+0xe7/0x4d0
[  545.721278][  T242]  iommu_bus_notifier+0x37/0x100
[  545.721373][  T242]  blocking_notifier_call_chain+0x62/0x100
[  545.721485][  T242]  bus_notify+0x97/0xc0
[  545.721567][  T242]  device_add+0x222/0x400
[  545.721646][  T242]  pci_device_add+0x488/0x6e0
[  545.721746][  T242]  pci_iov_add_virtfn+0x2f2/0x3e0
[  545.721935][  T242]  sriov_add_vfs+0x2c/0x60
[  545.722064][  T242]  sriov_enable+0x306/0x4a0
[  545.722157][  T242]  virtio_pci_sriov_configure+0x6a/0x90
[  545.722257][  T242]  sriov_numvfs_store+0xd7/0x1c0
[  545.722349][  T242]  kernfs_fop_write_iter+0xfe/0x180
[  545.722453][  T242]  vfs_write+0x2e4/0x460
[  545.722533][  T242]  ksys_write+0x70/0x100
[  545.722613][  T242]  do_syscall_64+0xd1/0x8b0
[  545.722832][  T242]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[  545.725241][  T242] pci 0000:01:00.1: pci_fixup_device
[  545.725339][  T242] WARNING: drivers/pci/quirks.c:227 at pci_fixup_device+0xd5/0x230, CPU#0: bash/242
[  545.727806][  T242]  pci_bus_add_device+0x31/0x90
[  545.727902][  T242]  pci_iov_add_virtfn+0x3c9/0x3e0
[  545.728118][  T242]  sriov_add_vfs+0x2c/0x60
[  545.728226][  T242]  sriov_enable+0x306/0x4a0
[  545.728320][  T242]  virtio_pci_sriov_configure+0x6a/0x90
[  545.728416][  T242]  sriov_numvfs_store+0xd7/0x1c0
[  545.728512][  T242]  kernfs_fop_write_iter+0xfe/0x180
[  545.728609][  T242]  vfs_write+0x2e4/0x460
[  545.728690][  T242]  ksys_write+0x70/0x100
[  545.728771][  T242]  do_syscall_64+0xd1/0x8b0
[  545.728866][  T242]  ? clear_bhb_loop+0x30/0x80
[  545.728961][  T242]  entry_SYSCALL_64_after_hwframe+0x76/0x7e

My naive reading of the PCI hotplug code makes me think it has the same ordering as VF creation. More generally, any PCI device that is added after the IOMMU bus notifiers are set up will probe the device before final fixups are applied. I can clarify that in the commit message in v3.

Here is the patch I used to collect the traces above:

drivers/iommu/intel/iommu.c +1 -0
index ef7613b177b9..122cb8f610a3 100644
@@ -3254,6 +3254,7 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
        info->dev = dev;
        info->iommu = iommu;
        if (dev_is_pci(dev)) {
+               pci_WARN(to_pci_dev(dev), 1, "intel_iommu_probe_device\n");
                if (ecap_dev_iotlb_support(iommu->ecap) &&
                    pci_ats_supported(pdev) &&
                    dmar_ats_supported(pdev, iommu)) {
drivers/pci/quirks.c +1 -0
index 48946cca4be7..af9cb39c9236 100644
@@ -224,6 +224,7 @@ void pci_fixup_device(enum pci_fixup_pass pass, struct pci_dev *dev)
                break;
 
        case pci_fixup_final:
+               pci_WARN(dev, 1, "pci_fixup_device\n");
                if (!pci_apply_fixup_final_quirks)
                        return;
                start = __start_pci_fixups_final;