Disable ATS via iommu during PCI resets

The iommu_deferred_attach() function invokes __iommu_attach_device(), but doesn't hold the group->mutex like other __iommu_attach_device() callers.

Though there is no pratical bug being triggered so far, it would be better to apply the same locking to this __iommu_attach_device(), since the IOMMU drivers nowaday are more aware of the group->mutex -- some of them use the iommu_group_mutex_assert() function that could be potentially in the path of an attach_dev callback function invoked by the __iommu_attach_device().

Worth mentioning that the iommu_deferred_attach() will soon need to check group->resetting_domain that must be locked also.

Thus, grab the mutex to guard __iommu_attach_device() like other callers.

Reviewed-by Jason Gunthorpe <[email protected]>

Reviewed-by Kevin Tian <[email protected]>

Reviewed-by Lu Baolu <[email protected]>

Signed-off-by Nicolin Chen <[email protected]>

---
  drivers/iommu/iommu.c | 13 ++++++++++---
  1 file changed, 10 insertions(+), 3 deletions(-)

drivers/iommu/iommu.c +10 -3

index 2ca990dfbb884..170e522b5bda4 100644
@@ -2185,10 +2185,17 @@ EXPORT_SYMBOL_GPL(iommu_attach_device);
 
  int iommu_deferred_attach(struct device *dev, struct iommu_domain *domain)
  {
-	if (dev->iommu && dev->iommu->attach_deferred)
-		return __iommu_attach_device(domain, dev, NULL);
+	/*
+	 * This is called on the dma mapping fast path so avoid locking. This is
+	 * racy, but we have an expectation that the driver will setup its DMAs
+	 * inside probe while being single threaded to avoid racing.
+	 */
+	if (!dev->iommu || !dev->iommu->attach_deferred)
+		return 0;
 
-	return 0;
+	guard(mutex)(&dev->iommu_group->mutex);
+
+	return __iommu_attach_device(domain, dev, NULL);
  }
 
  void iommu_detach_device(struct iommu_domain *domain, struct device *dev)

Quoted 12 lines · Nicolin Chen

The iommu_deferred_attach() function invokes __iommu_attach_device(), but doesn't hold the group->mutex like other __iommu_attach_device() callers.

Though there is no pratical bug being triggered so far, it would be better to apply the same locking to this __iommu_attach_device(), since the IOMMU drivers nowaday are more aware of the group->mutex -- some of them use the iommu_group_mutex_assert() function that could be potentially in the path of an attach_dev callback function invoked by the __iommu_attach_device().

Worth mentioning that the iommu_deferred_attach() will soon need to check group->resetting_domain that must be locked also.

Quoted 2 lines

Thus, grab the mutex to guard __iommu_attach_device() like other callers.

Tested the series with PCI reset on PFs and VFs, including device pass-through to a Linux guest. All scenarios worked as expected.

Tested-by Dheeraj Kumar Srivastava <[email protected]>

Quoted 19 lines · Dheeraj Kumar

Quoted 9 lines · Nicolin Chen

The iommu_deferred_attach() function invokes __iommu_attach_device(), but doesn't hold the group->mutex like other __iommu_attach_device() callers.

Though there is no pratical bug being triggered so far, it would be better to apply the same locking to this __iommu_attach_device(), since the IOMMU drivers nowaday are more aware of the group->mutex -- some of them use the iommu_group_mutex_assert() function that could be potentially in the path of an attach_dev callback function invoked by the __iommu_attach_device().

Quoted 5 lines

Worth mentioning that the iommu_deferred_attach() will soon need to check group->resetting_domain that must be locked also.

Thus, grab the mutex to guard __iommu_attach_device() like other callers.

Tested the series with PCI reset on PFs and VFs, including device pass-through to a Linux guest. All scenarios worked as expected.

Quoted 1 line

Tested-by Dheeraj Kumar Srivastava <[email protected]>

Thanks for testing!

Yet, this is replying to PATCH-1. So, you might want to reply with your "Tested-by" tag to PATCH-0 :)

Otherwise, default B4 command might miss your tag in other patches:

✗ [PATCH v7 1/5] iommu: Lock group->mutex in iommu_deferred_attach()
+ Tested-by: Dheeraj Kumar Srivastava <[email protected]> (✓ DKIM/amd.com)
✗ [PATCH v7 2/5] iommu: Tidy domain for iommu_setup_dma_ops()
+ Reviewed-by: Jason Gunthorpe <[email protected]> (✗ DKIM/Nvidia.com)
✗ [PATCH v7 3/5] iommu: Add iommu_driver_get_domain_for_dev() helper
+ Reviewed-by: Jason Gunthorpe <[email protected]> (✗ DKIM/Nvidia.com)
✗ [PATCH v7 4/5] iommu: Introduce pci_dev_reset_iommu_prepare/done()
+ Reviewed-by: Jason Gunthorpe <[email protected]> (✗ DKIM/Nvidia.com)
✗ [PATCH v7 5/5] PCI: Suspend iommu function prior to resetting a device
+ Reviewed-by: Jason Gunthorpe <[email protected]> (✗ DKIM/Nvidia.com)

Thank you
Nicolin

This function can only be called on the default_domain. Trivally pass it in. In all three existing cases, the default domain was just attached to the device.

This avoids iommu_setup_dma_ops() calling iommu_get_domain_for_dev() that will be used by external callers.

Suggested-by Jason Gunthorpe <[email protected]>

Reviewed-by Kevin Tian <[email protected]>

Reviewed-by Lu Baolu <[email protected]>

Signed-off-by Nicolin Chen <[email protected]>

---
 drivers/iommu/dma-iommu.h | 5 +++--
 drivers/iommu/dma-iommu.c | 4 +---
 drivers/iommu/iommu.c     | 6 +++---
 3 files changed, 7 insertions(+), 8 deletions(-)

There is a need to stage a resetting PCI device to temporarily the blocked domain and then attach back to its previously attached domain after reset.

This can be simply done by keeping the "previously attached domain" in the iommu_group->domain pointer while adding an iommu_group->resetting_domain, which gives troubles to IOMMU drivers using the iommu_get_domain_for_dev() for a device's physical domain in order to program IOMMU hardware.

And in such for-driver use cases, the iommu_group->mutex must be held, so it doesn't fit in external callers that don't hold the iommu_group->mutex.

Introduce a new iommu_driver_get_domain_for_dev() helper, exclusively for driver use cases that hold the iommu_group->mutex, to separate from those external use cases.

Add a lockdep_assert_not_held to the existing iommu_get_domain_for_dev() and highlight that in a kdoc.

Reviewed-by Kevin Tian <[email protected]>

Reviewed-by Lu Baolu <[email protected]>

Signed-off-by Nicolin Chen <[email protected]>

---
 include/linux/iommu.h                       |  1 +
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |  5 ++--
 drivers/iommu/iommu.c                       | 28 +++++++++++++++++++++
 3 files changed, 32 insertions(+), 2 deletions(-)

PCIe permits a device to ignore ATS invalidation TLPs while processing a reset. This creates a problem visible to the OS where an ATS invalidation command will time out. E.g. an SVA domain will have no coordination with a reset event and can racily issue ATS invalidations to a resetting device.

The OS should do something to mitigate this as we do not want production systems to be reporting critical ATS failures, especially in a hypervisor environment. Broadly, OS could arrange to ignore the timeouts, block page table mutations to prevent invalidations, or disable and block ATS.

The PCIe r6.0, sec 10.3.1 IMPLEMENTATION NOTE recommends SW to disable and block ATS before initiating a Function Level Reset. It also mentions that other reset methods could have the same vulnerability as well.

Provide a callback from the PCI subsystem that will enclose the reset and have the iommu core temporarily change all the attached RID/PASID domains group->blocking_domain so that the IOMMU hardware would fence any incoming ATS queries. And IOMMU drivers should also synchronously stop issuing new ATS invalidations and wait for all ATS invalidations to complete. This can avoid any ATS invaliation timeouts.

However, if there is a domain attachment/replacement happening during an ongoing reset, ATS routines may be re-activated between the two function calls. So, introduce a new resetting_domain in the iommu_group structure to reject any concurrent attach_dev/set_dev_pasid call during a reset for a concern of compatibility failure. Since this changes the behavior of an attach operation, update the uAPI accordingly.

Note that there are two corner cases:
1. Devices in the same iommu_group
Since an attachment is always per iommu_group, this means that any sibling devices in the iommu_group cannot change domain, to prevent race conditions.
2. An SR-IOV PF that is being reset while its VF is not
In such case, the VF itself is already broken. So, there is no point in preventing PF from going through the iommu reset.

Reviewed-by Lu Baolu <[email protected]>

Reviewed-by Kevin Tian <[email protected]>

Signed-off-by Nicolin Chen <[email protected]>

---
 include/linux/iommu.h     |  13 +++
 include/uapi/linux/vfio.h |   4 +
 drivers/iommu/iommu.c     | 173 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 190 insertions(+)

PCIe permits a device to ignore ATS invalidation TLPs while processing a reset. This creates a problem visible to the OS where an ATS invalidation command will time out: e.g. an SVA domain will have no coordination with a reset event and can racily issue ATS invalidations to a resetting device.

The PCIe r6.0, sec 10.3.1 IMPLEMENTATION NOTE recommends SW to disable and block ATS before initiating a Function Level Reset. It also mentions that other reset methods could have the same vulnerability as well.

The IOMMU subsystem provides pci_dev_reset_iommu_prepare/done() callback helpers for this matter. Use them in all the existing reset functions.

This will attach the device to its iommu_group->blocking_domain during the device reset, so as to allow IOMMU driver to:
- invoke pci_disable_ats() and pci_enable_ats(), if necessary
- wait for all ATS invalidations to complete
- stop issuing new ATS invalidations
- fence any incoming ATS queries

Reviewed-by Kevin Tian <[email protected]>

Signed-off-by Nicolin Chen <[email protected]>

---
 drivers/pci/pci-acpi.c | 13 +++++++--
 drivers/pci/pci.c      | 65 +++++++++++++++++++++++++++++++++++++-----
 drivers/pci/quirks.c   | 19 +++++++++++-
 3 files changed, 87 insertions(+), 10 deletions(-)

PCIe permits a device to ignore ATS invalidation TLPs while processing a reset. This creates a problem visible to the OS where an ATS invalidation command will time out: e.g. an SVA domain will have no coordination with a reset event and can racily issue ATS invalidations to a resetting device.

The PCIe r6.0, sec 10.3.1 IMPLEMENTATION NOTE recommends SW to disable and block ATS before initiating a Function Level Reset. It also mentions that other reset methods could have the same vulnerability as well.

The IOMMU subsystem provides pci_dev_reset_iommu_prepare/done() callback helpers for this matter. Use them in all the existing reset functions.

This will attach the device to its iommu_group->blocking_domain during the device reset, so as to allow IOMMU driver to:
- invoke pci_disable_ats() and pci_enable_ats(), if necessary
- wait for all ATS invalidations to complete
- stop issuing new ATS invalidations
- fence any incoming ATS queries

Reviewed-by Kevin Tian <[email protected]>

Signed-off-by Nicolin Chen <[email protected]>

---
 drivers/pci/pci-acpi.c | 13 +++++++--
 drivers/pci/pci.c      | 65 +++++++++++++++++++++++++++++++++++++-----
 drivers/pci/quirks.c   | 19 +++++++++++-
 3 files changed, 87 insertions(+), 10 deletions(-)

drivers/pci/pci-acpi.c +11 -2

index 9369377725fa0..651d9b5561fff 100644
@@ -9,6 +9,7 @@
 
 #include <linux/delay.h>
 #include <linux/init.h>
+#include <linux/iommu.h>
 #include <linux/irqdomain.h>
 #include <linux/pci.h>
 #include <linux/msi.h>

@@ -971,6 +972,7 @@ void pci_set_acpi_fwnode(struct pci_dev *dev)
 int pci_dev_acpi_reset(struct pci_dev *dev, bool probe)
 {
 	acpi_handle handle = ACPI_HANDLE(&dev->dev);
+	int ret;
 
 	if (!handle || !acpi_has_method(handle, "_RST"))
 		return -ENOTTY;

@@ -978,12 +980,19 @@ int pci_dev_acpi_reset(struct pci_dev *dev, bool probe)
 	if (probe)
 		return 0;
 
+	ret = pci_dev_reset_iommu_prepare(dev);
+	if (ret) {
+		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
+		return ret;
+	}
+
 	if (ACPI_FAILURE(acpi_evaluate_object(handle, "_RST", NULL, NULL))) {
 		pci_warn(dev, "ACPI _RST failed\n");
-		return -ENOTTY;
+		ret = -ENOTTY;
 	}
 
-	return 0;
+	pci_dev_reset_iommu_done(dev);
+	return ret;
 }
 
 bool acpi_pci_power_manageable(struct pci_dev *dev)

drivers/pci/pci.c +58 -7

index b14dd064006cc..da0cf0f041516 100644
@@ -13,6 +13,7 @@
 #include <linux/delay.h>
 #include <linux/dmi.h>
 #include <linux/init.h>
+#include <linux/iommu.h>
 #include <linux/msi.h>
 #include <linux/of.h>
 #include <linux/pci.h>

@@ -25,6 +26,7 @@
 #include <linux/logic_pio.h>
 #include <linux/device.h>
 #include <linux/pm_runtime.h>
+#include <linux/pci-ats.h>
 #include <linux/pci_hotplug.h>
 #include <linux/vmalloc.h>
 #include <asm/dma.h>

@@ -4478,13 +4480,22 @@ EXPORT_SYMBOL(pci_wait_for_pending_transaction);
  */
 int pcie_flr(struct pci_dev *dev)
 {
+	int ret;
+
 	if (!pci_wait_for_pending_transaction(dev))
 		pci_err(dev, "timed out waiting for pending transaction; performing function level reset anyway\n");
 
+	/* Have to call it after waiting for pending DMA transaction */
+	ret = pci_dev_reset_iommu_prepare(dev);
+	if (ret) {
+		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
+		return ret;
+	}
+
 	pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_BCR_FLR);
 
 	if (dev->imm_ready)
-		return 0;
+		goto done;
 
 	/*
 	 * Per PCIe r4.0, sec 6.6.2, a device must complete an FLR within

@@ -4493,7 +4504,10 @@ int pcie_flr(struct pci_dev *dev)
 	 */
 	msleep(100);
 
-	return pci_dev_wait(dev, "FLR", PCIE_RESET_READY_POLL_MS);
+	ret = pci_dev_wait(dev, "FLR", PCIE_RESET_READY_POLL_MS);
+done:
+	pci_dev_reset_iommu_done(dev);
+	return ret;
 }
 EXPORT_SYMBOL_GPL(pcie_flr);
 

@@ -4521,6 +4535,7 @@ EXPORT_SYMBOL_GPL(pcie_reset_flr);
 
 static int pci_af_flr(struct pci_dev *dev, bool probe)
 {
+	int ret;
 	int pos;
 	u8 cap;
 

@@ -4547,10 +4562,17 @@ static int pci_af_flr(struct pci_dev *dev, bool probe)
 				 PCI_AF_STATUS_TP << 8))
 		pci_err(dev, "timed out waiting for pending transaction; performing AF function level reset anyway\n");
 
+	/* Have to call it after waiting for pending DMA transaction */
+	ret = pci_dev_reset_iommu_prepare(dev);
+	if (ret) {
+		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
+		return ret;
+	}
+
 	pci_write_config_byte(dev, pos + PCI_AF_CTRL, PCI_AF_CTRL_FLR);
 
 	if (dev->imm_ready)
-		return 0;
+		goto done;
 
 	/*
 	 * Per Advanced Capabilities for Conventional PCI ECN, 13 April 2006,

@@ -4560,7 +4582,10 @@ static int pci_af_flr(struct pci_dev *dev, bool probe)
 	 */
 	msleep(100);
 
-	return pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS);
+	ret = pci_dev_wait(dev, "AF_FLR", PCIE_RESET_READY_POLL_MS);
+done:
+	pci_dev_reset_iommu_done(dev);
+	return ret;
 }
 
 /**

@@ -4581,6 +4606,7 @@ static int pci_af_flr(struct pci_dev *dev, bool probe)
 static int pci_pm_reset(struct pci_dev *dev, bool probe)
 {
 	u16 csr;
+	int ret;
 
 	if (!dev->pm_cap || dev->dev_flags & PCI_DEV_FLAGS_NO_PM_RESET)
 		return -ENOTTY;

@@ -4595,6 +4621,12 @@ static int pci_pm_reset(struct pci_dev *dev, bool probe)
 	if (dev->current_state != PCI_D0)
 		return -EINVAL;
 
+	ret = pci_dev_reset_iommu_prepare(dev);
+	if (ret) {
+		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
+		return ret;
+	}
+
 	csr &= ~PCI_PM_CTRL_STATE_MASK;
 	csr |= PCI_D3hot;
 	pci_write_config_word(dev, dev->pm_cap + PCI_PM_CTRL, csr);

@@ -4605,7 +4637,9 @@ static int pci_pm_reset(struct pci_dev *dev, bool probe)
 	pci_write_config_word(dev, dev->pm_cap + PCI_PM_CTRL, csr);
 	pci_dev_d3_sleep(dev);
 
-	return pci_dev_wait(dev, "PM D3hot->D0", PCIE_RESET_READY_POLL_MS);
+	ret = pci_dev_wait(dev, "PM D3hot->D0", PCIE_RESET_READY_POLL_MS);
+	pci_dev_reset_iommu_done(dev);
+	return ret;
 }
 
 /**

@@ -5033,10 +5067,20 @@ static int pci_reset_bus_function(struct pci_dev *dev, bool probe)
 		return -ENOTTY;
 	}
 
+	rc = pci_dev_reset_iommu_prepare(dev);
+	if (rc) {
+		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", rc);
+		return rc;
+	}
+
 	rc = pci_dev_reset_slot_function(dev, probe);
 	if (rc != -ENOTTY)
-		return rc;
-	return pci_parent_bus_reset(dev, probe);
+		goto done;
+
+	rc = pci_parent_bus_reset(dev, probe);
+done:
+	pci_dev_reset_iommu_done(dev);
+	return rc;
 }
 
 static int cxl_reset_bus_function(struct pci_dev *dev, bool probe)

@@ -5060,6 +5104,12 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe)
 	if (rc)
 		return -ENOTTY;
 
+	rc = pci_dev_reset_iommu_prepare(dev);
+	if (rc) {
+		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", rc);
+		return rc;
+	}
+
 	if (reg & PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR) {
 		val = reg;
 	} else {

@@ -5074,6 +5124,7 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe)
 		pci_write_config_word(bridge, dvsec + PCI_DVSEC_CXL_PORT_CTL,
 				      reg);
 
+	pci_dev_reset_iommu_done(dev);
 	return rc;
 }
 

drivers/pci/quirks.c +18 -1

index b9c252aa6fe08..c6b999045c70a 100644
@@ -21,6 +21,7 @@
 #include <linux/pci.h>
 #include <linux/isa-dma.h> /* isa_dma_bridge_buggy */
 #include <linux/init.h>
+#include <linux/iommu.h>
 #include <linux/delay.h>
 #include <linux/acpi.h>
 #include <linux/dmi.h>

@@ -4228,6 +4229,22 @@ static const struct pci_dev_reset_methods pci_dev_reset_methods[] = {
 	{ 0 }
 };
 
+static int __pci_dev_specific_reset(struct pci_dev *dev, bool probe,
+				    const struct pci_dev_reset_methods *i)
+{
+	int ret;
+
+	ret = pci_dev_reset_iommu_prepare(dev);
+	if (ret) {
+		pci_err(dev, "failed to stop IOMMU for a PCI reset: %d\n", ret);
+		return ret;
+	}
+
+	ret = i->reset(dev, probe);
+	pci_dev_reset_iommu_done(dev);
+	return ret;
+}
+
 /*
  * These device-specific reset methods are here rather than in a driver
  * because when a host assigns a device to a guest VM, the host may need

@@ -4242,7 +4259,7 @@ int pci_dev_specific_reset(struct pci_dev *dev, bool probe)
 		     i->vendor == (u16)PCI_ANY_ID) &&
 		    (i->device == dev->device ||
 		     i->device == (u16)PCI_ANY_ID))
-			return i->reset(dev, probe);
+			return __pci_dev_specific_reset(dev, probe, i);
 	}
 
 	return -ENOTTY;

Signature · 1 line

2.43.0

Quoted 19 lines · Nicolin Chen

PCIe permits a device to ignore ATS invalidation TLPs while processing a reset. This creates a problem visible to the OS where an ATS invalidation command will time out: e.g. an SVA domain will have no coordination with a reset event and can racily issue ATS invalidations to a resetting device.

The PCIe r6.0, sec 10.3.1 IMPLEMENTATION NOTE recommends SW to disable and block ATS before initiating a Function Level Reset. It also mentions that other reset methods could have the same vulnerability as well.

The IOMMU subsystem provides pci_dev_reset_iommu_prepare/done() callback helpers for this matter. Use them in all the existing reset functions.

This will attach the device to its iommu_group->blocking_domain during the device reset, so as to allow IOMMU driver to:
- invoke pci_disable_ats() and pci_enable_ats(), if necessary
- wait for all ATS invalidations to complete
- stop issuing new ATS invalidations
- fence any incoming ATS queries

Quoted 2 lines

Reviewed-by Kevin Tian <[email protected]>

Signed-off-by Nicolin Chen <[email protected]>

Acked-by Bjorn Helgaas <[email protected]>

Nicolin Chen (5):
iommu: Lock group->mutex in iommu_deferred_attach()
iommu: Tidy domain for iommu_setup_dma_ops()
iommu: Add iommu_driver_get_domain_for_dev() helper
iommu: Introduce pci_dev_reset_iommu_prepare/done()
PCI: Suspend iommu function prior to resetting a device

 drivers/iommu/dma-iommu.h                   |   5 +-
 include/linux/iommu.h                       |  14 ++
 include/uapi/linux/vfio.h                   |   4 +
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |   5 +-
 drivers/iommu/dma-iommu.c                   |   4 +-
 drivers/iommu/iommu.c                       | 220 +++++++++++++++++++-
 drivers/pci/pci-acpi.c                      |  13 +-
 drivers/pci/pci.c                           |  65 +++++-
 drivers/pci/quirks.c                        |  19 +-
 9 files changed, 326 insertions(+), 23 deletions(-)

Quoted 7 lines · Nicolin Chen

Nicolin Chen (5):
iommu: Lock group->mutex in iommu_deferred_attach()
iommu: Tidy domain for iommu_setup_dma_ops()
iommu: Add iommu_driver_get_domain_for_dev() helper
iommu: Introduce pci_dev_reset_iommu_prepare/done()
PCI: Suspend iommu function prior to resetting a device

Quoted 10 lines

 drivers/iommu/dma-iommu.h                   |   5 +-
 include/linux/iommu.h                       |  14 ++
 include/uapi/linux/vfio.h                   |   4 +
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |   5 +-
 drivers/iommu/dma-iommu.c                   |   4 +-
 drivers/iommu/iommu.c                       | 220 +++++++++++++++++++-
 drivers/pci/pci-acpi.c                      |  13 +-
 drivers/pci/pci.c                           |  65 +++++-
 drivers/pci/quirks.c                        |  19 +-
 9 files changed, 326 insertions(+), 23 deletions(-)

Looks good from an IOMMU perspective, but needs Ack from PCI side.

Hi all,

PCIe permits a device to ignore ATS invalidation TLPs while processing a reset. This creates a problem visible to the OS where an ATS invalidation command will time out: e.g. an SVA domain will have no coordination with a reset event and can racily issue ATS invalidations to a resetting device.

The OS should do something to mitigate this as we do not want production systems to be reporting critical ATS failures, especially in a hypervisor environment. Broadly, OS could arrange to ignore the timeouts, block page table mutations to prevent invalidations, or disable and block ATS.

The PCIe spec in sec 10.3.1 IMPLEMENTATION NOTE recommends to disable and block ATS before initiating a Function Level Reset. It also mentions that other reset methods could have the same vulnerability as well.

Provide a callback from the PCI subsystem that will enclose the reset and have the iommu core temporarily change domains to group->blocking_domain, so IOMMU drivers would fence any incoming ATS queries, synchronously stop issuing new ATS invalidations, and wait for existing ATS invalidations to complete. Doing this can avoid any ATS invaliation timeouts.

When a device is resetting, any new domain attachment has to be rejected, until the reset is finished, to prevent ATS activity from being activated between the two callback functions. Introduce a new resetting_domain, and reject a concurrent __iommu_attach_device/set_group_pasid().

Finally, call these pci_dev_reset_iommu/done() functions in the PCI reset functions.

This is on Github:
https://github.com/nicolinc/iommufd/commits/iommu_dev_reset-v7

Changelog
v7
* Rebase on Joerg's next tree
* Add Reviewed-by from Kevin
* [iommu] Fix inline functions when !CONFIG_IOMMU_API
v6
https://lore.kernel.org/all/[email protected]/
* Add Reviewed-by from Baolu and Kevin
* Revise inline comments, kdocs, commit messages, uAPI
* [iommu] s/iommu_dev_reset/pci_dev_reset_iommu/g for PCI exclusively
* [iommu] Disallow iommu group sibling devices to attach concurrently
* [pci] Drop unnecessary initializations to "ret" and "rc"
* [pci] Improve pci_err message unpon a prepare() failure
* [pci] Move pci_ats_supported() check inside the IOMMU callbacks
* [pci] Apply callbacks to pci_reset_bus_function() that was missed
v5
https://lore.kernel.org/all/[email protected]/
* Rebase on Joerg's next tree
* [iommu] Skip in shared iommu_group cases
* [iommu] Pass in default_domain to iommu_setup_dma_ops
* [iommu] Add kdocs to iommu_get_domain_for_dev_locked()
* [iommu] s/get_domain_for_dev_locked/driver_get_domain_for_dev
* [iommu] Replace per-gdev pending_reset with per-group resetting_domain
v4
https://lore.kernel.org/all/[email protected]/
* Add Reviewed-by from Baolu
* [iommu] Use guard(mutex)
* [iommu] Update kdocs for typos and revisings
* [iommu] Skip two corner cases (alias and SRIOV)
* [iommu] Rework attach_dev to pass in old domain pointer
* [iommu] Reject concurrent attach_dev/set_dev_pasid for compatibility concern
* [smmuv3] Drop the old_domain depedency in its release_dev callback
* [pci] Add pci_reset_iommu_prepare/_done() wrappers checking ATS cap
v3
https://lore.kernel.org/all/[email protected]/
* Add Reviewed-by from Jason
* [iommu] Add a fast return in iommu_deferred_attach()
* [iommu] Update kdocs, inline comments, and commit logs
* [iommu] Use group->blocking_domain v.s. ops->blocked_domain
* [iommu] Drop require_direct, iommu_group_get(), and xa_lock()
* [iommu] Set the pending_reset flag after RID/PASID domain setups
* [iommu] Do not bypass PASID domains when RID domain is already the blocking_domain
* [iommu] Add iommu_get_domain_for_dev_locked to correctly return the blocking_domain
v2
https://lore.kernel.org/all/[email protected]/
* [iommu] Update kdocs, inline comments, and commit logs
* [iommu] Replace long-holding group->mutex with a pending_reset flag
* [pci] Abort reset routines if iommu_dev_reset_prepare() fails
* [pci] Apply the same vulnerability fix to other reset functions
v1
https://lore.kernel.org/all/[email protected]/

Thanks
Nicolin

Nicolin Chen (5):
iommu: Lock group->mutex in iommu_deferred_attach()
iommu: Tidy domain for iommu_setup_dma_ops()
iommu: Add iommu_driver_get_domain_for_dev() helper
iommu: Introduce pci_dev_reset_iommu_prepare/done()
PCI: Suspend iommu function prior to resetting a device

  drivers/iommu/dma-iommu.h                   |   5 +-
  include/linux/iommu.h                       |  14 ++
  include/uapi/linux/vfio.h                   |   4 +
  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |   5 +-
  drivers/iommu/dma-iommu.c                   |   4 +-
  drivers/iommu/iommu.c                       | 220 +++++++++++++++++++-
  drivers/pci/pci-acpi.c                      |  13 +-
  drivers/pci/pci.c                           |  65 +++++-
  drivers/pci/quirks.c                        |  19 +-
  9 files changed, 326 insertions(+), 23 deletions(-)