Skip to contentSkip to conversation

iommu/amd: Enhance def_domain_type to handle untrusted device

PATCH In review Vasant Hegde 5 participants 5 messages Apr 23, 2024 – Apr 26, 2024

Subsystems (2)
lore
Share & tools
lore

Vasant Hegde

author maintainer
#0 lore raw

PATCH v2 1/1 iommu/amd: Enhance def_domain_type to handle untrusted device

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined state.

This patch adds untrusted check in AMD IOMMU driver code. So that it allows eGPUs behind Thunderbolt work again.

Fine tuning amd_iommu_def_domain_type() will be done later.

Fixes 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()")
Cc Robin Murphy <[email protected]>
Cc Jason Gunthorpe <[email protected]>

---
Changes in v2:
- Added dev_is_pci() check - Thanks Lu Baolu

-Vasant

 drivers/iommu/amd/iommu.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/iommu/amd/iommu.c +4 -0
index e692217fcb28..fb727f5b0b82 100644
@@ -2754,6 +2754,10 @@ static int amd_iommu_def_domain_type(struct device *dev)
 	if (!dev_data)
 		return 0;
 
+	/* Always use DMA domain for untrusted device */
+	if (dev_is_pci(dev) && to_pci_dev(dev)->untrusted)
+		return IOMMU_DOMAIN_DMA;
+
 	/*
 	 * Do not identity map IOMMUv2 capable devices when:
 	 *  - memory encryption is active, because some of those devices
Signature · 1 line
2.31.1
reviewer
#1 lore raw
Quoted 11 lines · Vasant Hegde

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined state.

This patch adds untrusted check in AMD IOMMU driver code. So that it allows eGPUs behind Thunderbolt work again.

Sorry, but I firmly disagree with this approach - regardless of what the AMD driver is currently doing and how appropriate it may or may not be, we should not be duplicating core code policy in drivers, and we certainly shouldn't be doing so to bodge around a breakage in how the core code implements said policy.

Thanks,
Robin.

Quoted 34 lines

Fine tuning amd_iommu_def_domain_type() will be done later.

Fixes 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()")
Cc Robin Murphy <[email protected]>
Cc Jason Gunthorpe <[email protected]>

---
Changes in v2:
- Added dev_is_pci() check - Thanks Lu Baolu

(quoted diff) +0 -1
-Vasant
 
  drivers/iommu/amd/iommu.c | 4 ++++
  1 file changed, 4 insertions(+)
drivers/iommu/amd/iommu.c +4 -0
index e692217fcb28..fb727f5b0b82 100644
@@ -2754,6 +2754,10 @@ static int amd_iommu_def_domain_type(struct device *dev)
  	if (!dev_data)
  		return 0;
 
+	/* Always use DMA domain for untrusted device */
+	if (dev_is_pci(dev) && to_pci_dev(dev)->untrusted)
+		return IOMMU_DOMAIN_DMA;
+
  	/*
  	 * Do not identity map IOMMUv2 capable devices when:
  	 *  - memory encryption is active, because some of those devices
#2 lore raw

Hi Robin,

Quoted 10 lines · Robin Murphy
Quoted 9 lines · Vasant Hegde

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined state.

Quoted 7 lines
Quoted 2 lines

This patch adds untrusted check in AMD IOMMU driver code. So that it allows eGPUs behind Thunderbolt work again.

Sorry, but I firmly disagree with this approach - regardless of what the AMD driver is currently doing and how appropriate it may or may not be, we should not be duplicating core code policy in drivers, and we certainly shouldn't be doing so to bodge around a breakage in how the core code implements said policy.

Ack. I still think fixing in core layer (what your patch does) is the right fix.

@Joerg,
This is alternative to Robin's original patch [1]. If everyone agrees with
that fix then we don't need this patch.

[1]
https://lore.kernel.org/linux-iommu/fa14583e94cbf540b60a6be94b41bb24d0037e75.1713272443.git.robin.murphy@arm.com/

-Vasant

Quoted 39 lines

Thanks,
Robin.

Quoted 35 lines

Fine tuning amd_iommu_def_domain_type() will be done later.

Link:
https://lore.kernel.org/linux-iommu/CAHudX3zLH6CsRmLE-yb+gRjhh-v4bU5_1jW_xCcxOo_oUUZKYg@mail.gmail.com

Fixes 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to

respect def_domain_type()")

Cc Robin Murphy <[email protected]>
Cc Jason Gunthorpe <[email protected]>

---
Changes in v2:
   - Added dev_is_pci() check - Thanks Lu Baolu

-Vasant

  drivers/iommu/amd/iommu.c | 4 ++++

  1 file changed, 4 insertions(+)
drivers/iommu/amd/iommu.c +0 -0
index e692217fcb28..fb727f5b0b82 100644
@@ -2754,6 +2754,10 @@ static int amd_iommu_def_domain_type(struct device *dev)
       if (!dev_data)
           return 0;

  +    /* Always use DMA domain for untrusted device */
+    if (dev_is_pci(dev) && to_pci_dev(dev)->untrusted)
+        return IOMMU_DOMAIN_DMA;
+
      /*
       * Do not identity map IOMMUv2 capable devices when:
       *  - memory encryption is active, because some of those devices

#3 lore raw
Quoted 20 lines · Vasant Hegde

Hi Robin,

Quoted 16 lines · Robin Murphy
Quoted 11 lines · Vasant Hegde

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined state.

This patch adds untrusted check in AMD IOMMU driver code. So that it allows eGPUs behind Thunderbolt work again.

Sorry, but I firmly disagree with this approach - regardless of what the AMD driver is currently doing and how appropriate it may or may not be, we should not be duplicating core code policy in drivers,

Then AMD should remove the PASID stuff from def_domain_type and we need to fix it with properly designed core support for this limitation before merging the SVA support.

Quoted 2 lines
Quoted 2 lines

and we certainly shouldn't be
doing so to bodge around a breakage in how the core code implements said policy.

It isn't breakage. It is evolution of the API.

Hacking in the AMD driver is temprary and Vasant will eventually fix it. Regressing the core code to go back to allowing drivers to inform policy is much harder to undo.

Jason

maintainer
#4 lore raw
In reply to Vasant Hegde
Quoted 2 lines · Vasant Hegde
 drivers/iommu/amd/iommu.c | 4 ++++
 1 file changed, 4 insertions(+)

Since this fixes a regression I think it makes sense to apply as-is for v6.9. Things can be done more properly later on-top of this fix.

So: Applied, thanks.