Skip to contentSkip to conversation

iommu/amd: Enhance def_domain_type to handle untrusted device

PATCH Superseded Vasant Hegde 6 participants 7 messages Apr 19, 2024 – Apr 23, 2024

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Vasant Hegde

author maintainer
#0 lore raw

PATCH 1/1 iommu/amd: Enhance def_domain_type to handle untrusted device

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined sstate.

This patch adds untrusted check in AMD IOMMU driver code. So that it lets eGPUs behind Thunderbolt work again.

Fine tuning amd_iommu_def_domain_type() will be done later.

Fixes 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()")
Cc Robin Murphy <[email protected]>
Cc Jason Gunthorpe <[email protected]>

---
@Eric,
Can you please test this patch?

-Vasant

 drivers/iommu/amd/iommu.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/iommu/amd/iommu.c +4 -0
index e692217fcb28..e405714640b0 100644
@@ -2754,6 +2754,10 @@ static int amd_iommu_def_domain_type(struct device *dev)
 	if (!dev_data)
 		return 0;
 
+	/* Always use DMA domain for untrusted device */
+	if (to_pci_dev(dev)->untrusted)
+		return IOMMU_DOMAIN_DMA;
+
 	/*
 	 * Do not identity map IOMMUv2 capable devices when:
 	 *  - memory encryption is active, because some of those devices
Signature · 1 line
2.31.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 9 lines · Vasant Hegde

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined sstate.

Quoted 4 lines

This patch adds untrusted check in AMD IOMMU driver code. So that it lets eGPUs behind Thunderbolt work again.

Fine tuning amd_iommu_def_domain_type() will be done later.

As long as this is done, and with the commitment that identity rid with pasid will be eventually supported as well

Jason

#2 lore raw
In reply to Vasant Hegde

[Public]

Quoted 27 lines

-----Original Message-----
From: Hegde, Vasant <[email protected]>
Sent: Friday, April 19, 2024 7:47 AM
To: [email protected]; [email protected]
Cc: Suthikulpanit, Suravee <[email protected]>; Deucher, Alexander <[email protected]>; Hegde, Vasant <[email protected]>; Eric Wagner <[email protected]>; Robin Murphy <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]
Subject: [PATCH] iommu/amd: Enhance def_domain_type to handle untrusted device

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined sstate.

This patch adds untrusted check in AMD IOMMU driver code. So that it lets eGPUs behind Thunderbolt work again.

Fine tuning amd_iommu_def_domain_type() will be done later.

Quoted 7 lines

[email protected]

Fixes 59ddce4418da ("iommu: Reorganize

iommu_get_default_domain_type() to respect def_domain_type()")

Cc Robin Murphy <[email protected]>
Cc Jason Gunthorpe <[email protected]>

Assuming it fixes the issue, please add:

Alex

Quoted 27 lines

---
@Eric,
Can you please test this patch?

(quoted diff) +0 -1
-Vasant
 
 drivers/iommu/amd/iommu.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/iommu/amd/iommu.c +4 -0
index e692217fcb28..e405714640b0 100644
@@ -2754,6 +2754,10 @@ static int amd_iommu_def_domain_type(struct
 device *dev)
      if (!dev_data)
              return 0;
 
+     /* Always use DMA domain for untrusted device */
+     if (to_pci_dev(dev)->untrusted)
+             return IOMMU_DOMAIN_DMA;
+
      /*
       * Do not identity map IOMMUv2 capable devices when:

* - memory encryption is active, because some of those devices
--
2.31.1

#3 lore raw

Alex,

Quoted 37 lines · Alexander

[Public]

Quoted 27 lines

-----Original Message-----
From: Hegde, Vasant <[email protected]>
Sent: Friday, April 19, 2024 7:47 AM
To: [email protected]; [email protected]
Cc: Suthikulpanit, Suravee <[email protected]>; Deucher, Alexander <[email protected]>; Hegde, Vasant <[email protected]>; Eric Wagner <[email protected]>; Robin Murphy <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]
Subject: [PATCH] iommu/amd: Enhance def_domain_type to handle untrusted device

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined sstate.

This patch adds untrusted check in AMD IOMMU driver code. So that it lets eGPUs behind Thunderbolt work again.

Fine tuning amd_iommu_def_domain_type() will be done later.

Quoted 7 lines

[email protected]

Fixes 59ddce4418da ("iommu: Reorganize

iommu_get_default_domain_type() to respect def_domain_type()")

Cc Robin Murphy <[email protected]>
Cc Jason Gunthorpe <[email protected]>
Quoted 2 lines

Assuming it fixes the issue, please add:

Sure. Will add it in v2.

-Vasant

Tested-by on v1 1/1
#4 lore raw
In reply to Vasant Hegde
Quoted 47 lines · Vasant Hegde

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined sstate.

This patch adds untrusted check in AMD IOMMU driver code. So that it lets eGPUs behind Thunderbolt work again.

Fine tuning amd_iommu_def_domain_type() will be done later.

Fixes 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()")
Cc Robin Murphy <[email protected]>
Cc Jason Gunthorpe <[email protected]>

---
@Eric,
Can you please test this patch?

(quoted diff) +0 -1
-Vasant
 
 drivers/iommu/amd/iommu.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/iommu/amd/iommu.c +4 -1
index e692217fcb28..e405714640b0 100644
@@ -2754,6 +2754,10 @@ static int amd_iommu_def_domain_type(struct device *dev)
        if (!dev_data)
                return 0;
 
+       /* Always use DMA domain for untrusted device */
+       if (to_pci_dev(dev)->untrusted)
+               return IOMMU_DOMAIN_DMA;
+
        /*
         * Do not identity map IOMMUv2 capable devices when:
         *  - memory encryption is active, because some of those devices
--
Quoted 2 lines

2.31.1

Tested the patch and confirmed it fixes the reported issue for me.

#5 lore raw
In reply to Vasant Hegde
Quoted 35 lines · Vasant Hegde

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined sstate.

This patch adds untrusted check in AMD IOMMU driver code. So that it lets eGPUs behind Thunderbolt work again.

Fine tuning amd_iommu_def_domain_type() will be done later.

Fixes 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()")
Cc Robin Murphy<[email protected]>
Cc Jason Gunthorpe<[email protected]>

---
@Eric,
Can you please test this patch?

(quoted diff) +0 -1
-Vasant
 
  drivers/iommu/amd/iommu.c | 4 ++++
  1 file changed, 4 insertions(+)
drivers/iommu/amd/iommu.c +0 -0
index e692217fcb28..e405714640b0 100644

drivers/iommu/amd/iommu.c static int amd_iommu_def_domain_type(struct device *dev) · line 2759

Quoted 6 lines
drivers/iommu/amd/iommu.c +3 -0
@@ -2754,6 +2754,10 @@ static int amd_iommu_def_domain_type(struct device *dev)
  	if (!dev_data)
  		return 0;
 
+	/* Always use DMA domain for untrusted device */
+	if (to_pci_dev(dev)->untrusted)
+		return IOMMU_DOMAIN_DMA;

Are you sure that dev always represents a pci device? If not, perhaps here it needs a dev_is_pci() check?

Best regards,
baolu

#6 lore raw

Hi,

Quoted 32 lines · Baolu Lu
Quoted 31 lines · Vasant Hegde

Previously, IOMMU core layer was forcing IOMMU_DOMAIN_DMA domain for untrusted device. This always took precedence over driver's def_domain_type(). Commit 59ddce4418da ("iommu: Reorganize iommu_get_default_domain_type() to respect def_domain_type()") changed the behaviour. Current code calls def_domain_type() but if it doesn't return IOMMU_DOMAIN_DMA for untrusted device it throws error. This results in IOMMU group (and potentially IOMMU itself) in undetermined sstate.

This patch adds untrusted check in AMD IOMMU driver code. So that it lets eGPUs behind Thunderbolt work again.

Fine tuning amd_iommu_def_domain_type() will be done later.

respect def_domain_type()")

Cc Robin Murphy<[email protected]>
Cc Jason Gunthorpe<[email protected]>
Cc [email protected]  # v6.7+

---
@Eric,
   Can you please test this patch?

-Vasant

  drivers/iommu/amd/iommu.c | 4 ++++

  1 file changed, 4 insertions(+)
Quoted 13 lines
Quoted 10 lines
drivers/iommu/amd/iommu.c +0 -0
index e692217fcb28..e405714640b0 100644
@@ -2754,6 +2754,10 @@ static int amd_iommu_def_domain_type(struct device *dev)
       if (!dev_data)
           return 0;

  +    /* Always use DMA domain for untrusted device */ +    if (to_pci_dev(dev)->untrusted)
+        return IOMMU_DOMAIN_DMA;

Are you sure that dev always represents a pci device? If not, perhaps here it needs a dev_is_pci() check?

You are right. I need to add check to verify whether its PCI device or not.

-Vasant