Skip to contentSkip to conversation

xen/arm: smmu: Set/clear IOMMU domain for device

PATCH Superseded Oleksandr Andrushchenko 3 participants 4 messages Aug 11, 2021 – Aug 18, 2021

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen/arm: smmu: Set/clear IOMMU domain for device

From: Oleksandr Andrushchenko <[email protected]>

When a device is assigned/de-assigned it is required to properly set IOMMU domain used to protect the device. This assignment was missing, thus it was not possible to de-assign the device:

(XEN) Deassigning device 0000:03:00.0 from dom2 (XEN) smmu: 0000:03:00.0: not attached to domain 2 (XEN) d2: deassign (0000:03:00.0) failed (-3)

Fix this by assigning IOMMU domain on arm_smmu_assign_dev and reset it to NULL on arm_smmu_deassign_dev.

---
 xen/drivers/passthrough/arm/smmu.c | 4 ++++
 1 file changed, 4 insertions(+)
xen/drivers/passthrough/arm/smmu.c +4 -0
index c234ad9c7f1e..373d9d4d123a 100644
@@ -2768,6 +2768,7 @@ static int arm_smmu_assign_dev(struct domain *d, u8 devfn,
 			arm_smmu_destroy_iommu_domain(domain);
 	} else {
 		atomic_inc(&domain->ref);
+		dev_iommu_domain(dev) = domain;
 	}
 
 out:
@@ -2794,7 +2795,10 @@ static int arm_smmu_deassign_dev(struct domain *d, struct device *dev)
 	atomic_dec(&domain->ref);
 
 	if (domain->ref.counter == 0)
+	{
 		arm_smmu_destroy_iommu_domain(domain);
+		dev_iommu_domain(dev) = NULL;
+	}
 
 	spin_unlock(&xen_domain->lock);
 
Signature · 1 line
2.25.1
Reviewed-by on v1 1/1
#1 lore raw

Hi Oleksandr,

Quoted 12 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

When a device is assigned/de-assigned it is required to properly set IOMMU domain used to protect the device. This assignment was missing, thus it was not possible to de-assign the device:

(XEN) Deassigning device 0000:03:00.0 from dom2 (XEN) smmu: 0000:03:00.0: not attached to domain 2 (XEN) d2: deassign (0000:03:00.0) failed (-3)

Quoted 5 lines

Fix this by assigning IOMMU domain on arm_smmu_assign_dev and reset it to NULL on arm_smmu_deassign_dev.

Regards,
Rahul

#2 lore raw
In reply to Oleksandr Andrushchenko

Hi Oleksandr,

Apologies for the late answer.

Quoted 6 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

When a device is assigned/de-assigned it is required to properly set IOMMU domain used to protect the device. This assignment was missing, thus it was not possible to de-assign the device:

Quoted 6 lines

(XEN) Deassigning device 0000:03:00.0 from dom2 (XEN) smmu: 0000:03:00.0: not attached to domain 2 (XEN) d2: deassign (0000:03:00.0) failed (-3)

Fix this by assigning IOMMU domain on arm_smmu_assign_dev and reset it to NULL on arm_smmu_deassign_dev.

I think this was introduced by commit 06d1f7a278dd "xen/arm: smmuv1: Keep track of S2CR state". If so, please add:

Fixes 06d1f7a278dd ("xen/arm: smmuv1: Keep track of S2CR state")

Looking at the commit message, the IOMMU domain used to be set/unset in arm_smmu_{attach, detach}_dev() but Linux drop it because they now rely in the core IOMMU framework to track the domain.

So I agree with the new position for...

Quoted 12 lines
---
  xen/drivers/passthrough/arm/smmu.c | 4 ++++
  1 file changed, 4 insertions(+)
xen/drivers/passthrough/arm/smmu.c +0 -0
index c234ad9c7f1e..373d9d4d123a 100644

xen/drivers/passthrough/arm/smmu.c static int arm_smmu_assign_dev(struct domain *d, u8 devfn, · line 2771

Quoted 4 lines
xen/drivers/passthrough/arm/smmu.c +1 -0
@@ -2768,6 +2768,7 @@ static int arm_smmu_assign_dev(struct domain *d, u8 devfn,
  			arm_smmu_destroy_iommu_domain(domain);
  	} else {
  		atomic_inc(&domain->ref);
+		dev_iommu_domain(dev) = domain;

... this one. However...

xen/drivers/passthrough/arm/smmu.c static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) · line 2801

Quoted 11 lines
xen/drivers/passthrough/arm/smmu.c +3 -0
  	}
 
  out:
@@ -2794,7 +2795,10 @@ static int arm_smmu_deassign_dev(struct domain *d, struct device *dev)
  	atomic_dec(&domain->ref);
 
  	if (domain->ref.counter == 0)
+	{
  		arm_smmu_destroy_iommu_domain(domain);
+		dev_iommu_domain(dev) = NULL;
+	}

... I think this one is incorrect because you would only unset dev_iommu_domain() is the refcount drop to 0. You can have multiple device in the same domain, so the ref.counter may not be 0.

So I think this needs to be moved outside of the if. Note that it is also a good practice to remove any reference (e.g. set to NULL) before freeing even if it doesn't much matter here.

Lastly, the file is using the Linux coding style. So { needs to be on the same line as the if.

Cheers,

Signature · 1 line
Julien Grall
#3 lore raw

Hi, Julien!

Quoted 15 lines · Julien Grall

Hi Oleksandr,

Apologies for the late answer.

Quoted 10 lines · Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <[email protected]>

When a device is assigned/de-assigned it is required to properly set IOMMU domain used to protect the device. This assignment was missing, thus it was not possible to de-assign the device:

(XEN) Deassigning device 0000:03:00.0 from dom2 (XEN) smmu: 0000:03:00.0:  not attached to domain 2 (XEN) d2: deassign (0000:03:00.0) failed (-3)

Quoted 5 lines
Quoted 2 lines

Fix this by assigning IOMMU domain on arm_smmu_assign_dev and reset it to NULL on arm_smmu_deassign_dev.

I think this was introduced by commit 06d1f7a278dd "xen/arm: smmuv1: Keep track of S2CR state". If so, please add:

Fixes 06d1f7a278dd ("xen/arm: smmuv1: Keep track of S2CR state")

Will do

Quoted 36 lines

Looking at the commit message, the IOMMU domain used to be set/unset in arm_smmu_{attach, detach}_dev() but Linux drop it because they now rely in the core IOMMU framework to track the domain.

So I agree with the new position for...

Quoted 12 lines

---
  xen/drivers/passthrough/arm/smmu.c | 4 ++++

  1 file changed, 4 insertions(+)
xen/drivers/passthrough/arm/smmu.c +0 -0
index c234ad9c7f1e..373d9d4d123a 100644

xen/drivers/passthrough/arm/smmu.c static int arm_smmu_assign_dev(struct domain *d, u8 devfn, · line 2771

Quoted 4 lines
xen/drivers/passthrough/arm/smmu.c +1 -0
@@ -2768,6 +2768,7 @@ static int arm_smmu_assign_dev(struct domain *d, u8 devfn,
               arm_smmu_destroy_iommu_domain(domain);
       } else {
           atomic_inc(&domain->ref);
+        dev_iommu_domain(dev) = domain;

... this one. However...

xen/drivers/passthrough/arm/smmu.c static int arm_smmu_deassign_dev(struct domain *d, struct device *dev) · line 2800

Quoted 9 lines
xen/drivers/passthrough/arm/smmu.c +0 -0
       }

    out:

xen/drivers/passthrough/arm/smmu.c +3 -0
@@ -2794,7 +2795,10 @@ static int arm_smmu_deassign_dev(struct domain *d, struct device *dev)
       atomic_dec(&domain->ref);
         if (domain->ref.counter == 0)
+    {
           arm_smmu_destroy_iommu_domain(domain);
+        dev_iommu_domain(dev) = NULL;
+    }

... I think this one is incorrect because you would only unset dev_iommu_domain() is the refcount drop to 0. You can have multiple device in the same domain, so the ref.counter may not be 0.

Quoted 1 line

So I think this needs to be moved outside of the if.

Yes, absolutely

Quoted 6 lines

Note that it is also a good practice to remove any reference (e.g. set to NULL) before freeing even if it doesn't much matter here.

Lastly, the file is using the Linux coding style. So { needs to be on the same line as the if.

Cheers,

Thanks,

Oleksandr