Skip to contentSkip to conversation

iommu: Make pasid array per device

PATCH In review Lu Baolu 3 participants 12 messages Aug 14, 2023 – Aug 22, 2023

Subsystems (2)
lore
Share & tools
lore

Lu Baolu

author
#0 lore raw

v2 cover iommu: Make pasid array per device

The PCI PASID enabling interface guarantees that the address space used by each PASID is unique. This is achieved by checking that the PCI ACS path is enabled for the device. If the path is not enabled, then the PASID feature cannot be used.

if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF))
return -EINVAL;

The PASID array is not an attribute of the IOMMU group. It is more natural to store the PASID array in the per-device IOMMU data. This makes the code clearer and easier to understand.

Please help review and suggest.

Change log:
v2:
- Add an explict check for single-device group in the domain attaching device pasid interface. (Jason)
- Make assert_pasid_dma_ownership() returns true or false. Refactor the code in a way that does not change its exsiting behavior. (Kevin)

v1: https://lore.kernel.org/linux-iommu/[email protected]/

Lu Baolu (3):
iommu: Make single-device group for PASID explicit
iommu: Consolidate pasid dma ownership check
iommu: Move pasid array from group to device

 include/linux/iommu.h |   2 +
 drivers/iommu/iommu.c | 102 +++++++++++++++++++-----------------------
 2 files changed, 49 insertions(+), 55 deletions(-)
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH v2 1/3 iommu: Make single-device group for PASID explicit

The PASID interfaces have always supported only single-device groups. This was first introduced in commit 26b25a2b98e45 ("iommu: Bind process address spaces to devices"), and has been kept consistent in subsequent commits.

However, the core code doesn't explicitly check for this requirement after commit 201007ef707a8 ("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit.

Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.

---
 drivers/iommu/iommu.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/iommu/iommu.c +5 -0
index 71b9c41f2a9e..f1eba60e573f 100644
@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
 		return -ENODEV;
 
 	mutex_lock(&group->mutex);
+	if (list_count_nodes(&group->devices) != 1) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
 	curr = xa_cmpxchg(&group->pasid_array, pasid, NULL, domain, GFP_KERNEL);
 	if (curr) {
 		ret = xa_err(curr) ? : -EBUSY;
Signature · 1 line
2.34.1
#2 lore raw

RE: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit

Quoted 25 lines

From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AM

The PASID interfaces have always supported only single-device groups. This was first introduced in commit 26b25a2b98e45 ("iommu: Bind process address spaces to devices"), and has been kept consistent in subsequent commits.

However, the core code doesn't explicitly check for this requirement after commit 201007ef707a8 ("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit.

Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.

---
 drivers/iommu/iommu.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/iommu/iommu.c +0 -0
index 71b9c41f2a9e..f1eba60e573f 100644

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct · line 3416

Quoted 9 lines
drivers/iommu/iommu.c +5 -0
@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct
 iommu_domain *domain,
 		return -ENODEV;
 
 	mutex_lock(&group->mutex);
+	if (list_count_nodes(&group->devices) != 1) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+

I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the check alone at attach time doesn't mean this group won't be expanded to have multiple devices later.

author
#3 lore raw

Re: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit

Quoted 28 lines · Kevin
Quoted 28 lines

From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AM

The PASID interfaces have always supported only single-device groups. This was first introduced in commit 26b25a2b98e45 ("iommu: Bind process address spaces to devices"), and has been kept consistent in subsequent commits.

However, the core code doesn't explicitly check for this requirement after commit 201007ef707a8 ("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit.

Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.

---
  drivers/iommu/iommu.c | 5 +++++
  1 file changed, 5 insertions(+)
drivers/iommu/iommu.c +0 -0
index 71b9c41f2a9e..f1eba60e573f 100644
@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct
 iommu_domain *domain,
  		return -ENODEV;
 
Quoted 9 lines

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3415

Quoted 6 lines
drivers/iommu/iommu.c +5 -0
  	mutex_lock(&group->mutex);
+	if (list_count_nodes(&group->devices) != 1) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+

I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the

This has been guaranteed by pci_enable_pasid():

if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF))
return -EINVAL;

Quoted 2 lines

check alone at attach time doesn't mean this group won't be expanded to have multiple devices later.

Best regards,
baolu

#4 lore raw

RE: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit

Quoted 43 lines

From: Baolu Lu <[email protected]>
Sent: Monday, August 21, 2023 1:45 PM

Quoted 5 lines · Kevin
Quoted 5 lines

From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AM

The PASID interfaces have always supported only single-device groups. This was first introduced in commit 26b25a2b98e45 ("iommu: Bind

process

Quoted 23 lines
Quoted 23 lines

address spaces to devices"), and has been kept consistent in subsequent commits.

However, the core code doesn't explicitly check for this requirement after commit 201007ef707a8 ("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit.

Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.

---
  drivers/iommu/iommu.c | 5 +++++
  1 file changed, 5 insertions(+)
drivers/iommu/iommu.c +0 -0
index 71b9c41f2a9e..f1eba60e573f 100644
@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct
 iommu_domain *domain,
  		return -ENODEV;
 
Quoted 9 lines

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3415

Quoted 6 lines
drivers/iommu/iommu.c +5 -0
  	mutex_lock(&group->mutex);
+	if (list_count_nodes(&group->devices) != 1) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+

I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the

Quoted 5 lines

This has been guaranteed by pci_enable_pasid():

if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF))
return -EINVAL;

well since you are adding generic core check then it's not good to rely on the fact of a specific bus...

author
#5 lore raw

Re: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit

Quoted 43 lines · Kevin
Quoted 43 lines

From: Baolu Lu <[email protected]>
Sent: Monday, August 21, 2023 1:45 PM

Quoted 5 lines · Kevin
Quoted 5 lines

From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AM

The PASID interfaces have always supported only single-device groups. This was first introduced in commit 26b25a2b98e45 ("iommu: Bind

process

Quoted 32 lines
Quoted 20 lines

address spaces to devices"), and has been kept consistent in subsequent commits.

However, the core code doesn't explicitly check for this requirement after commit 201007ef707a8 ("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit.

Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.

---
   drivers/iommu/iommu.c | 5 +++++
   1 file changed, 5 insertions(+)
drivers/iommu/iommu.c +0 -0
index 71b9c41f2a9e..f1eba60e573f 100644

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct · line 3416

Quoted 9 lines
drivers/iommu/iommu.c +5 -0
@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct
 iommu_domain *domain,
   		return -ENODEV;
 
   	mutex_lock(&group->mutex);
+	if (list_count_nodes(&group->devices) != 1) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+

I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the

Quoted 8 lines
Quoted 5 lines

This has been guaranteed by pci_enable_pasid():

if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF))
return -EINVAL;

well since you are adding generic core check then it's not good to rely on the fact of a specific bus...

We attempted to do this in the patch linked below.

https://lore.kernel.org/linux-iommu/[email protected]/

After long discussion, we decided to move it to the pci_enable_pasid() interface. The non-static single device group is only relevant to PCI fabrics that support hot-plugging without ACS support on the upstream path.

Best regards,
baolu

#6 lore raw

RE: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit

Quoted 60 lines

From: Baolu Lu <[email protected]>
Sent: Tuesday, August 22, 2023 2:37 PM

Quoted 11 lines · Kevin
Quoted 11 lines

From: Baolu Lu <[email protected]>
Sent: Monday, August 21, 2023 1:45 PM

Quoted 5 lines · Kevin
Quoted 5 lines

From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AM

The PASID interfaces have always supported only single-device groups. This was first introduced in commit 26b25a2b98e45 ("iommu: Bind

process

Quoted 1 line
Quoted 1 line

address spaces to devices"), and has been kept consistent in

subsequent

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

commits.

However, the core code doesn't explicitly check for this requirement after commit 201007ef707a8 ("PCI: Enable PASID only when ACS RR &

UF

Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
Quoted 15 lines

enabled on upstream path"), which made this requirement implicit.

Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.

---
   drivers/iommu/iommu.c | 5 +++++
   1 file changed, 5 insertions(+)
drivers/iommu/iommu.c +0 -0
index 71b9c41f2a9e..f1eba60e573f 100644

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct · line 3416

Quoted 9 lines
drivers/iommu/iommu.c +5 -0
@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct
 iommu_domain *domain,
   		return -ENODEV;
 
   	mutex_lock(&group->mutex);
+	if (list_count_nodes(&group->devices) != 1) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+

I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the

Quoted 8 lines
Quoted 5 lines

This has been guaranteed by pci_enable_pasid():

if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF))
return -EINVAL;

well since you are adding generic core check then it's not good to rely on the fact of a specific bus...

We attempted to do this in the patch linked below.

Quoted 8 lines

https://lore.kernel.org/linux-iommu/20220705050710.2887204-5-
[email protected]/

After long discussion, we decided to move it to the pci_enable_pasid() interface. The non-static single device group is only relevant to PCI fabrics that support hot-plugging without ACS support on the upstream path.

If that's the case better add a comment to include this fact. So another one looking at this code won't fall into the same puzzle wondering what about a group becoming non-singleton after above check. 😊

author
#7 lore raw

Re: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit

Quoted 69 lines · Kevin
Quoted 60 lines

From: Baolu Lu <[email protected]>
Sent: Tuesday, August 22, 2023 2:37 PM

Quoted 11 lines · Kevin
Quoted 11 lines

From: Baolu Lu <[email protected]>
Sent: Monday, August 21, 2023 1:45 PM

Quoted 5 lines · Kevin
Quoted 5 lines

From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AM

The PASID interfaces have always supported only single-device groups. This was first introduced in commit 26b25a2b98e45 ("iommu: Bind

process

Quoted 1 line
Quoted 1 line

address spaces to devices"), and has been kept consistent in

subsequent

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

commits.

However, the core code doesn't explicitly check for this requirement after commit 201007ef707a8 ("PCI: Enable PASID only when ACS RR &

UF

Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
Quoted 15 lines

enabled on upstream path"), which made this requirement implicit.

Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.

---
    drivers/iommu/iommu.c | 5 +++++
    1 file changed, 5 insertions(+)
drivers/iommu/iommu.c +0 -0
index 71b9c41f2a9e..f1eba60e573f 100644

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct · line 3416

Quoted 9 lines
drivers/iommu/iommu.c +5 -0
@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct
 iommu_domain *domain,
    		return -ENODEV;
 
    	mutex_lock(&group->mutex);
+	if (list_count_nodes(&group->devices) != 1) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+

I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the

Quoted 8 lines
Quoted 5 lines

This has been guaranteed by pci_enable_pasid():

if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF))
return -EINVAL;

well since you are adding generic core check then it's not good to rely on the fact of a specific bus...

We attempted to do this in the patch linked below.

Quoted 8 lines

https://lore.kernel.org/linux-iommu/20220705050710.2887204-5-
[email protected]/

After long discussion, we decided to move it to the pci_enable_pasid() interface. The non-static single device group is only relevant to PCI fabrics that support hot-plugging without ACS support on the upstream path.

Quoted 4 lines

If that's the case better add a comment to include this fact. So another one looking at this code won't fall into the same puzzle wondering what about a group becoming non-singleton after above check. 😊

Yeah, fair enough.

Best regards,
baolu

author
#8 lore raw

PATCH v2 2/3 iommu: Consolidate pasid dma ownership check

In reply to Lu Baolu

When switching device DMA ownership, it is required that all the device's pasid DMA be disabled. This is done by checking if the pasid array of the group is empty. Consolidate all the open code into a single helper. No intentional functionality change.

---
 drivers/iommu/iommu.c | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
drivers/iommu/iommu.c +18 -5
index f1eba60e573f..d4a06a37ce39 100644
@@ -3127,6 +3127,19 @@ static bool iommu_is_default_domain(struct iommu_group *group)
 	return false;
 }
 
+/*
+ * Assert no PASID DMA when claiming or releasing group's DMA ownership.
+ * The device pasid interfaces are only for device drivers that have
+ * claimed the DMA ownership. Return true if no pasid DMA setup, otherwise
+ * return false with a WARN().
+ */
+static bool assert_pasid_dma_ownership(struct iommu_group *group)
+{
+	lockdep_assert_held(&group->mutex);
+
+	return !WARN_ON(!xa_empty(&group->pasid_array));
+}
+
 /**
  * iommu_device_use_default_domain() - Device driver wants to handle device
  *                                     DMA through the kernel DMA API.
@@ -3147,7 +3160,7 @@ int iommu_device_use_default_domain(struct device *dev)
 	mutex_lock(&group->mutex);
 	if (group->owner_cnt) {
 		if (group->owner || !iommu_is_default_domain(group) ||
-		    !xa_empty(&group->pasid_array)) {
+		    !assert_pasid_dma_ownership(group)) {
 			ret = -EBUSY;
 			goto unlock_out;
 		}
@@ -3177,7 +3190,7 @@ void iommu_device_unuse_default_domain(struct device *dev)
 		return;
 
 	mutex_lock(&group->mutex);
-	if (!WARN_ON(!group->owner_cnt || !xa_empty(&group->pasid_array)))
+	if (!WARN_ON(!group->owner_cnt) && assert_pasid_dma_ownership(group))
 		group->owner_cnt--;
 
 	mutex_unlock(&group->mutex);
@@ -3211,7 +3224,7 @@ static int __iommu_take_dma_ownership(struct iommu_group *group, void *owner)
 	int ret;
 
 	if ((group->domain && group->domain != group->default_domain) ||
-	    !xa_empty(&group->pasid_array))
+	    !assert_pasid_dma_ownership(group))
 		return -EBUSY;
 
 	ret = __iommu_group_alloc_blocking_domain(group);
@@ -3296,8 +3309,8 @@ EXPORT_SYMBOL_GPL(iommu_device_claim_dma_owner);
 
 static void __iommu_release_dma_ownership(struct iommu_group *group)
 {
-	if (WARN_ON(!group->owner_cnt || !group->owner ||
-		    !xa_empty(&group->pasid_array)))
+	if (WARN_ON(!group->owner_cnt || !group->owner) ||
+	    !assert_pasid_dma_ownership(group))
 		return;
 
 	group->owner_cnt = 0;
Signature · 1 line
2.34.1
#9 lore raw

Re: [PATCH v2 2/3] iommu: Consolidate pasid dma ownership check

Quoted 15 lines · Lu Baolu

When switching device DMA ownership, it is required that all the device's pasid DMA be disabled. This is done by checking if the pasid array of the group is empty. Consolidate all the open code into a single helper. No intentional functionality change.

---
 drivers/iommu/iommu.c | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
drivers/iommu/iommu.c +0 -0
index f1eba60e573f..d4a06a37ce39 100644

drivers/iommu/iommu.c static bool iommu_is_default_domain(struct iommu_group *group) · line 3131

Quoted 5 lines
drivers/iommu/iommu.c +2 -0
@@ -3127,6 +3127,19 @@ static bool iommu_is_default_domain(struct iommu_group *group)
        return false;
 }
 
+/*
+ * Assert no PASID DMA when claiming or releasing group's DMA ownership.
             ^
	     |...

drivers/iommu/iommu.c static bool iommu_is_default_domain(struct iommu_group *group) · line 3136

Quoted 5 lines
drivers/iommu/iommu.c +5 -0
+ * The device pasid interfaces are only for device drivers that have
+ * claimed the DMA ownership. Return true if no pasid DMA setup, otherwise
+ * return false with a WARN().
+ */
+static bool assert_pasid_dma_ownership(struct iommu_group *group)

... should it be assert_no_pasid_dma_ownership?

Nic

author
#10 lore raw

Re: [PATCH v2 2/3] iommu: Consolidate pasid dma ownership check

Quoted 25 lines · Nicolin Chen
Quoted 15 lines · Lu Baolu

When switching device DMA ownership, it is required that all the device's pasid DMA be disabled. This is done by checking if the pasid array of the group is empty. Consolidate all the open code into a single helper. No intentional functionality change.

---
  drivers/iommu/iommu.c | 23 ++++++++++++++++++-----
  1 file changed, 18 insertions(+), 5 deletions(-)
drivers/iommu/iommu.c +0 -0
index f1eba60e573f..d4a06a37ce39 100644

drivers/iommu/iommu.c static bool iommu_is_default_domain(struct iommu_group *group) · line 3131

Quoted 5 lines
drivers/iommu/iommu.c +2 -0
@@ -3127,6 +3127,19 @@ static bool iommu_is_default_domain(struct iommu_group *group)
         return false;
  }
 
+/*
+ * Assert no PASID DMA when claiming or releasing group's DMA ownership.
              ^
	     |...
Quoted 6 lines

drivers/iommu/iommu.c static bool iommu_is_default_domain(struct iommu_group *group) · line 3136

Quoted 5 lines
drivers/iommu/iommu.c +5 -0
+ * The device pasid interfaces are only for device drivers that have
+ * claimed the DMA ownership. Return true if no pasid DMA setup, otherwise
+ * return false with a WARN().
+ */
+static bool assert_pasid_dma_ownership(struct iommu_group *group)

... should it be assert_no_pasid_dma_ownership?

Fair enough. Or, perhaps just assert_no_pasid_dma()?

Best regards,
baolu

author
#11 lore raw

PATCH v2 3/3 iommu: Move pasid array from group to device

In reply to Lu Baolu

The PASID (Process Address Space ID) feature is a device feature that allows a device driver to manage the PASID value and attach or detach its domain to the pasid. The pasid array, which is used to store the domain of each pasid, is currently stored in iommu group struct, but it would be more natural to move it to the device so that the device drivers don't need to understand the internal iommu group concept.

---
 include/linux/iommu.h |  2 ++
 drivers/iommu/iommu.c | 80 +++++++++++++++----------------------------
 2 files changed, 29 insertions(+), 53 deletions(-)
include/linux/iommu.h +2 -0
index 411ce9b998dc..76c960741449 100644
@@ -412,6 +412,7 @@ struct iommu_fault_param {
  * @iopf_param:	 I/O Page Fault queue and data
  * @fwspec:	 IOMMU fwspec data
  * @iommu_dev:	 IOMMU device this device is linked to
+ * @pasid_array: pasid-indexed array of domains attached to pasid
  * @priv:	 IOMMU Driver private data
  * @max_pasids:  number of PASIDs this device can consume
  * @attach_deferred: the dma domain attachment is deferred
@@ -427,6 +428,7 @@ struct dev_iommu {
 	struct iopf_device_param	*iopf_param;
 	struct iommu_fwspec		*fwspec;
 	struct iommu_device		*iommu_dev;
+	struct xarray			pasid_array;
 	void				*priv;
 	u32				max_pasids;
 	u32				attach_deferred:1;
drivers/iommu/iommu.c +27 -53

149 lines not on this page · raw message

Signature · 1 line
2.34.1