Skip to contentSkip to conversation

Two enhancements to iommu_attach_device_pasid()

PATCH In review Yi Liu 4 participants 11 messages Jan 20, 2025 – Jan 21, 2025 IOMMU

lore
Share & tools
lore

Yi Liu

author
#0 lore raw

cover Two enhancements to iommu_attach_device_pasid()

It's spotted during the review process of iommufd pasid series [1]. The first one is about storing entry to group->pasid_array, and the second one is to avoid the potential PRIs anchored on the failed domain. Details can be found in the individial patch.

[1] https://lore.kernel.org/linux-iommu/[email protected]/

Yi Liu (2):
iommu: Store either domain or handle in group->pasid_array
iommu: Swap the order of setting group->pasid_array and

    __iommu_set_group_pasid()
 drivers/iommu/iommu.c | 40 ++++++++++++++++++++++++++++++----------
 1 file changed, 30 insertions(+), 10 deletions(-)
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH 1/2 iommu: Store either domain or handle in group->pasid_array

iommu_attach_device_pasid() only stores handle to group->pasid_array when there is a valid handle input. However, it makes the iommu_attach_device_pasid() unable to detect if the pasid has been attached or not. To be complete, let the iommu_attach_device_pasid() store the domain to group->pasid_array if no valid handle.

---
 drivers/iommu/iommu.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
drivers/iommu/iommu.c +17 -6
index 28ffd836592b..278c4eb8f225 100644
@@ -45,6 +45,8 @@ static unsigned int iommu_def_domain_type __read_mostly;
 static bool iommu_dma_strict __read_mostly = IS_ENABLED(CONFIG_IOMMU_DEFAULT_DMA_STRICT);
 static u32 iommu_cmd_line __read_mostly;
 
+enum { IOMMU_PASID_ARRAY_DOMAIN, IOMMU_PASID_ARRAY_HANDLE };
+
 struct iommu_group {
 	struct kobject kobj;
 	struct kobject *devices_kobj;
@@ -3374,6 +3376,7 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
 	struct iommu_group *group = dev->iommu_group;
 	struct group_device *device;
 	const struct iommu_ops *ops;
+	void *xa_entry;
 	int ret;
 
 	if (!group)
@@ -3397,10 +3400,14 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
 		}
 	}
 
-	if (handle)
+	if (handle) {
 		handle->domain = domain;
+		xa_entry = xa_tag_pointer(handle, IOMMU_PASID_ARRAY_HANDLE);
+	} else {
+		xa_entry = xa_tag_pointer(domain, IOMMU_PASID_ARRAY_DOMAIN);
+	}
 
-	ret = xa_insert(&group->pasid_array, pasid, handle, GFP_KERNEL);
+	ret = xa_insert(&group->pasid_array, pasid, xa_entry, GFP_KERNEL);
 	if (ret)
 		goto out_unlock;
 
@@ -3480,13 +3487,17 @@ struct iommu_attach_handle *
 iommu_attach_handle_get(struct iommu_group *group, ioasid_t pasid, unsigned int type)
 {
 	struct iommu_attach_handle *handle;
+	void *xa_entry;
 
 	xa_lock(&group->pasid_array);
-	handle = xa_load(&group->pasid_array, pasid);
-	if (!handle)
+	xa_entry = xa_load(&group->pasid_array, pasid);
+	if (xa_pointer_tag(xa_entry) == IOMMU_PASID_ARRAY_HANDLE) {
+		handle = xa_untag_pointer(xa_entry);
+		if (type && handle->domain->type != type)
+			handle = ERR_PTR(-EBUSY);
+	} else {
 		handle = ERR_PTR(-ENOENT);
-	else if (type && handle->domain->type != type)
-		handle = ERR_PTR(-EBUSY);
+	}
 	xa_unlock(&group->pasid_array);
 
 	return handle;
Signature · 1 line
2.34.1
Reviewed-by on v1 1/2
#2 lore raw

RE: [PATCH 1/2] iommu: Store either domain or handle in group->pasid_array

Quoted 11 lines

From: Yi Liu <[email protected]>
Sent: Monday, January 20, 2025 11:09 AM

iommu_attach_device_pasid() only stores handle to group->pasid_array when there is a valid handle input. However, it makes the iommu_attach_device_pasid() unable to detect if the pasid has been attached or not. To be complete, let the iommu_attach_device_pasid() store the domain to group->pasid_array if no valid handle.

#3 lore raw

Re: [PATCH 1/2] iommu: Store either domain or handle in group->pasid_array

In reply to Yi Liu
Quoted 17 lines · Yi Liu

iommu_attach_device_pasid() only stores handle to group->pasid_array when there is a valid handle input. However, it makes the iommu_attach_device_pasid() unable to detect if the pasid has been attached or not. To be complete, let the iommu_attach_device_pasid() store the domain to group->pasid_array if no valid handle.

---
 drivers/iommu/iommu.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
drivers/iommu/iommu.c +0 -0
index 28ffd836592b..278c4eb8f225 100644

drivers/iommu/iommu.c static unsigned int iommu_def_domain_type __read_mostly; · line 48

Quoted 4 lines
drivers/iommu/iommu.c +1 -0
@@ -45,6 +45,8 @@ static unsigned int iommu_def_domain_type __read_mostly;
 static bool iommu_dma_strict __read_mostly = IS_ENABLED(CONFIG_IOMMU_DEFAULT_DMA_STRICT);
 static u32 iommu_cmd_line __read_mostly;
 
+enum { IOMMU_PASID_ARRAY_DOMAIN, IOMMU_PASID_ARRAY_HANDLE };

Since xarray says:

* @tag: Tag value (0, 1 or 3).

I would initialize those enums and add a small comment

/* tags used with xa_tag_pointer() */
enum { IOMMU_PASID_ARRAY_DOMAIN = 0, IOMMU_PASID_ARRAY_HANDLE = 1 };

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3408

Quoted 11 lines
drivers/iommu/iommu.c +5 -1
@@ -3397,10 +3400,14 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
 		}
 	}
 
-	if (handle)
+	if (handle) {
 		handle->domain = domain;
+		xa_entry = xa_tag_pointer(handle, IOMMU_PASID_ARRAY_HANDLE);
+	} else {
+		xa_entry = xa_tag_pointer(domain, IOMMU_PASID_ARRAY_DOMAIN);
+	}

What about iommu_attach_group_handle() ?

	mutex_lock(&group->mutex);
	ret = xa_insert(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL);
	if (ret)

And iommu_replace_group_handle() ?

	curr = xa_store(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL);

I think it should be consistent and it should call xa_tag_poiner() too (even though it implicitly always is since the domain is stored in the group)??

Jason

#4 lore raw

Re: [PATCH 1/2] iommu: Store either domain or handle in group->pasid_array

Quoted 17 lines · Jason Gunthorpe

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3408

Quoted 11 lines
drivers/iommu/iommu.c +5 -1
@@ -3397,10 +3400,14 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
  		}
  	}
 
-	if (handle)
+	if (handle) {
  		handle->domain = domain;
+		xa_entry = xa_tag_pointer(handle, IOMMU_PASID_ARRAY_HANDLE);
+	} else {
+		xa_entry = xa_tag_pointer(domain, IOMMU_PASID_ARRAY_DOMAIN);
+	}

What about iommu_attach_group_handle() ?

	mutex_lock(&group->mutex);
	ret = xa_insert(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL);
	if (ret)
Quoted 7 lines

And iommu_replace_group_handle() ?

	curr = xa_store(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL);

I think it should be consistent and it should call xa_tag_poiner() too (even though it implicitly always is since the domain is stored in the group)??

We may eventually consider retiring group->domain.

---
baolu

author
#5 lore raw

Re: [PATCH 1/2] iommu: Store either domain or handle in group->pasid_array

In reply to Jason Gunthorpe
Quoted 21 lines · Jason Gunthorpe
Quoted 20 lines · Yi Liu

iommu_attach_device_pasid() only stores handle to group->pasid_array when there is a valid handle input. However, it makes the iommu_attach_device_pasid() unable to detect if the pasid has been attached or not. To be complete, let the iommu_attach_device_pasid() store the domain to group->pasid_array if no valid handle.

---
  drivers/iommu/iommu.c | 23 +++++++++++++++++------
  1 file changed, 17 insertions(+), 6 deletions(-)
drivers/iommu/iommu.c +0 -0
index 28ffd836592b..278c4eb8f225 100644
@@ -45,6 +45,8 @@ static unsigned int iommu_def_domain_type __read_mostly;
  static bool iommu_dma_strict __read_mostly = IS_ENABLED(CONFIG_IOMMU_DEFAULT_DMA_STRICT);
  static u32 iommu_cmd_line __read_mostly;
 
Quoted 7 lines

drivers/iommu/iommu.c static unsigned int iommu_def_domain_type __read_mostly; · line 48

Quoted 1 line
drivers/iommu/iommu.c +1 -0
+enum { IOMMU_PASID_ARRAY_DOMAIN, IOMMU_PASID_ARRAY_HANDLE };

Since xarray says:

* @tag: Tag value (0, 1 or 3).

I would initialize those enums and add a small comment

yes.

Quoted 21 lines

/* tags used with xa_tag_pointer() */
enum { IOMMU_PASID_ARRAY_DOMAIN = 0, IOMMU_PASID_ARRAY_HANDLE = 1 };

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3408

Quoted 11 lines
drivers/iommu/iommu.c +5 -1
@@ -3397,10 +3400,14 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
  		}
  	}
 
-	if (handle)
+	if (handle) {
  		handle->domain = domain;
+		xa_entry = xa_tag_pointer(handle, IOMMU_PASID_ARRAY_HANDLE);
+	} else {
+		xa_entry = xa_tag_pointer(domain, IOMMU_PASID_ARRAY_DOMAIN);
+	}

What about iommu_attach_group_handle() ?

	mutex_lock(&group->mutex);
	ret = xa_insert(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL);
	if (ret)
Quoted 7 lines

And iommu_replace_group_handle() ?

	curr = xa_store(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL);

I think it should be consistent and it should call xa_tag_poiner() too (even though it implicitly always is since the domain is stored in the group)??

yes, it needs to be fixed as well. I asked a help from Baolu in prior email, but I can enlarge this series to cover both RID and PASID path. :)

Signature · 2 lines
Regards,
Yi Liu
author
#6 lore raw

PATCH 2/2 iommu: Swap the order of setting group->pasid_array and __iommu_set_group_pasid()

In reply to Yi Liu

Following the current order of setting group->pasid_array and __iommu_set_group_pasid() in iommu_attach_device_pasid(), PRIs may be forwarded to the domain before the attach succeeds. If the attach failed in the end, the PRIs on the domain need to be flushed in the caller side. Caller can do it, but it can be avoided by swapping the order. This is more self-contained.

---
 drivers/iommu/iommu.c | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)
drivers/iommu/iommu.c +16 -7
index 278c4eb8f225..b69bcf559839 100644
@@ -3376,7 +3376,7 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
 	struct iommu_group *group = dev->iommu_group;
 	struct group_device *device;
 	const struct iommu_ops *ops;
-	void *xa_entry;
+	void *curr, *xa_entry;
 	int ret;
 
 	if (!group)
@@ -3400,6 +3400,16 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
 		}
 	}
 
+	curr = xa_load(&group->pasid_array, pasid);
+	if (curr) {
+		ret = -EBUSY;
+		goto out_unlock;
+	}
+
+	ret = __iommu_set_group_pasid(domain, group, pasid);
+	if (ret)
+		goto out_unlock;
+
 	if (handle) {
 		handle->domain = domain;
 		xa_entry = xa_tag_pointer(handle, IOMMU_PASID_ARRAY_HANDLE);
@@ -3407,13 +3417,12 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
 		xa_entry = xa_tag_pointer(domain, IOMMU_PASID_ARRAY_DOMAIN);
 	}
 
-	ret = xa_insert(&group->pasid_array, pasid, xa_entry, GFP_KERNEL);
-	if (ret)
-		goto out_unlock;
+	curr = xa_store(&group->pasid_array, pasid, handle, GFP_KERNEL);
+	if (curr) {
+		__iommu_remove_group_pasid(group, pasid, domain);
+		ret = xa_err(curr);
+	}
 
-	ret = __iommu_set_group_pasid(domain, group, pasid);
-	if (ret)
-		xa_erase(&group->pasid_array, pasid);
 out_unlock:
 	mutex_unlock(&group->mutex);
 	return ret;
Signature · 1 line
2.34.1
#7 lore raw

RE: [PATCH 2/2] iommu: Swap the order of setting group->pasid_array and __iommu_set_group_pasid()

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3424

Quoted 12 lines

From: Yi Liu <[email protected]>
Sent: Monday, January 20, 2025 11:09 AM

drivers/iommu/iommu.c +5 -3
-     ret = xa_insert(&group->pasid_array, pasid, xa_entry, GFP_KERNEL);
-     if (ret)
-             goto out_unlock;
+     curr = xa_store(&group->pasid_array, pasid, handle, GFP_KERNEL);
+     if (curr) {
+             __iommu_remove_group_pasid(group, pasid, domain);
+             ret = xa_err(curr);
+     }

s/handle/xa_entry/

btw you may continue to use xa_insert() to save one line for xa_err()

author
#8 lore raw

Re: [PATCH 2/2] iommu: Swap the order of setting group->pasid_array and __iommu_set_group_pasid()

Quoted 14 lines · Kevin

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3424

Quoted 12 lines

From: Yi Liu <[email protected]>
Sent: Monday, January 20, 2025 11:09 AM

drivers/iommu/iommu.c +5 -3
-	ret = xa_insert(&group->pasid_array, pasid, xa_entry, GFP_KERNEL);
-	if (ret)
-		goto out_unlock;
+	curr = xa_store(&group->pasid_array, pasid, handle, GFP_KERNEL);
+	if (curr) {
+		__iommu_remove_group_pasid(group, pasid, domain);
+		ret = xa_err(curr);
+	}

s/handle/xa_entry/

oops. will fix it.

Quoted 1 line

btw you may continue to use xa_insert() to save one line for xa_err()

got it.

Signature · 2 lines
Regards,
Yi Liu
#9 lore raw

Re: [PATCH 2/2] iommu: Swap the order of setting group->pasid_array and __iommu_set_group_pasid()

In reply to Yi Liu
Quoted 6 lines · Yi Liu

Following the current order of setting group->pasid_array and __iommu_set_group_pasid() in iommu_attach_device_pasid(), PRIs may be forwarded to the domain before the attach succeeds. If the attach failed in the end, the PRIs on the domain need to be flushed in the caller side. Caller can do it, but it can be avoided by swapping the order. This is more self-contained.

Can the caller do it? I thought only the driver could manage it?

Quoted 14 lines
drivers/iommu/iommu.c +1 -1
index 278c4eb8f225..b69bcf559839 100644
@@ -3376,7 +3376,7 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
 	struct iommu_group *group = dev->iommu_group;
 	struct group_device *device;
 	const struct iommu_ops *ops;
-	void *xa_entry;
+	void *curr, *xa_entry;
 	int ret;
 
 	if (!group)

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3404

Quoted 5 lines
drivers/iommu/iommu.c +2 -0
@@ -3400,6 +3400,16 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
 		}
 	}
 
+	curr = xa_load(&group->pasid_array, pasid);
+	if (curr) {

No need for curr

if (xa_load()) {

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3423

Quoted 11 lines
drivers/iommu/iommu.c +4 -3
@@ -3407,13 +3417,12 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
 		xa_entry = xa_tag_pointer(domain, IOMMU_PASID_ARRAY_DOMAIN);
 	}
 
-	ret = xa_insert(&group->pasid_array, pasid, xa_entry, GFP_KERNEL);
-	if (ret)
-		goto out_unlock;
+	curr = xa_store(&group->pasid_array, pasid, handle, GFP_KERNEL);
+	if (curr) {
+		__iommu_remove_group_pasid(group, pasid, domain);
+		ret = xa_err(curr);

curr could be a valid non-error entry, but that
would be a WARN_ON condition due to the above check. Like this:

      ret = xa_insert(&group->pasid_array, pasid, xa_entry,GFP_KERNEL);
      if (ret) {
		WARN_ON(ret == -EBUSY);
		goto out_remove;
      }

Jason

author
#10 lore raw

Re: [PATCH 2/2] iommu: Swap the order of setting group->pasid_array and __iommu_set_group_pasid()

Quoted 9 lines · Jason Gunthorpe
Quoted 6 lines · Yi Liu

Following the current order of setting group->pasid_array and __iommu_set_group_pasid() in iommu_attach_device_pasid(), PRIs may be forwarded to the domain before the attach succeeds. If the attach failed in the end, the PRIs on the domain need to be flushed in the caller side. Caller can do it, but it can be avoided by swapping the order. This is more self-contained.

Can the caller do it? I thought only the driver could manage it?

I think yes. Take iommufd as example, if the hwpt is fault-able, the PRIs would be forwarded to iommufd, and recorded in hwpt->fault->deliver list. This cannot be done in driver alone, iommufd needs to flush it. Otherwise, the PRIs would be kept until the hwpt is destroyed.

Quoted 17 lines
Quoted 17 lines
drivers/iommu/iommu.c +1 -1
index 278c4eb8f225..b69bcf559839 100644
@@ -3376,7 +3376,7 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
  	struct iommu_group *group = dev->iommu_group;
  	struct group_device *device;
  	const struct iommu_ops *ops;
-	void *xa_entry;
+	void *curr, *xa_entry;
  	int ret;
 
  	if (!group)
@@ -3400,6 +3400,16 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
  		}
  	}
 
Quoted 6 lines

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3404

Quoted 2 lines
drivers/iommu/iommu.c +2 -0
+	curr = xa_load(&group->pasid_array, pasid);
+	if (curr) {

No need for curr

if (xa_load()) {

got it.

Quoted 15 lines

drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3423

Quoted 11 lines
drivers/iommu/iommu.c +4 -3
@@ -3407,13 +3417,12 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,
  		xa_entry = xa_tag_pointer(domain, IOMMU_PASID_ARRAY_DOMAIN);
  	}
 
-	ret = xa_insert(&group->pasid_array, pasid, xa_entry, GFP_KERNEL);
-	if (ret)
-		goto out_unlock;
+	curr = xa_store(&group->pasid_array, pasid, handle, GFP_KERNEL);
+	if (curr) {
+		__iommu_remove_group_pasid(group, pasid, domain);
+		ret = xa_err(curr);

curr could be a valid non-error entry, but that
would be a WARN_ON condition due to the above check. Like this:

Quoted 5 lines
       ret = xa_insert(&group->pasid_array, pasid, xa_entry,GFP_KERNEL);
       if (ret) {
		WARN_ON(ret == -EBUSY);
		goto out_remove;
       }

yes. it should be a warn_on after the above check.

Signature · 2 lines
Regards,
Yi Liu