Two enhancements to iommu_attach_device_pasid()
cover Two enhancements to iommu_attach_device_pasid()
It's spotted during the review process of iommufd pasid series [1]. The first one is about storing entry to group->pasid_array, and the second one is to avoid the potential PRIs anchored on the failed domain. Details can be found in the individial patch.
[1] https://lore.kernel.org/linux-iommu/[email protected]/
Yi Liu (2):
iommu: Store either domain or handle in group->pasid_array
iommu: Swap the order of setting group->pasid_array and
__iommu_set_group_pasid()
drivers/iommu/iommu.c | 40 ++++++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 10 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/2 iommu: Store either domain or handle in group->pasid_array
iommu_attach_device_pasid() only stores handle to group->pasid_array when there is a valid handle input. However, it makes the iommu_attach_device_pasid() unable to detect if the pasid has been attached or not. To be complete, let the iommu_attach_device_pasid() store the domain to group->pasid_array if no valid handle.
--- drivers/iommu/iommu.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-)
drivers/iommu/iommu.c +17 -6
@@ -45,6 +45,8 @@ static unsigned int iommu_def_domain_type __read_mostly;static bool iommu_dma_strict__read_mostly =IS_ENABLED (CONFIG_IOMMU_DEFAULT_DMA_STRICT );static u32 iommu_cmd_line__read_mostly ; +enum {IOMMU_PASID_ARRAY_DOMAIN ,IOMMU_PASID_ARRAY_HANDLE }; +struct iommu_group {struct kobject kobj;struct kobject *devices_kobj;
@@ -3374,6 +3376,7 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,struct iommu_group *group = dev->iommu_group;struct group_device *device;const struct iommu_ops *ops; +void *xa_entry;int ret;if (!group)
@@ -3397,10 +3400,14 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, } } -if (handle) +if (handle) { handle->domain = domain; + xa_entry =xa_tag_pointer (handle,IOMMU_PASID_ARRAY_HANDLE ); + }else { + xa_entry =xa_tag_pointer (domain,IOMMU_PASID_ARRAY_DOMAIN ); + } - ret =xa_insert (&group->pasid_array, pasid, handle,GFP_KERNEL ); + ret =xa_insert (&group->pasid_array, pasid, xa_entry,GFP_KERNEL );if (ret) goto out_unlock;
@@ -3480,13 +3487,17 @@ struct iommu_attach_handle * iommu_attach_handle_get(struct iommu_group *group, ioasid_t pasid, unsigned int type) { struct iommu_attach_handle *handle; +void *xa_entry;xa_lock (&group->pasid_array); - handle =xa_load (&group->pasid_array, pasid); -if (!handle) + xa_entry =xa_load (&group->pasid_array, pasid); +if (xa_pointer_tag (xa_entry) ==IOMMU_PASID_ARRAY_HANDLE ) { + handle =xa_untag_pointer (xa_entry); +if (type && handle->domain->type != type) + handle =ERR_PTR (-EBUSY ); + }else { handle =ERR_PTR (-ENOENT ); -else if (type && handle->domain->type != type) - handle =ERR_PTR (-EBUSY ); + }xa_unlock (&group->pasid_array);return handle;
Signature · 1 line
2.34.1
RE: [PATCH 1/2] iommu: Store either domain or handle in group->pasid_array
Quoted 11 lines
From: Yi Liu <[email protected]>
Sent: Monday, January 20, 2025 11:09 AM
iommu_attach_device_pasid()only stores handle togroup->pasid_arraywhen there is a valid handle input. However, it makes theiommu_attach_device_pasid()unable to detect if the pasid has been attached or not. To be complete, let theiommu_attach_device_pasid()store the domain togroup->pasid_arrayif no valid handle.Suggested-by Jason Gunthorpe <[email protected]>Signed-off-by Yi Liu <[email protected]>
Re: [PATCH 1/2] iommu: Store either domain or handle in group->pasid_array
In reply to Yi LiuQuoted 17 lines · Yi Liu
iommu_attach_device_pasid()only stores handle togroup->pasid_arraywhen there is a valid handle input. However, it makes theiommu_attach_device_pasid()unable to detect if the pasid has been attached or not. To be complete, let theiommu_attach_device_pasid()store the domain togroup->pasid_arrayif no valid handle.Suggested-by Jason Gunthorpe <[email protected]>Signed-off-by Yi Liu <[email protected]>--- drivers/iommu/iommu.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-)drivers/iommu/iommu.c +0
-0
drivers/iommu/iommu.c static unsigned int iommu_def_domain_type __read_mostly; · line 48
Quoted 4 lines
drivers/iommu/iommu.c +1
-0@@ -45,6 +45,8 @@ static unsigned int iommu_def_domain_type __read_mostly;static bool iommu_dma_strict__read_mostly =IS_ENABLED (CONFIG_IOMMU_DEFAULT_DMA_STRICT );static u32 iommu_cmd_line__read_mostly ; +enum {IOMMU_PASID_ARRAY_DOMAIN ,IOMMU_PASID_ARRAY_HANDLE };
Since xarray says:
* @tag: Tag value (0, 1 or 3).
I would initialize those enums and add a small comment
/* tags used with xa_tag_pointer() */
enum { IOMMU_PASID_ARRAY_DOMAIN = 0, IOMMU_PASID_ARRAY_HANDLE = 1 };
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3408
Quoted 11 lines
drivers/iommu/iommu.c +5
-1@@ -3397,10 +3400,14 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, } } -if (handle) +if (handle) { handle->domain = domain; + xa_entry =xa_tag_pointer (handle,IOMMU_PASID_ARRAY_HANDLE ); + }else { + xa_entry =xa_tag_pointer (domain,IOMMU_PASID_ARRAY_DOMAIN ); + }
What about iommu_attach_group_handle() ?
mutex_lock(&group->mutex); ret = xa_insert(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL); if (ret)
And iommu_replace_group_handle() ?
curr = xa_store(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL);
I think it should be consistent and it should call xa_tag_poiner() too (even though it implicitly always is since the domain is stored in the group)??
Jason
Re: [PATCH 1/2] iommu: Store either domain or handle in group->pasid_array
Quoted 17 lines · Jason Gunthorpe
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3408
Quoted 11 lines
drivers/iommu/iommu.c +5
-1@@ -3397,10 +3400,14 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, } } -if (handle) +if (handle) { handle->domain = domain; + xa_entry =xa_tag_pointer (handle,IOMMU_PASID_ARRAY_HANDLE ); + }else { + xa_entry =xa_tag_pointer (domain,IOMMU_PASID_ARRAY_DOMAIN ); + }What about
iommu_attach_group_handle()?mutex_lock(&group->mutex); ret = xa_insert(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL); if (ret)
Quoted 7 lines
And
iommu_replace_group_handle()?curr = xa_store(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL);I think it should be consistent and it should call
xa_tag_poiner()too (even though it implicitly always is since the domain is stored in the group)??
We may eventually consider retiring group->domain.
---
baolu
Re: [PATCH 1/2] iommu: Store either domain or handle in group->pasid_array
In reply to Jason GunthorpeQuoted 21 lines · Jason Gunthorpe
Quoted 20 lines · Yi Liu
iommu_attach_device_pasid()only stores handle togroup->pasid_arraywhen there is a valid handle input. However, it makes theiommu_attach_device_pasid()unable to detect if the pasid has been attached or not. To be complete, let theiommu_attach_device_pasid()store the domain togroup->pasid_arrayif no valid handle.Suggested-by Jason Gunthorpe <[email protected]>Signed-off-by Yi Liu <[email protected]>--- drivers/iommu/iommu.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-)drivers/iommu/iommu.c +0
-0@@ -45,6 +45,8 @@ static unsigned int iommu_def_domain_type __read_mostly;static bool iommu_dma_strict__read_mostly =IS_ENABLED (CONFIG_IOMMU_DEFAULT_DMA_STRICT );static u32 iommu_cmd_line__read_mostly ;
Quoted 7 lines
drivers/iommu/iommu.c static unsigned int iommu_def_domain_type __read_mostly; · line 48
Quoted 1 line
drivers/iommu/iommu.c +1
-0+enum {IOMMU_PASID_ARRAY_DOMAIN ,IOMMU_PASID_ARRAY_HANDLE };Since xarray says:
* @tag: Tag value (0, 1 or 3).
I would initialize those enums and add a small comment
yes.
Quoted 21 lines
/* tags used with
xa_tag_pointer()*/
enum {IOMMU_PASID_ARRAY_DOMAIN= 0,IOMMU_PASID_ARRAY_HANDLE= 1 };drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3408
Quoted 11 lines
drivers/iommu/iommu.c +5
-1@@ -3397,10 +3400,14 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, } } -if (handle) +if (handle) { handle->domain = domain; + xa_entry =xa_tag_pointer (handle,IOMMU_PASID_ARRAY_HANDLE ); + }else { + xa_entry =xa_tag_pointer (domain,IOMMU_PASID_ARRAY_DOMAIN ); + }What about
iommu_attach_group_handle()?mutex_lock(&group->mutex); ret = xa_insert(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL); if (ret)
Quoted 7 lines
And
iommu_replace_group_handle()?curr = xa_store(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL);I think it should be consistent and it should call
xa_tag_poiner()too (even though it implicitly always is since the domain is stored in the group)??
yes, it needs to be fixed as well. I asked a help from Baolu in prior email, but I can enlarge this series to cover both RID and PASID path. :)
Signature · 2 lines
Regards, Yi Liu
PATCH 2/2 iommu: Swap the order of setting group->pasid_array and __iommu_set_group_pasid()
In reply to Yi LiuFollowing the current order of setting group->pasid_array and __iommu_set_group_pasid() in iommu_attach_device_pasid(), PRIs may be forwarded to the domain before the attach succeeds. If the attach failed in the end, the PRIs on the domain need to be flushed in the caller side. Caller can do it, but it can be avoided by swapping the order. This is more self-contained.
--- drivers/iommu/iommu.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)
drivers/iommu/iommu.c +16 -7
@@ -3376,7 +3376,7 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,struct iommu_group *group = dev->iommu_group;struct group_device *device;const struct iommu_ops *ops; -void *xa_entry; +void *curr, *xa_entry;int ret;if (!group)
@@ -3400,6 +3400,16 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, } } + curr =xa_load (&group->pasid_array, pasid); +if (curr) { + ret = -EBUSY ; + goto out_unlock; + } + + ret =__iommu_set_group_pasid (domain, group, pasid); +if (ret) + goto out_unlock; +if (handle) { handle->domain = domain; xa_entry =xa_tag_pointer (handle,IOMMU_PASID_ARRAY_HANDLE );
@@ -3407,13 +3417,12 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, xa_entry =xa_tag_pointer (domain,IOMMU_PASID_ARRAY_DOMAIN ); } - ret =xa_insert (&group->pasid_array, pasid, xa_entry,GFP_KERNEL ); -if (ret) - goto out_unlock; + curr =xa_store (&group->pasid_array, pasid, handle,GFP_KERNEL ); +if (curr) { +__iommu_remove_group_pasid (group, pasid, domain); + ret =xa_err (curr); + } - ret =__iommu_set_group_pasid (domain, group, pasid); -if (ret) -xa_erase (&group->pasid_array, pasid); out_unlock:mutex_unlock (&group->mutex);return ret;
Signature · 1 line
2.34.1
RE: [PATCH 2/2] iommu: Swap the order of setting group->pasid_array and __iommu_set_group_pasid()
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3424
Quoted 12 lines
From: Yi Liu <[email protected]>
Sent: Monday, January 20, 2025 11:09 AMdrivers/iommu/iommu.c +5
-3- ret =xa_insert (&group->pasid_array, pasid, xa_entry,GFP_KERNEL ); -if (ret) - goto out_unlock; + curr =xa_store (&group->pasid_array, pasid, handle,GFP_KERNEL ); +if (curr) { +__iommu_remove_group_pasid (group, pasid, domain); + ret =xa_err (curr); + }
s/handle/xa_entry/
btw you may continue to use xa_insert() to save one line for xa_err()
Re: [PATCH 2/2] iommu: Swap the order of setting group->pasid_array and __iommu_set_group_pasid()
Quoted 14 lines · Kevin
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3424
Quoted 12 lines
From: Yi Liu <[email protected]>
Sent: Monday, January 20, 2025 11:09 AMdrivers/iommu/iommu.c +5
-3- ret =xa_insert (&group->pasid_array, pasid, xa_entry,GFP_KERNEL ); -if (ret) - goto out_unlock; + curr =xa_store (&group->pasid_array, pasid, handle,GFP_KERNEL ); +if (curr) { +__iommu_remove_group_pasid (group, pasid, domain); + ret =xa_err (curr); + }s/handle/xa_entry/
oops. will fix it.
Quoted 1 line
btw you may continue to use
xa_insert()to save one line forxa_err()
got it.
Signature · 2 lines
Regards, Yi Liu
Re: [PATCH 2/2] iommu: Swap the order of setting group->pasid_array and __iommu_set_group_pasid()
In reply to Yi LiuQuoted 6 lines · Yi Liu
Following the current order of setting
group->pasid_arrayand__iommu_set_group_pasid()iniommu_attach_device_pasid(), PRIs may be forwarded to the domain before the attach succeeds. If the attach failed in the end, the PRIs on the domain need to be flushed in the caller side. Caller can do it, but it can be avoided by swapping the order. This is more self-contained.
Can the caller do it? I thought only the driver could manage it?
Quoted 14 lines
drivers/iommu/iommu.c +1
-1@@ -3376,7 +3376,7 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,struct iommu_group *group = dev->iommu_group;struct group_device *device;const struct iommu_ops *ops; -void *xa_entry; +void *curr, *xa_entry;int ret;if (!group)
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3404
Quoted 5 lines
drivers/iommu/iommu.c +2
-0@@ -3400,6 +3400,16 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, } } + curr =xa_load (&group->pasid_array, pasid); +if (curr) {
No need for curr
if (xa_load()) {
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3423
Quoted 11 lines
drivers/iommu/iommu.c +4
-3@@ -3407,13 +3417,12 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, xa_entry =xa_tag_pointer (domain,IOMMU_PASID_ARRAY_DOMAIN ); } - ret =xa_insert (&group->pasid_array, pasid, xa_entry,GFP_KERNEL ); -if (ret) - goto out_unlock; + curr =xa_store (&group->pasid_array, pasid, handle,GFP_KERNEL ); +if (curr) { +__iommu_remove_group_pasid (group, pasid, domain); + ret =xa_err (curr);
curr could be a valid non-error entry, but that
would be a WARN_ON condition due to the above check. Like this:
ret = xa_insert(&group->pasid_array, pasid, xa_entry,GFP_KERNEL);
if (ret) {
WARN_ON(ret == -EBUSY);
goto out_remove;
}Jason
Re: [PATCH 2/2] iommu: Swap the order of setting group->pasid_array and __iommu_set_group_pasid()
Quoted 9 lines · Jason Gunthorpe
Quoted 6 lines · Yi Liu
Following the current order of setting
group->pasid_arrayand__iommu_set_group_pasid()iniommu_attach_device_pasid(), PRIs may be forwarded to the domain before the attach succeeds. If the attach failed in the end, the PRIs on the domain need to be flushed in the caller side. Caller can do it, but it can be avoided by swapping the order. This is more self-contained.Can the caller do it? I thought only the driver could manage it?
I think yes. Take iommufd as example, if the hwpt is fault-able, the PRIs would be forwarded to iommufd, and recorded in hwpt->fault->deliver list. This cannot be done in driver alone, iommufd needs to flush it. Otherwise, the PRIs would be kept until the hwpt is destroyed.
Quoted 17 lines
Quoted 17 lines
drivers/iommu/iommu.c +1
-1@@ -3376,7 +3376,7 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,struct iommu_group *group = dev->iommu_group;struct group_device *device;const struct iommu_ops *ops; -void *xa_entry; +void *curr, *xa_entry;int ret;if (!group)@@ -3400,6 +3400,16 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, } }
Quoted 6 lines
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3404
Quoted 2 lines
drivers/iommu/iommu.c +2
-0+ curr =xa_load (&group->pasid_array, pasid); +if (curr) {No need for curr
if (
xa_load()) {
got it.
Quoted 15 lines
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3423
Quoted 11 lines
drivers/iommu/iommu.c +4
-3@@ -3407,13 +3417,12 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, xa_entry =xa_tag_pointer (domain,IOMMU_PASID_ARRAY_DOMAIN ); } - ret =xa_insert (&group->pasid_array, pasid, xa_entry,GFP_KERNEL ); -if (ret) - goto out_unlock; + curr =xa_store (&group->pasid_array, pasid, handle,GFP_KERNEL ); +if (curr) { +__iommu_remove_group_pasid (group, pasid, domain); + ret =xa_err (curr);curr could be a valid non-error entry, but that
would be aWARN_ONcondition due to the above check. Like this:
Quoted 5 lines
ret = xa_insert(&group->pasid_array, pasid, xa_entry,GFP_KERNEL);
if (ret) {
WARN_ON(ret == -EBUSY);
goto out_remove;
}yes. it should be a warn_on after the above check.
Signature · 2 lines
Regards, Yi Liu