iommu: Make pasid array per device
v2 cover iommu: Make pasid array per device
The PCI PASID enabling interface guarantees that the address space used by each PASID is unique. This is achieved by checking that the PCI ACS path is enabled for the device. If the path is not enabled, then the PASID feature cannot be used.
if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF))
return -EINVAL;
The PASID array is not an attribute of the IOMMU group. It is more natural to store the PASID array in the per-device IOMMU data. This makes the code clearer and easier to understand.
Please help review and suggest.
Change log:
v2:
- Add an explict check for single-device group in the domain attaching device pasid interface. (Jason)
- Make assert_pasid_dma_ownership() returns true or false. Refactor the code in a way that does not change its exsiting behavior. (Kevin)
v1: https://lore.kernel.org/linux-iommu/[email protected]/
Lu Baolu (3):
iommu: Make single-device group for PASID explicit
iommu: Consolidate pasid dma ownership check
iommu: Move pasid array from group to device
include/linux/iommu.h | 2 + drivers/iommu/iommu.c | 102 +++++++++++++++++++----------------------- 2 files changed, 49 insertions(+), 55 deletions(-)
Signature · 1 line
2.34.1
PATCH v2 1/3 iommu: Make single-device group for PASID explicit
The PASID interfaces have always supported only single-device groups. This was first introduced in commit 26b25a2b98e45 ("iommu: Bind process address spaces to devices"), and has been kept consistent in subsequent commits.
However, the core code doesn't explicitly check for this requirement after commit 201007ef707a8 ("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit.
Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.
--- drivers/iommu/iommu.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/iommu/iommu.c +5 -0
@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct iommu_domain *domain,return -ENODEV ;mutex_lock (&group->mutex); +if (list_count_nodes (&group->devices) !=1 ) { + ret = -EINVAL ; + goto out_unlock; + } + curr =xa_cmpxchg (&group->pasid_array, pasid,NULL , domain,GFP_KERNEL );if (curr) { ret =xa_err (curr) ? : -EBUSY ;
Signature · 1 line
2.34.1
RE: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit
Quoted 25 lines
From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AMThe PASID interfaces have always supported only single-device groups. This was first introduced in commit
26b25a2b98e45("iommu: Bind process address spaces to devices"), and has been kept consistent in subsequent commits.However, the core code doesn't explicitly check for this requirement after commit
201007ef707a8("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit.Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.
Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/iommu.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/iommu/iommu.c +0
-0
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct · line 3416
Quoted 9 lines
drivers/iommu/iommu.c +5
-0@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, return -ENODEV ;mutex_lock (&group->mutex); +if (list_count_nodes (&group->devices) !=1 ) { + ret = -EINVAL ; + goto out_unlock; + } +
I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the check alone at attach time doesn't mean this group won't be expanded to have multiple devices later.
Re: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit
Quoted 28 lines · Kevin
Quoted 28 lines
From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AMThe PASID interfaces have always supported only single-device groups. This was first introduced in commit
26b25a2b98e45("iommu: Bind process address spaces to devices"), and has been kept consistent in subsequent commits.However, the core code doesn't explicitly check for this requirement after commit
201007ef707a8("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit.Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.
Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/iommu.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/iommu/iommu.c +0
-0@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, return -ENODEV ;
Quoted 9 lines
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3415
Quoted 6 lines
drivers/iommu/iommu.c +5
-0mutex_lock (&group->mutex); +if (list_count_nodes (&group->devices) !=1 ) { + ret = -EINVAL ; + goto out_unlock; + } +I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the
This has been guaranteed by pci_enable_pasid():
if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF))
return -EINVAL;
Quoted 2 lines
check alone at attach time doesn't mean this group won't be expanded to have multiple devices later.
Best regards,
baolu
RE: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit
Quoted 43 lines
From: Baolu Lu <[email protected]>
Sent: Monday, August 21, 2023 1:45 PMQuoted 5 lines · Kevin
Quoted 5 lines
From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AMThe PASID interfaces have always supported only single-device groups. This was first introduced in commit
26b25a2b98e45("iommu: Bindprocess
Quoted 23 lines
Quoted 23 lines
address spaces to devices"), and has been kept consistent in subsequent commits.
However, the core code doesn't explicitly check for this requirement after commit
201007ef707a8("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit.Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.
Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/iommu.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/iommu/iommu.c +0
-0@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, return -ENODEV ;Quoted 9 lines
drivers/iommu/iommu.c int iommu_attach_device_pasid(struct iommu_domain *domain, · line 3415
Quoted 6 lines
drivers/iommu/iommu.c +5
-0mutex_lock (&group->mutex); +if (list_count_nodes (&group->devices) !=1 ) { + ret = -EINVAL ; + goto out_unlock; + } +I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the
Quoted 5 lines
This has been guaranteed by
pci_enable_pasid():if (!pci_acs_path_enabled(pdev, NULL,
PCI_ACS_RR|PCI_ACS_UF))
return-EINVAL;
well since you are adding generic core check then it's not good to rely on the fact of a specific bus...
Re: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit
Quoted 43 lines · Kevin
Quoted 43 lines
From: Baolu Lu <[email protected]>
Sent: Monday, August 21, 2023 1:45 PMQuoted 5 lines · Kevin
Quoted 5 lines
From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AMThe PASID interfaces have always supported only single-device groups. This was first introduced in commit
26b25a2b98e45("iommu: Bindprocess
Quoted 32 lines
Quoted 20 lines
address spaces to devices"), and has been kept consistent in subsequent commits.
However, the core code doesn't explicitly check for this requirement after commit
201007ef707a8("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit.Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.
Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/iommu.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/iommu/iommu.c +0
-0drivers/iommu/iommu.c int iommu_attach_device_pasid(struct · line 3416
Quoted 9 lines
drivers/iommu/iommu.c +5
-0@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, return -ENODEV ;mutex_lock (&group->mutex); +if (list_count_nodes (&group->devices) !=1 ) { + ret = -EINVAL ; + goto out_unlock; + } +I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the
Quoted 8 lines
Quoted 5 lines
This has been guaranteed by
pci_enable_pasid():if (!pci_acs_path_enabled(pdev, NULL,
PCI_ACS_RR|PCI_ACS_UF))
return-EINVAL;well since you are adding generic core check then it's not good to rely on the fact of a specific bus...
We attempted to do this in the patch linked below.
https://lore.kernel.org/linux-iommu/[email protected]/
After long discussion, we decided to move it to the pci_enable_pasid() interface. The non-static single device group is only relevant to PCI fabrics that support hot-plugging without ACS support on the upstream path.
Best regards,
baolu
RE: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit
Quoted 60 lines
From: Baolu Lu <[email protected]>
Sent: Tuesday, August 22, 2023 2:37 PMQuoted 11 lines · Kevin
Quoted 11 lines
From: Baolu Lu <[email protected]>
Sent: Monday, August 21, 2023 1:45 PMQuoted 5 lines · Kevin
Quoted 5 lines
From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AMThe PASID interfaces have always supported only single-device groups. This was first introduced in commit
26b25a2b98e45("iommu: Bindprocess
Quoted 1 line
Quoted 1 line
address spaces to devices"), and has been kept consistent in
subsequent
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
commits.
However, the core code doesn't explicitly check for this requirement after commit
201007ef707a8("PCI: Enable PASID only when ACS RR &UF
Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
Quoted 15 lines
enabled on upstream path"), which made this requirement implicit.
Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.
Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/iommu.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/iommu/iommu.c +0
-0drivers/iommu/iommu.c int iommu_attach_device_pasid(struct · line 3416
Quoted 9 lines
drivers/iommu/iommu.c +5
-0@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, return -ENODEV ;mutex_lock (&group->mutex); +if (list_count_nodes (&group->devices) !=1 ) { + ret = -EINVAL ; + goto out_unlock; + } +I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the
Quoted 8 lines
Quoted 5 lines
This has been guaranteed by
pci_enable_pasid():if (!pci_acs_path_enabled(pdev, NULL,
PCI_ACS_RR|PCI_ACS_UF))
return-EINVAL;well since you are adding generic core check then it's not good to rely on the fact of a specific bus...
We attempted to do this in the patch linked below.
Quoted 8 lines
https://lore.kernel.org/linux-iommu/20220705050710.2887204-5-
[email protected]/After long discussion, we decided to move it to the
pci_enable_pasid()interface. The non-static single device group is only relevant to PCI fabrics that support hot-plugging without ACS support on the upstream path.
If that's the case better add a comment to include this fact. So another one looking at this code won't fall into the same puzzle wondering what about a group becoming non-singleton after above check. 😊
Re: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit
Quoted 69 lines · Kevin
Quoted 60 lines
From: Baolu Lu <[email protected]>
Sent: Tuesday, August 22, 2023 2:37 PMQuoted 11 lines · Kevin
Quoted 11 lines
From: Baolu Lu <[email protected]>
Sent: Monday, August 21, 2023 1:45 PMQuoted 5 lines · Kevin
Quoted 5 lines
From: Lu Baolu <[email protected]>
Sent: Monday, August 14, 2023 9:18 AMThe PASID interfaces have always supported only single-device groups. This was first introduced in commit
26b25a2b98e45("iommu: Bindprocess
Quoted 1 line
Quoted 1 line
address spaces to devices"), and has been kept consistent in
subsequent
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
commits.
However, the core code doesn't explicitly check for this requirement after commit
201007ef707a8("PCI: Enable PASID only when ACS RR &UF
Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
Quoted 15 lines
enabled on upstream path"), which made this requirement implicit.
Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups.
Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/iommu.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/iommu/iommu.c +0
-0drivers/iommu/iommu.c int iommu_attach_device_pasid(struct · line 3416
Quoted 9 lines
drivers/iommu/iommu.c +5
-0@@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, return -ENODEV ;mutex_lock (&group->mutex); +if (list_count_nodes (&group->devices) !=1 ) { + ret = -EINVAL ; + goto out_unlock; + } +I wonder whether we should also block adding new device to this group once the single-device has pasid enabled. Otherwise the
Quoted 8 lines
Quoted 5 lines
This has been guaranteed by
pci_enable_pasid():if (!pci_acs_path_enabled(pdev, NULL,
PCI_ACS_RR|PCI_ACS_UF))
return-EINVAL;well since you are adding generic core check then it's not good to rely on the fact of a specific bus...
We attempted to do this in the patch linked below.
Quoted 8 lines
https://lore.kernel.org/linux-iommu/20220705050710.2887204-5-
[email protected]/After long discussion, we decided to move it to the
pci_enable_pasid()interface. The non-static single device group is only relevant to PCI fabrics that support hot-plugging without ACS support on the upstream path.
Quoted 4 lines
If that's the case better add a comment to include this fact. So another one looking at this code won't fall into the same puzzle wondering what about a group becoming non-singleton after above check. 😊
Yeah, fair enough.
Best regards,
baolu
PATCH v2 2/3 iommu: Consolidate pasid dma ownership check
In reply to Lu BaoluWhen switching device DMA ownership, it is required that all the device's pasid DMA be disabled. This is done by checking if the pasid array of the group is empty. Consolidate all the open code into a single helper. No intentional functionality change.
--- drivers/iommu/iommu.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-)
drivers/iommu/iommu.c +18 -5
@@ -3127,6 +3127,19 @@ static bool iommu_is_default_domain(struct iommu_group *group)return false; } +/* +* Assert no PASID DMA when claiming or releasing group's DMA ownership. +* The device pasid interfaces are only for device drivers that have +* claimed the DMA ownership. Return true if no pasid DMA setup, otherwise +* return false with a WARN(). +*/ +static bool assert_pasid_dma_ownership (struct iommu_group *group) +{ +lockdep_assert_held (&group->mutex); + +return !WARN_ON (!xa_empty (&group->pasid_array)); +} +/** * iommu_device_use_default_domain() - Device driver wants to handle device * DMA through the kernel DMA API.
@@ -3147,7 +3160,7 @@ int iommu_device_use_default_domain(struct device *dev)mutex_lock (&group->mutex);if (group->owner_cnt) {if (group->owner || !iommu_is_default_domain (group) || - !xa_empty (&group->pasid_array)) { + !assert_pasid_dma_ownership (group)) { ret = -EBUSY ; goto unlock_out; }
@@ -3177,7 +3190,7 @@ void iommu_device_unuse_default_domain(struct device *dev)return ;mutex_lock (&group->mutex); -if (!WARN_ON (!group->owner_cnt || !xa_empty (&group->pasid_array))) +if (!WARN_ON (!group->owner_cnt) &&assert_pasid_dma_ownership (group)) group->owner_cnt--;mutex_unlock (&group->mutex);
@@ -3211,7 +3224,7 @@ static int __iommu_take_dma_ownership(struct iommu_group *group, void *owner)int ret;if ((group->domain && group->domain != group->default_domain) || - !xa_empty (&group->pasid_array)) + !assert_pasid_dma_ownership (group))return -EBUSY ; ret =__iommu_group_alloc_blocking_domain (group);
@@ -3296,8 +3309,8 @@ EXPORT_SYMBOL_GPL(iommu_device_claim_dma_owner);static void __iommu_release_dma_ownership (struct iommu_group *group) { -if (WARN_ON (!group->owner_cnt || !group->owner || - !xa_empty (&group->pasid_array))) +if (WARN_ON (!group->owner_cnt || !group->owner) || + !assert_pasid_dma_ownership (group))return ; group->owner_cnt =0 ;
Signature · 1 line
2.34.1
Re: [PATCH v2 2/3] iommu: Consolidate pasid dma ownership check
Quoted 15 lines · Lu Baolu
When switching device DMA ownership, it is required that all the device's pasid DMA be disabled. This is done by checking if the pasid array of the group is empty. Consolidate all the open code into a single helper. No intentional functionality change.
Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/iommu.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-)drivers/iommu/iommu.c +0
-0
drivers/iommu/iommu.c static bool iommu_is_default_domain(struct iommu_group *group) · line 3131
Quoted 5 lines
drivers/iommu/iommu.c +2
-0@@ -3127,6 +3127,19 @@ static bool iommu_is_default_domain(struct iommu_group *group)return false; } +/* +* Assert no PASID DMA when claiming or releasing group's DMA ownership.
^ |...
drivers/iommu/iommu.c static bool iommu_is_default_domain(struct iommu_group *group) · line 3136
Quoted 5 lines
drivers/iommu/iommu.c +5
-0+* The device pasid interfaces are only for device drivers that have +* claimed the DMA ownership. Return true if no pasid DMA setup, otherwise +* return false with a WARN(). +*/ +static bool assert_pasid_dma_ownership (struct iommu_group *group)
... should it be assert_no_pasid_dma_ownership?
Nic
Re: [PATCH v2 2/3] iommu: Consolidate pasid dma ownership check
Quoted 25 lines · Nicolin Chen
Quoted 15 lines · Lu Baolu
When switching device DMA ownership, it is required that all the device's pasid DMA be disabled. This is done by checking if the pasid array of the group is empty. Consolidate all the open code into a single helper. No intentional functionality change.
Signed-off-by Lu Baolu<[email protected]>--- drivers/iommu/iommu.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-)drivers/iommu/iommu.c +0
-0drivers/iommu/iommu.c static bool iommu_is_default_domain(struct iommu_group *group) · line 3131
Quoted 5 lines
drivers/iommu/iommu.c +2
-0@@ -3127,6 +3127,19 @@ static bool iommu_is_default_domain(struct iommu_group *group)return false; } +/* +* Assert no PASID DMA when claiming or releasing group's DMA ownership. ^ |...
Quoted 6 lines
drivers/iommu/iommu.c static bool iommu_is_default_domain(struct iommu_group *group) · line 3136
Quoted 5 lines
drivers/iommu/iommu.c +5
-0+* The device pasid interfaces are only for device drivers that have +* claimed the DMA ownership. Return true if no pasid DMA setup, otherwise +* return false with a WARN(). +*/ +static bool assert_pasid_dma_ownership (struct iommu_group *group)... should it be
assert_no_pasid_dma_ownership?
Fair enough. Or, perhaps just assert_no_pasid_dma()?
Best regards,
baolu
PATCH v2 3/3 iommu: Move pasid array from group to device
In reply to Lu BaoluThe PASID (Process Address Space ID) feature is a device feature that allows a device driver to manage the PASID value and attach or detach its domain to the pasid. The pasid array, which is used to store the domain of each pasid, is currently stored in iommu group struct, but it would be more natural to move it to the device so that the device drivers don't need to understand the internal iommu group concept.
--- include/linux/iommu.h | 2 ++ drivers/iommu/iommu.c | 80 +++++++++++++++---------------------------- 2 files changed, 29 insertions(+), 53 deletions(-)
include/linux/iommu.h +2 -0
@@ -412,6 +412,7 @@ struct iommu_fault_param {* @iopf_param: I/O Page Fault queue and data * @fwspec: IOMMU fwspec data * @iommu_dev: IOMMU device this device is linked to +* @pasid_array: pasid-indexed array of domains attached to pasid * @priv: IOMMU Driver private data * @max_pasids: number of PASIDs this device can consume * @attach_deferred: the dma domain attachment is deferred
@@ -427,6 +428,7 @@ struct dev_iommu {struct iopf_device_param *iopf_param;struct iommu_fwspec *fwspec;struct iommu_device *iommu_dev; +struct xarray pasid_array;void *priv;u32 max_pasids; u32 attach_deferred:1;
drivers/iommu/iommu.c +27 -53
149 lines not on this page · raw message
Signature · 1 line
2.34.1