Define EINVAL as device/domain incompatibility
Subsystems (13)
v3 cover Define EINVAL as device/domain incompatibility
This series is to replace the previous EMEDIUMTYPE patch in a VFIO series:
https://lore.kernel.org/kvm/[email protected]/
The purpose is to regulate all existing ->attach_dev callback functions to use EINVAL exclusively for an incompatibility error between a device and a domain. This allows VFIO and IOMMUFD to detect such a soft error, and then try a different domain with the same device.
Among all the patches, the first two are preparatory changes. And then one patch to update kdocs and another three patches for the enforcement effort.
Although it might be ideal to merge the previous VFIO series together with this series, given the number of new changes, the review in the IOMMU list might need a couple of rounds to finalize. Also, considering that v6.0 is at rc5 now, perhaps we could merge this IOMMU series and the VFIO one in different cycles to avoid merge conflicts. If there's less concern for it, I can respin the finalized version of this series with the previous VFIO one to merge together into the VFIO tree.
This series is also available on Github:
https://github.com/nicolinc/iommufd/commits/iommu_attach_dev-v3
Changelog
v3:
* Added "Reviewed-by" from Vasant to the AMD patch
* Dropped all unnecessary errno enforcement patches
* Updated kdocs and brought back the kdocs for the helpers
* Added a separate patch to propagate "ret" for potential EINVALs
* Converted to ENODEV those existing EINVAL places that are device-specific
v2: https://lore.kernel.org/linux-iommu/[email protected]/
* Fixed kdocs format
* Grouped with the virtio patch from Jean (with a small change)
* Separated previous ENODEV and EINVAL changes to per-driver ones
* Redone some of the changes to have explicit return values in the ->attach_dev() callback functions or their direct sub-functions.
v1: https://lore.kernel.org/linux-iommu/[email protected]/
Thanks!
Nicolin Chen (6):
iommu/msm: Add missing __disable_clocks calls iommu/amd: Drop unnecessary checks in amd_iommu_attach_device()
iommu: Add return value rules to attach_dev op and APIs
iommu: Regulate EINVAL in ->attach_dev callback functions
iommu: Use EINVAL for incompatible device/domain in ->attach_dev
iommu: Propagate ret for a potential soft failure EINVAL
drivers/iommu/amd/iommu.c | 12 ++--------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 11 +--------- drivers/iommu/arm/arm-smmu/arm-smmu.c | 3 --- drivers/iommu/arm/arm-smmu/qcom_iommu.c | 7 +----- drivers/iommu/fsl_pamu.c | 2 +- drivers/iommu/fsl_pamu_domain.c | 4 ++-- drivers/iommu/intel/iommu.c | 10 +++------ drivers/iommu/intel/pasid.c | 6 ++++-- drivers/iommu/iommu.c | 24 +++++++++++++++++++++ drivers/iommu/ipmmu-vmsa.c | 2 -- drivers/iommu/msm_iommu.c | 2 ++ drivers/iommu/mtk_iommu.c | 4 ++-- drivers/iommu/omap-iommu.c | 6 +++--- drivers/iommu/sprd-iommu.c | 4 +--- drivers/iommu/tegra-gart.c | 2 +- drivers/iommu/virtio-iommu.c | 5 ++--- include/linux/iommu.h | 12 +++++++++++ 17 files changed, 61 insertions(+), 55 deletions(-)
Signature · 1 line
2.17.1
PATCH v3 1/6 iommu/msm: Add missing __disable_clocks calls
The clock is not symmetrically disabled in the error-out routines.
--- drivers/iommu/msm_iommu.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/iommu/msm_iommu.c +2 -0
@@ -418,6 +418,7 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev)list_for_each_entry (master, &iommu->ctx_list, list) {if (master->num) {dev_err (dev,"domain already attached" ); +__disable_clocks (iommu); ret = -EEXIST ; goto fail; }
@@ -425,6 +426,7 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev)msm_iommu_alloc_ctx (iommu->context_map,0 , iommu->ncb);if (IS_ERR_VALUE (master->num)) { +__disable_clocks (iommu); ret = -ENODEV ; goto fail; }
Signature · 1 line
2.17.1
RE: [PATCH v3 1/6] iommu/msm: Add missing __disable_clocks calls
Quoted 31 lines
From: Nicolin Chen <[email protected]>
Sent: Thursday, September 15, 2022 3:56 PMThe clock is not symmetrically disabled in the error-out routines.
Fixes 109bd48ea2e1 ("iommu/msm: Add DT adaptation")Cc Sricharan R <[email protected]>Cc Andy Gross <[email protected]>Cc Bjorn Andersson <[email protected]>Cc Konrad Dybcio <[email protected]>Signed-off-by Nicolin Chen <[email protected]>--- drivers/iommu/msm_iommu.c | 2 ++ 1 file changed, 2 insertions(+)drivers/iommu/msm_iommu.c +1
-0@@ -418,6 +418,7 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) list_for_each_entry(master, &iommu->ctx_list, list) {if (master->num) {dev_err (dev,"domain already attached"); +__disable_clocks (iommu); ret = -EEXIST ;drivers/iommu/msm_iommu.c +0
-0goto fail; }
drivers/iommu/msm_iommu.c static int msm_iommu_attach_dev(struct · line 431
Quoted 6 lines
drivers/iommu/msm_iommu.c +1
-0@@ -425,6 +426,7 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) msm_iommu_alloc_ctx(iommu- >context_map, 0, iommu->ncb);if (IS_ERR_VALUE (master->num)) { +__disable_clocks (iommu);
also need to free_ctx() for already walked nodes.
btw it's a bit weird that although here is coded based on a list in reality there is at most one node per list. According to insert_iommu_master() a master object is allocated and inserted to the ctx_list only if the ctx_list is currently empty...
Quoted 5 lines
drivers/iommu/msm_iommu.c +0
-0ret = -ENODEV ;drivers/iommu/msm_iommu.c +0
-1goto fail; } --2.17.1
Re: [PATCH v3 1/6] iommu/msm: Add missing __disable_clocks calls
Quoted 36 lines · Kevin
External email: Use caution opening links or attachments
Quoted 33 lines
From: Nicolin Chen <[email protected]>
Sent: Thursday, September 15, 2022 3:56 PMThe clock is not symmetrically disabled in the error-out routines.
Fixes 109bd48ea2e1 ("iommu/msm: Add DT adaptation")Cc Sricharan R <[email protected]>Cc Andy Gross <[email protected]>Cc Bjorn Andersson <[email protected]>Cc Konrad Dybcio <[email protected]>Signed-off-by Nicolin Chen <[email protected]>--- drivers/iommu/msm_iommu.c | 2 ++ 1 file changed, 2 insertions(+)drivers/iommu/msm_iommu.c +1
-0@@ -418,6 +418,7 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) list_for_each_entry(master, &iommu->ctx_list, list) {if (master->num) {dev_err (dev,"domain already attached"); +__disable_clocks (iommu); ret = -EEXIST ;goto fail; }drivers/iommu/msm_iommu.c +0
-0@@ -425,6 +426,7 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) msm_iommu_alloc_ctx(iommu-
Quoted 6 lines
drivers/iommu/msm_iommu.c static int msm_iommu_attach_dev(struct · line 431
Quoted 4 lines
drivers/iommu/msm_iommu.c +1
-0>context_map, 0, iommu->ncb);if (IS_ERR_VALUE (master->num)) { +__disable_clocks (iommu);also need to
free_ctx()for already walked nodes.
Oooo...yes. Probably could reuse the detach() -- [1].
Quoted 4 lines
btw it's a bit weird that although here is coded based on a list in reality there is at most one node per list. According to
insert_iommu_master()a master object is allocated and inserted to thectx_listonly if thectx_listis currently empty...
Yea. The insert_iommu_master() indicates that there would be only one master on a cts_list, while the rest part of the driver tries to take care of a potential multi-master per cts_list case, which practically won't happen by looking at the DT file. But the driver existed for the legacy platform data configuration too, so I don't intend to change too much...
Thanks!
Nic
[1]
drivers/iommu/msm_iommu.c +32 -28
@@ -394,6 +394,33 @@ static struct iommu_device *msm_iommu_probe_device(struct device *dev)return &iommu->iommu; } +static void msm_iommu_detach_dev (struct iommu_domain *domain, +struct device *dev) +{ +struct msm_priv *priv =to_msm_priv (domain); +unsigned long flags; +struct msm_iommu_dev *iommu; +struct msm_iommu_ctx_dev *master; +int ret; + +free_io_pgtable_ops (priv->iop); + +spin_lock_irqsave (&msm_iommu_lock, flags); +list_for_each_entry (iommu, &priv->list_attached, dom_node) { + ret =__enable_clocks (iommu); +if (ret) + goto fail; + +list_for_each_entry (master, &iommu->ctx_list, list) { +msm_iommu_free_ctx (iommu->context_map, master->num); +__reset_context (iommu->base, master->num); + } +__disable_clocks (iommu); + } +fail: +spin_unlock_irqrestore (&msm_iommu_lock, flags); +} +static int msm_iommu_attach_dev (struct iommu_domain *domain,struct device *dev) {int ret =0 ;
@@ -418,13 +445,15 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev)list_for_each_entry (master, &iommu->ctx_list, list) {if (master->num) {dev_err (dev,"domain already attached" ); +__disable_clocks (iommu); ret = -EEXIST ; goto fail; } master->num =msm_iommu_alloc_ctx (iommu->context_map,0 , iommu->ncb); -if (IS_ERR_VALUE (master->num)) { +if (master->num <0 ) { +__disable_clocks (iommu); ret = -ENODEV ; goto fail; }
@@ -439,37 +468,12 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) fail:spin_unlock_irqrestore (&msm_iommu_lock, flags); +if (ret) +msm_iommu_detach_dev (domain, dev);return ret; } -static void msm_iommu_detach_dev (struct iommu_domain *domain, -struct device *dev) -{ -struct msm_priv *priv =to_msm_priv (domain); -unsigned long flags; -struct msm_iommu_dev *iommu; -struct msm_iommu_ctx_dev *master; -int ret; - -free_io_pgtable_ops (priv->iop); - -spin_lock_irqsave (&msm_iommu_lock, flags); -list_for_each_entry (iommu, &priv->list_attached, dom_node) { - ret =__enable_clocks (iommu); -if (ret) - goto fail; - -list_for_each_entry (master, &iommu->ctx_list, list) { -msm_iommu_free_ctx (iommu->context_map, master->num); -__reset_context (iommu->base, master->num); - } -__disable_clocks (iommu); - } -fail: -spin_unlock_irqrestore (&msm_iommu_lock, flags); -} -static int msm_iommu_map (struct iommu_domain *domain,unsigned long iova,phys_addr_t pa,size_t len,int prot,gfp_t gfp) {
RE: [PATCH v3 1/6] iommu/msm: Add missing __disable_clocks calls
Quoted 55 lines
From: Nicolin Chen <[email protected]>
Sent: Wednesday, September 21, 2022 2:47 AMQuoted 19 lines · Kevin
External email: Use caution opening links or attachments
Quoted 16 lines
From: Nicolin Chen <[email protected]>
Sent: Thursday, September 15, 2022 3:56 PMThe clock is not symmetrically disabled in the error-out routines.
Fixes 109bd48ea2e1 ("iommu/msm: Add DT adaptation")Cc Sricharan R <[email protected]>Cc Andy Gross <[email protected]>Cc Bjorn Andersson <[email protected]>Cc Konrad Dybcio <[email protected]>Signed-off-by Nicolin Chen <[email protected]>--- drivers/iommu/msm_iommu.c | 2 ++ 1 file changed, 2 insertions(+)Quoted 1 line
Review
Quoted 1 line
(quoted diff) +0
-0
b/drivers/iommu/msm_iommu.cQuoted 16 lines
Quoted 16 lines
index
6a24aa804ea3..a7d41ba4a47b100644drivers/iommu/msm_iommu.c +1
-0@@ -418,6 +418,7 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) list_for_each_entry(master, &iommu->ctx_list, list) {if (master->num) {dev_err (dev,"domain already attached"); +__disable_clocks (iommu); ret = -EEXIST ;goto fail; }drivers/iommu/msm_iommu.c +0
-0@@ -425,6 +426,7 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) msm_iommu_alloc_ctx(iommu-Quoted 6 lines
drivers/iommu/msm_iommu.c static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) · line 456
Quoted 4 lines
drivers/iommu/msm_iommu.c +1
-0>context_map, 0, iommu->ncb);if (IS_ERR_VALUE (master->num)) { +__disable_clocks (iommu);also need to
free_ctx()for already walked nodes.Oooo...yes. Probably could reuse the
detach()-- [1].Quoted 4 lines
btw it's a bit weird that although here is coded based on a list in reality there is at most one node per list. According to
insert_iommu_master()a master object is allocated and inserted to thectx_listonly if thectx_listis currently empty...
Quoted 6 lines
Yea. The
insert_iommu_master()indicates that there would be only one master on acts_list, while the rest part of the driver tries to take care of a potential multi-master percts_listcase, which practically won't happen by looking at the DT file. But the driver existed for the legacy platform data configuration too, so I don't intend to change too much...
it's also problematic that it assumes that the attached device must be the first master in the list. But anyway I agree to not change too much given that mess.
Quoted 109 lines
Thanks!
Nic[1]
drivers/iommu/msm_iommu.c +27
-0@@ -394,6 +394,33 @@ static struct iommu_device *msm_iommu_probe_device(struct device *dev) return &iommu->iommu; } +static void msm_iommu_detach_dev (struct iommu_domain *domain, +struct device *dev) +{ +struct msm_priv *priv =to_msm_priv (domain); +unsigned long flags; +struct msm_iommu_dev *iommu; +struct msm_iommu_ctx_dev *master; +int ret; + +free_io_pgtable_ops (priv->iop); + +spin_lock_irqsave (&msm_iommu_lock, flags); +list_for_each_entry (iommu, &priv->list_attached, dom_node) { + ret =__enable_clocks (iommu); +if (ret) + goto fail; + +list_for_each_entry (master, &iommu->ctx_list, list) { +msm_iommu_free_ctx (iommu->context_map, master->num); +__reset_context (iommu->base, master->num); + } +__disable_clocks (iommu); + } +fail: +spin_unlock_irqrestore (&msm_iommu_lock, flags); +} + static int msm_iommu_attach_dev(struct iommu_domain *domain, structdevice *dev)
{drivers/iommu/msm_iommu.c +3
-1int ret =0 ;@@ -418,13 +445,15 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) list_for_each_entry(master, &iommu->ctx_list, list) {if (master->num) {dev_err (dev,"domain already attached"); +__disable_clocks (iommu); ret = -EEXIST ; goto fail; } master->num =msm_iommu_alloc_ctx (iommu- >context_map,0 , iommu->ncb); -if (IS_ERR_VALUE (master->num)) { +if (master->num <0 ) { +__disable_clocks (iommu);drivers/iommu/msm_iommu.c +2
-27ret = -ENODEV ; goto fail; }@@ -439,37 +468,12 @@ static int msm_iommu_attach_dev(struct iommu_domain *domain, struct device *dev) fail: spin_unlock_irqrestore(&msm_iommu_lock, flags); +if (ret) +msm_iommu_detach_dev (domain, dev);return ret; } -static void msm_iommu_detach_dev (struct iommu_domain *domain, -struct device *dev) -{ -struct msm_priv *priv =to_msm_priv (domain); -unsigned long flags; -struct msm_iommu_dev *iommu; -struct msm_iommu_ctx_dev *master; -int ret; - -free_io_pgtable_ops (priv->iop); - -spin_lock_irqsave (&msm_iommu_lock, flags); -list_for_each_entry (iommu, &priv->list_attached, dom_node) { - ret =__enable_clocks (iommu); -if (ret) - goto fail; - -list_for_each_entry (master, &iommu->ctx_list, list) { -msm_iommu_free_ctx (iommu->context_map, master->num); -__reset_context (iommu->base, master->num); - } -__disable_clocks (iommu); - } -fail: -spin_unlock_irqrestore (&msm_iommu_lock, flags); -} - static int msm_iommu_map(struct iommu_domain *domain, unsigned longiova,
phys_addr_tpa,size_tlen, int prot,gfp_tgfp)
{
PATCH v3 2/6 iommu/amd: Drop unnecessary checks in amd_iommu_attach_device()
In reply to Nicolin ChenThe same checks are done in amd_iommu_probe_device(). If any of them fails there, then the device won't get a group, so there's no way for it to even reach amd_iommu_attach_device anymore.
--- drivers/iommu/amd/iommu.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
drivers/iommu/amd/iommu.c +2 -10
@@ -2135,21 +2135,13 @@ static void amd_iommu_detach_device(struct iommu_domain *dom,static int amd_iommu_attach_device (struct iommu_domain *dom,struct device *dev) { +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev);struct protection_domain *domain =to_pdomain (dom); -struct iommu_dev_data *dev_data; -struct amd_iommu *iommu; +struct amd_iommu *iommu =rlookup_amd_iommu (dev);int ret; -if (!check_device (dev)) -return -EINVAL ; - - dev_data =dev_iommu_priv_get (dev); dev_data->defer_attach = false; - iommu =rlookup_amd_iommu (dev); -if (!iommu) -return -EINVAL ; -if (dev_data->domain)detach_device (dev);
Signature · 1 line
2.17.1
PATCH v3 3/6 iommu: Add return value rules to attach_dev op and APIs
In reply to Nicolin ChenCases like VFIO wish to attach a device to an existing domain that was not allocated specifically from the device. This raises a condition where the IOMMU driver can fail the domain attach because the domain and device are incompatible with each other.
This is a soft failure that can be resolved by using a different domain.
Provide a dedicated errno EINVAL from the IOMMU driver during attach that the reason why the attach failed is because of domain incompatibility.
VFIO can use this to know that the attach is a soft failure and it should continue searching. Otherwise, the attach will be a hard failure and VFIO will return the code to userspace.
Update kdocs to add rules of return value to the attach_dev op and APIs.
--- drivers/iommu/iommu.c | 24 ++++++++++++++++++++++++ include/linux/iommu.h | 12 ++++++++++++ 2 files changed, 36 insertions(+)
drivers/iommu/iommu.c +24 -0
@@ -1975,6 +1975,18 @@ static int __iommu_attach_device(struct iommu_domain *domain,return ret; } +/** +* iommu_attach_device - Attach a device to an IOMMU domain +* @domain: IOMMU domain to attach +* @dev: Device that will be attached +* +* Returns 0 on success and error code on failure +* +* Note that EINVAL may be returned as a soft failure if the domain and device +* are incompatible: if the domain has already been used or configured in some +* way, attaching the same device to a different domain may succeed. Otherwise, +* it may still represent some fundamental problem. +*/ int iommu_attach_device (struct iommu_domain *domain,struct device *dev) {struct iommu_group *group;
@@ -2101,6 +2113,18 @@ static int __iommu_attach_group(struct iommu_domain *domain,return ret; } +/** +* iommu_attach_group - Attach an IOMMU group to an IOMMU domain +* @domain: IOMMU domain to attach +* @group: IOMMU group that will be attached +* +* Returns 0 on success and error code on failure +* +* Note that EINVAL may be returned as a soft failure if the domain and group +* are incompatible: if the domain has already been used or configured in some +* way, attaching the same group to a different domain may succeed. Otherwise, +* it may still represent some fundamental problem. +*/ int iommu_attach_group (struct iommu_domain *domain,struct iommu_group *group) {int ret;
include/linux/iommu.h +12 -0
@@ -266,6 +266,18 @@ struct iommu_ops {/** * struct iommu_domain_ops - domain specific operations * @attach_dev: attach an iommu domain to a device +* Return: +* * 0 - success +* * EINVAL - the device and domain are incompatible. If this is due to some +* previous configuration of the domain, drivers shouldn't log an +* error, since it is legitimate for a caller to test reuse of an +* existing domain. Otherwise, it may still represent some other +* fundamental problem +* * ENOMEM - out of memory +* * ENOSPC - non-ENOMEM type of resource allocation failures +* * EBUSY - device is attached to a domain and cannot be changed +* * ENODEV - device specific errors, not able to be attached +* * <others> - treated as ENODEV by the caller. Use is discouraged * @detach_dev: detach an iommu domain from a device * @map: map a physically contiguous memory region to an iommu domain * @map_pages: map a physically contiguous set of pages of the same size to
Signature · 1 line
2.17.1
RE: [PATCH v3 3/6] iommu: Add return value rules to attach_dev op and APIs
Quoted 14 lines
From: Nicolin Chen <[email protected]>
Sent: Thursday, September 15, 2022 3:54 PM+/**
+ *iommu_attach_device- Attach a device to an IOMMU domain
+ * @domain: IOMMU domain to attach
+ * @dev: Device that will be attached
+ *
+ * Returns 0 on success and error code on failure
+ *
+ * Note that EINVAL may be returned as a soft failure if the domain and device
+ * are incompatible: if the domain has already been used or configured in some
I didn't get the meaning of the 'if' part.
Quoted 3 lines
+ * way, attaching the same device to a different domain may succeed. Otherwise,
+ * it may still represent some fundamental problem.
I'm not sure what the sentence after 'otherwise' actually adds to the caller. There is no way to differentiate incompatibility vs. fundamental problem, hence pointless for the caller to know this fact.
IMHO just state that the caller can treat -EINVAL as soft failure indicating incompatibility issue between domain and device.
Later for @attach_dev you can add that driver may return (but not recommend) -EINVAL for some fundamental problems.
Re: [PATCH v3 3/6] iommu: Add return value rules to attach_dev op and APIs
Quoted 18 lines · Kevin
External email: Use caution opening links or attachments
Quoted 14 lines
From: Nicolin Chen <[email protected]>
Sent: Thursday, September 15, 2022 3:54 PM+/**
+ *iommu_attach_device- Attach a device to an IOMMU domain
+ * @domain: IOMMU domain to attach
+ * @dev: Device that will be attached
+ *
+ * Returns 0 on success and error code on failure
+ *
+ * Note that EINVAL may be returned as a soft failure if the domain and device
+ * are incompatible: if the domain has already been used or configured in some
Quoted 1 line
I didn't get the meaning of the 'if' part.
It means that the failure of attaching the device to the domain is due to the domain configuration.
Quoted 10 lines
Quoted 3 lines
+ * way, attaching the same device to a different domain may succeed. Otherwise,
+ * it may still represent some fundamental problem.I'm not sure what the sentence after 'otherwise' actually adds to the caller. There is no way to differentiate incompatibility vs. fundamental problem, hence pointless for the caller to know this fact.
IMHO just state that the caller can treat
-EINVALas soft failure indicating incompatibility issue between domain and device.
OK. I changed to:
+ * Note that EINVAL may be returned as a soft failure if the domain and device
+ * are incompatible due to some previous configuration of the domain, in which
+ * case attaching the same device to a different domain may succeed.
Quoted 2 lines
Later for @attach_dev you can add that driver may return (but not recommend)
-EINVALfor some fundamental problems.
OK.
PATCH v3 4/6 iommu: Regulate EINVAL in ->attach_dev callback functions
In reply to Nicolin ChenFollowing the new rules in include/linux/iommu.h kdocs, EINVAL now can be used to indicate that domain and device are incompatible by a caller that treats it as a soft failure and tries attaching to another domain.
On the other hand, there are ->attach_dev callback functions returning it for obvious device-specific errors. They will result in some inefficiency in the caller handling routine.
Update these places to corresponding errnos following the new rules.
--- drivers/iommu/fsl_pamu.c | 2 +- drivers/iommu/fsl_pamu_domain.c | 4 ++-- drivers/iommu/intel/pasid.c | 6 ++++-- drivers/iommu/mtk_iommu.c | 2 +- drivers/iommu/omap-iommu.c | 4 ++-- 5 files changed, 10 insertions(+), 8 deletions(-)
drivers/iommu/fsl_pamu.c +1 -1
@@ -211,7 +211,7 @@ int pamu_config_ppaace(int liodn, u32 omi, u32 stashid, int prot) ppaace->op_encode.index_ot.omi = omi; }else if (~omi !=0 ) {pr_debug ("bad operation mapping index: %d\n" , omi); -return -EINVAL ; +return -ENODEV ; }/* configure stash id */
drivers/iommu/fsl_pamu_domain.c +2 -2
@@ -258,7 +258,7 @@ static int fsl_pamu_attach_device(struct iommu_domain *domain, liodn =of_get_property (dev->of_node,"fsl,liodn" , &len);if (!liodn) {pr_debug ("missing fsl,liodn property at %pOF\n" , dev->of_node); -return -EINVAL ; +return -ENODEV ; }spin_lock_irqsave (&dma_domain->domain_lock, flags);
@@ -267,7 +267,7 @@ static int fsl_pamu_attach_device(struct iommu_domain *domain,if (liodn[i] >=PAACE_NUMBER_ENTRIES ) {pr_debug ("Invalid liodn %d, attach device failed for %pOF\n" , liodn[i], dev->of_node); - ret = -EINVAL ; + ret = -ENODEV ;break ; }
drivers/iommu/intel/pasid.c +4 -2
@@ -101,8 +101,10 @@ int intel_pasid_alloc_table(struct device *dev)might_sleep (); info =dev_iommu_priv_get (dev); -if (WARN_ON (!info || !dev_is_pci (dev) || info->pasid_table)) -return -EINVAL ; +if (WARN_ON (!info || !dev_is_pci (dev))) +return -ENODEV ; +if (WARN_ON (info->pasid_table)) +return -EBUSY ; pasid_table =kzalloc (sizeof (*pasid_table),GFP_KERNEL );if (!pasid_table)
drivers/iommu/mtk_iommu.c +1 -1
@@ -607,7 +607,7 @@ static int mtk_iommu_domain_finalise(struct mtk_iommu_domain *dom, dom->iop =alloc_io_pgtable_ops (ARM_V7S , &dom->cfg, data);if (!dom->iop) {dev_err (data->dev,"Failed to alloc io pgtable\n" ); -return -EINVAL ; +return -ENOMEM ; }/* Update our support page sizes bitmap */
drivers/iommu/omap-iommu.c +2 -2
@@ -1414,7 +1414,7 @@ static int omap_iommu_attach_init(struct device *dev, odomain->num_iommus =omap_iommu_count (dev);if (!odomain->num_iommus) -return -EINVAL ; +return -ENODEV ; odomain->iommus =kcalloc (odomain->num_iommus,sizeof (*iommu),GFP_ATOMIC );
@@ -1464,7 +1464,7 @@ omap_iommu_attach_dev(struct iommu_domain *domain, struct device *dev)if (!arch_data || !arch_data->iommu_dev) {dev_err (dev,"device doesn't have an associated iommu\n" ); -return -EINVAL ; +return -ENODEV ; }spin_lock (&omap_domain->lock);
Signature · 1 line
2.17.1
RE: [PATCH v3 4/6] iommu: Regulate EINVAL in ->attach_dev callback functions
Quoted 8 lines
From: Nicolin Chen <[email protected]>
Sent: Thursday, September 15, 2022 3:59 PMdrivers/iommu/intel/pasid.c +0
-0
drivers/iommu/intel/pasid.c int intel_pasid_alloc_table(struct device *dev) · line 107
Quoted 9 lines
drivers/iommu/intel/pasid.c +4
-2@@ -101,8 +101,10 @@ int intel_pasid_alloc_table(struct device *dev)might_sleep (); info =dev_iommu_priv_get (dev); -if (WARN_ON (!info || !dev_is_pci (dev) || info->pasid_table)) -return -EINVAL ; +if (WARN_ON (!info || !dev_is_pci (dev))) +return -ENODEV ; +if (WARN_ON (info->pasid_table)) +return -EBUSY ;
-EEXIST. It's not a busy condition which might change sometime after.
btw below -EVINAL in viommu should be converted to -ENODEV too since it's a check purely on device side:
viommu_domain_finalise()
viommu_page_size = 1UL << __ffs(viommu->pgsize_bitmap);
if (viommu_page_size > PAGE_SIZE) {
dev_err(vdev->dev,
"granule 0x%lx larger than system page size 0x%lx\n",
viommu_page_size, PAGE_SIZE);
return -EINVAL;
}Re: [PATCH v3 4/6] iommu: Regulate EINVAL in ->attach_dev callback functions
Quoted 26 lines · Kevin
External email: Use caution opening links or attachments
Quoted 8 lines
From: Nicolin Chen <[email protected]>
Sent: Thursday, September 15, 2022 3:59 PMdrivers/iommu/intel/pasid.c +0
-0drivers/iommu/intel/pasid.c int intel_pasid_alloc_table(struct device *dev) · line 107
Quoted 9 lines
drivers/iommu/intel/pasid.c +4
-2@@ -101,8 +101,10 @@ int intel_pasid_alloc_table(struct device *dev)might_sleep (); info =dev_iommu_priv_get (dev); -if (WARN_ON (!info || !dev_is_pci (dev) || info->pasid_table)) -return -EINVAL ; +if (WARN_ON (!info || !dev_is_pci (dev))) +return -ENODEV ; +if (WARN_ON (info->pasid_table)) +return -EBUSY ;
-EEXIST. It's not a busy condition which might change sometime after.btw below
-EVINALin viommu should be converted to-ENODEVtoo since it's a check purely on device side:
Quoted 8 lines
viommu_domain_finalise()viommu_page_size = 1UL << __ffs(viommu->pgsize_bitmap); if (viommu_page_size > PAGE_SIZE) { dev_err(vdev->dev, "granule 0x%lx larger than system page size 0x%lx\n", viommu_page_size, PAGE_SIZE); return -EINVAL; }
Fixed both places. Thanks!
PATCH v3 5/6 iommu: Use EINVAL for incompatible device/domain in ->attach_dev
In reply to Nicolin ChenFollowing the new rules in include/linux/iommu.h kdocs, update all drivers ->attach_dev callback functions to return EINVAL in the failure paths that are related to domain incompatibility.
Also, drop adjacent error prints to prevent a kernel log spam.
--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 11 +---------- drivers/iommu/arm/arm-smmu/arm-smmu.c | 3 --- drivers/iommu/arm/arm-smmu/qcom_iommu.c | 7 +------ drivers/iommu/intel/iommu.c | 10 +++------- drivers/iommu/ipmmu-vmsa.c | 2 -- drivers/iommu/omap-iommu.c | 2 +- drivers/iommu/sprd-iommu.c | 4 +--- drivers/iommu/tegra-gart.c | 2 +- drivers/iommu/virtio-iommu.c | 3 +-- 9 files changed, 9 insertions(+), 35 deletions(-)
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +1 -10
@@ -2429,23 +2429,14 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) goto out_unlock; } }else if (smmu_domain->smmu != smmu) { -dev_err (dev, -"cannot attach to SMMU %s (upstream of %s)\n" , -dev_name (smmu_domain->smmu->dev), -dev_name (smmu->dev)); - ret = -ENXIO ; + ret = -EINVAL ; goto out_unlock; }else if (smmu_domain->stage ==ARM_SMMU_DOMAIN_S1 && master->ssid_bits != smmu_domain->s1_cfg.s1cdmax) { -dev_err (dev, -"cannot attach to incompatible domain (%u SSID bits != %u)\n" , - smmu_domain->s1_cfg.s1cdmax, master->ssid_bits); ret = -EINVAL ; goto out_unlock; }else if (smmu_domain->stage ==ARM_SMMU_DOMAIN_S1 && smmu_domain->stall_enabled != master->stall_enabled) { -dev_err (dev,"cannot attach to stall-%s domain\n" , - smmu_domain->stall_enabled ?"enabled" :"disabled" ); ret = -EINVAL ; goto out_unlock; }
drivers/iommu/arm/arm-smmu/arm-smmu.c +0 -3
@@ -1167,9 +1167,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev)* different SMMUs. */ if (smmu_domain->smmu != smmu) { -dev_err (dev, -"cannot attach to SMMU %s whilst already attached to domain on SMMU %s\n" , -dev_name (smmu_domain->smmu->dev),dev_name (smmu->dev)); ret = -EINVAL ; goto rpm_put; }
drivers/iommu/arm/arm-smmu/qcom_iommu.c +1 -6
@@ -381,13 +381,8 @@ static int qcom_iommu_attach_dev(struct iommu_domain *domain, struct device *dev* Sanity check the domain. We don't support domains across * different IOMMUs. */ -if (qcom_domain->iommu != qcom_iommu) { -dev_err (dev,"cannot attach to IOMMU %s while already " -"attached to domain on IOMMU %s\n" , -dev_name (qcom_domain->iommu->dev), -dev_name (qcom_iommu->dev)); +if (qcom_domain->iommu != qcom_iommu)return -EINVAL ; - }return 0 ; }
drivers/iommu/intel/iommu.c +3 -7
@@ -4158,19 +4158,15 @@ static int prepare_domain_attach_device(struct iommu_domain *domain,return -ENODEV ;if (dmar_domain->force_snooping && !ecap_sc_support (iommu->ecap)) -return -EOPNOTSUPP ; +return -EINVAL ;/* check if this iommu agaw is sufficient for max mapped address */ addr_width =agaw_to_width (iommu->agaw);if (addr_width >cap_mgaw (iommu->cap)) addr_width =cap_mgaw (iommu->cap); -if (dmar_domain->max_addr > (1LL << addr_width)) { -dev_err (dev,"%s: iommu width (%d) is not " -"sufficient for the mapped address (%llx)\n" , - __func__, addr_width, dmar_domain->max_addr); -return -EFAULT ; - } +if (dmar_domain->max_addr > (1LL << addr_width)) +return -EINVAL ; dmar_domain->gaw = addr_width;/*
drivers/iommu/ipmmu-vmsa.c +0 -2
@@ -628,8 +628,6 @@ static int ipmmu_attach_device(struct iommu_domain *io_domain,* Something is wrong, we can't attach two devices using * different IOMMUs to the same domain. */ -dev_err (dev,"Can't attach IPMMU %s to domain on IPMMU %s\n" , -dev_name (mmu->dev),dev_name (domain->mmu->dev)); ret = -EINVAL ; }else dev_info (dev,"Reusing IPMMU context %u\n" , domain->context_id);
drivers/iommu/omap-iommu.c +1 -1
@@ -1472,7 +1472,7 @@ omap_iommu_attach_dev(struct iommu_domain *domain, struct device *dev)/* only a single client device can be attached to a domain */ if (omap_domain->dev) {dev_err (dev,"iommu domain is already attached\n" ); - ret = -EBUSY ; + ret = -EINVAL ; goto out; }
drivers/iommu/sprd-iommu.c +1 -3
@@ -237,10 +237,8 @@ static int sprd_iommu_attach_device(struct iommu_domain *domain,struct sprd_iommu_domain *dom =to_sprd_domain (domain);size_t pgt_size =sprd_iommu_pgt_size (domain); -if (dom->sdev) { -pr_err ("There's already a device attached to this domain.\n" ); +if (dom->sdev)return -EINVAL ; - } dom->pgt_va =dma_alloc_coherent (sdev->dev, pgt_size, &dom->pgt_pa,GFP_KERNEL );if (!dom->pgt_va)
drivers/iommu/tegra-gart.c +1 -1
@@ -112,7 +112,7 @@ static int gart_iommu_attach_dev(struct iommu_domain *domain,spin_lock (&gart->dom_lock);if (gart->active_domain && gart->active_domain != domain) { - ret = -EBUSY ; + ret = -EINVAL ; }else if (dev_iommu_priv_get (dev) != domain) {dev_iommu_priv_set (dev, domain); gart->active_domain = domain;
drivers/iommu/virtio-iommu.c +1 -2
@@ -733,8 +733,7 @@ static int viommu_attach_dev(struct iommu_domain *domain, struct device *dev)*/ ret =viommu_domain_finalise (vdev, domain); }else if (vdomain->viommu != vdev->viommu) { -dev_err (dev,"cannot attach to foreign vIOMMU\n" ); - ret = -EXDEV ; + ret = -EINVAL ; }mutex_unlock (&vdomain->mutex);
Signature · 1 line
2.17.1
RE: [PATCH v3 5/6] iommu: Use EINVAL for incompatible device/domain in ->attach_dev
Quoted 30 lines
From: Nicolin Chen <[email protected]>
Sent: Thursday, September 15, 2022 3:54 PMdrivers/iommu/intel/iommu.c +3
-7@@ -4158,19 +4158,15 @@ static int prepare_domain_attach_device(struct iommu_domain *domain, return -ENODEV ;if (dmar_domain->force_snooping && !ecap_sc_support (iommu- >ecap)) -return -EOPNOTSUPP ; +return -EINVAL ;/* check if this iommu agaw is sufficient for max mapped address */ addr_width =agaw_to_width (iommu->agaw);if (addr_width >cap_mgaw (iommu->cap)) addr_width =cap_mgaw (iommu->cap); -if (dmar_domain->max_addr > (1LL << addr_width)) { -dev_err (dev,"%s: iommu width (%d) is not " -"sufficient for the mapped address (%llx)\n" , - __func__, addr_width, dmar_domain->max_addr); -return -EFAULT ; - } +if (dmar_domain->max_addr > (1LL << addr_width)) +return -EINVAL ; dmar_domain->gaw = addr_width;
Quoted 1 line
/*
Above lacks of a conversion in intel-iommu:
intel_iommu_attach_device()
if (domain->type == IOMMU_DOMAIN_UNMANAGED &&
device_is_rmrr_locked(dev)) {
dev_warn(dev, "Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.\n");
return -EPERM;
}since it's based on the domain type, picking a different domain may work in theory though it won't apply to vfio which always creates unmanaged type.
Re: [PATCH v3 5/6] iommu: Use EINVAL for incompatible device/domain in ->attach_dev
Quoted 9 lines · Kevin
Above lacks of a conversion in intel-iommu:
intel_iommu_attach_device()if (domain->type == IOMMU_DOMAIN_UNMANAGED && device_is_rmrr_locked(dev)) { dev_warn(dev, "Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.\n"); return -EPERM; }
Quoted 3 lines
since it's based on the domain type, picking a different domain may work in theory though it won't apply to vfio which always creates unmanaged type.
IMHO this test shouldn't even be here, that is why it is so strange..
VFIO should be checking if somehow the device doesn't support unmanaged domains at all. We already have several drivers that can't support full featured unamanged domains that vfio needs, this is just another case of that.
But it isn't urgent to fix, I would just ignore this branch for this series.
Jason
RE: [PATCH v3 5/6] iommu: Use EINVAL for incompatible device/domain in ->attach_dev
Quoted 23 lines
From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, September 21, 2022 2:07 AMQuoted 6 lines · Kevin
Above lacks of a conversion in intel-iommu:
intel_iommu_attach_device()if (domain->type == IOMMU_DOMAIN_UNMANAGED && device_is_rmrr_locked(dev)) { dev_warn(dev, "Device is ineligible for IOMMU domainattach due to platform RMRR requirement. Contact your platform vendor.\n");
Quoted 6 lines
return -EPERM; }since it's based on the domain type, picking a different domain may work in theory though it won't apply to vfio which always creates unmanaged type.
IMHO this test shouldn't even be here, that is why it is so strange..
Quoted 8 lines
VFIO should be checking if somehow the device doesn't support unmanaged domains at all. We already have several drivers that can't support full featured unamanged domains that vfio needs, this is just another case of that.
But it isn't urgent to fix, I would just ignore this branch for this series.
OK, then let's leave it as is. Anyway this is a rare path. In reality most RMRR devices are gpu/usb which are already exempted from the above check.
PATCH v3 6/6 iommu: Propagate ret for a potential soft failure EINVAL
In reply to Nicolin ChenFollowing the new rules in include/linux/iommu.h kdocs, EINVAL now can be used to indicate that domain and device are incompatible by a caller that treats it as a soft failure and tries attaching to another domain.
Either mtk_iommu or virtio driver has a place that returns a hard failure instead of the return value from the function call, where an incompatible errno EINVAL could potentially occur.
Propagate the real return value to not miss a potential soft failure.
--- drivers/iommu/mtk_iommu.c | 2 +- drivers/iommu/virtio-iommu.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/iommu/mtk_iommu.c +1 -1
@@ -666,7 +666,7 @@ static int mtk_iommu_attach_device(struct iommu_domain *domain, ret =mtk_iommu_domain_finalise (dom, frstdata, region_id);if (ret) {mutex_unlock (&dom->mutex); -return -ENODEV ; +return ret; } dom->bank = &data->bank[bankid]; }
drivers/iommu/virtio-iommu.c +1 -1
@@ -696,7 +696,7 @@ static int viommu_domain_finalise(struct viommu_endpoint *vdev,if (ret) {ida_free (&viommu->domain_ids, vdomain->id); vdomain->viommu =NULL ; -return -EOPNOTSUPP ; +return ret; } }
Signature · 1 line
2.17.1
RE: [PATCH v3 6/6] iommu: Propagate ret for a potential soft failure EINVAL
Quoted 10 lines
From: Nicolin Chen <[email protected]>
Sent: Thursday, September 15, 2022 3:59 PMFollowing the new rules in
include/linux/iommu.hkdocs, EINVAL now can be used to indicate that domain and device are incompatible by a caller that treats it as a soft failure and tries attaching to another domain.Either
mtk_iommuor virtio driver has a place that returns a hard failure instead of the return value from the function call, where an incompatible errno EINVAL could potentially occur.
in both cases there is no EINVAL returned from the calling stack
IMHO error propagation is the right way even w/o talking about EINVAL otherwise we may miss ENOMEM etc.
Quoted 4 lines
Propagate the real return value to not miss a potential soft failure.
Signed-off-by Nicolin Chen <[email protected]>
Apart from that comment,
Re: [PATCH v3 6/6] iommu: Propagate ret for a potential soft failure EINVAL
Quoted 10 lines · Kevin
External email: Use caution opening links or attachments
Quoted 7 lines
From: Nicolin Chen <[email protected]>
Sent: Thursday, September 15, 2022 3:59 PMFollowing the new rules in
include/linux/iommu.hkdocs, EINVAL now can be used to indicate that domain and device are incompatible by a caller that treats it as a soft failure and tries attaching to another domain.
Quoted 8 lines
Quoted 3 lines
Either
mtk_iommuor virtio driver has a place that returns a hard failure instead of the return value from the function call, where an incompatible errno EINVAL could potentially occur.in both cases there is no EINVAL returned from the calling stack
IMHO error propagation is the right way even w/o talking about EINVAL otherwise we may miss ENOMEM etc.
OK. I changed to:
The mtk_iommu and virtio drivers have places in the ->attach_dev callback functions that return hardcode errnos instead of the returned values, but callers of these ->attach_dv callback functions may care. Propagate them directly without the extra conversions.
Quoted 7 lines
Quoted 3 lines
Propagate the real return value to not miss a potential soft failure.
Signed-off-by Nicolin Chen <[email protected]>Apart from that comment,
Reviewed-by Kevin Tian <[email protected]>
Added this too. Thanks!