iommu: of: Fix check for returning EPROBE_DEFER
Subsystems (6)
PATCH v2 1/3 iommu: of: Fix check for returning EPROBE_DEFER
Now with IOMMU probe deferral, we return -EPROBE_DEFER for masters that are connected to an IOMMU which is not probed yet, but going to get probed, so that we can attach the correct dma_ops. So while trying to defer the probe of the master, check if the of_iommu node that it is connected to is marked in DT as 'status=disabled', then the IOMMU is never is going to get probed. So simply return NULL and let the master work without an IOMMU.
---
[V2] Corrected spelling/case in commit log
drivers/iommu/of_iommu.c | 1 + 1 file changed, 1 insertion(+)
drivers/iommu/of_iommu.c +1 -0
@@ -118,6 +118,7 @@ static bool of_iommu_driver_present(struct device_node *np) ops =iommu_ops_from_fwnode (fwnode);if ((ops && !ops->of_xlate) || + !of_device_is_available (iommu_spec->np) || (!ops && !of_iommu_driver_present (iommu_spec->np)))return NULL ;
Signature · 1 line
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
PATCH v2 2/3 iommu: of: Ignore all errors except EPROBE_DEFER
While deferring the probe of IOMMU masters, xlate and add_device callback can pass back error values like -ENODEV, which means IOMMU cannot be connected with that master for real reasons. So rather than killing the master's probe for such errors, just ignore the errors and let the master work without an IOMMU.
---
[V2] Corrected spelling/case in commit log
drivers/iommu/of_iommu.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/iommu/of_iommu.c +6 -0
@@ -237,6 +237,12 @@ const struct iommu_ops *of_iommu_configure(struct device *dev, ops =ERR_PTR (err); } +/* Ignore all other errors apart from EPROBE_DEFER */ +if (IS_ERR (ops) && (PTR_ERR (ops) != -EPROBE_DEFER )) { +dev_info (dev,"Adding to IOMMU failed: %ld\n" ,PTR_ERR (ops)); + ops =NULL ; + } +return ops; }
Signature · 1 line
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
PATCH v2 3/3 ARM: dma-mapping: Don't tear third-party mappings
In reply to Sricharan RFrom: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/[email protected]>
arch_setup_dma_ops() is used in device probe code paths to create an IOMMU mapping and attach it to the device. The function assumes that the device is attached to a device-specific IOMMU instance (or at least a device-specific TLB in a shared IOMMU instance) and thus creates a separate mapping for every device.
On several systems (Renesas R-Car Gen2 being one of them), that assumption is not true, and IOMMU mappings must be shared between multiple devices. In those cases the IOMMU driver knows better than the generic ARM dma-mapping layer and attaches mapping to devices manually with arm_iommu_attach_device(), which sets the DMA ops for the device.
The arch_setup_dma_ops() function takes this into account and bails out immediately if the device already has DMA ops assigned. However, the corresponding arch_teardown_dma_ops() function, called from driver unbind code paths (including probe deferral), will tear the mapping down regardless of who created it. When the device is reprobed arch_setup_dma_ops() will be called again but won't perform any operation as the DMA ops will still be set.
We need to reset the DMA ops in arch_teardown_dma_ops() to fix this. However, we can't do so unconditionally, as then a new mapping would be created by arch_setup_dma_ops() when the device is reprobed, regardless of whether the device needs to share a mapping or not. We must thus keep track of whether arch_setup_dma_ops() created the mapping, and only in that case tear it down in arch_teardown_dma_ops().
Keep track of that information in the dev_archdata structure. As the structure is embedded in all instances of struct device let's not grow it, but turn the existing dma_coherent bool field into a bitfield that can be used for other purposes.
--- arch/arm/include/asm/device.h | 3 ++- arch/arm/mm/dma-mapping.c | 4 ++++ 2 files changed, 6 insertions(+), 1 deletion(-)
arch/arm/include/asm/device.h +2 -1
@@ -19,7 +19,8 @@ struct dev_archdata {#ifdef CONFIG_XEN const struct dma_map_ops *dev_dma_ops;#endif -bool dma_coherent; + unsigned int dma_coherent:1; + unsigned int dma_ops_setup:1; };struct omap_device ;
arch/arm/mm/dma-mapping.c +4 -0
@@ -2430,9 +2430,13 @@ void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size, dev->dma_ops = xen_dma_ops; }#endif + dev->archdata.dma_ops_setup = true; }void arch_teardown_dma_ops (struct device *dev) { +if (!dev->archdata.dma_ops_setup) +return ; +arm_teardown_iommu_dma_ops (dev); }
Signature · 1 line
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCH V2 2/3] iommu: of: Ignore all errors except EPROBE_DEFER
In reply to Sricharan RHi Sricharan,
Thank you for the patch.
Quoted 7 lines · On Thursday 18 May 2017 15:37:09 Sricharan R wrote:
While deferring the probe of IOMMU masters, xlate and
add_devicecallback can pass back error values like-ENODEV, which means IOMMU cannot be connected with that master for real reasons. So rather than killing the master's probe for such errors, just ignore the errors and let the master work without an IOMMU.
I don't think this is a good idea. Why should we allow IOMMU drivers to return an error if we're always going to ignore the error value ? That will lead to drivers implementing slightly different behaviours, which will be painful the day we'll need to start acting based on the error value.
At the very least, if you want to give a specific meaning to -ENODEV, you should check for that value specifically and not ignore all errors other than -EPROBE_DEFER. You also need to document the meaning of the error value. This can be done in the documentation of the of_xlate operation in include/linux/iommu.h:
include/linux/iommu.h +5 -1
@@ -181,7 +181,6 @@ struct iommu_resv_region {* @domain_window_disable: Disable a particular window for a domain * @domain_set_windows: Set the number of windows for a domain * @domain_get_windows: Return the number of windows for a domain -* @of_xlate: add OF master IDs to iommu grouping * @pgsize_bitmap: bitmap of all possible supported page sizes */ struct iommu_ops {
@@ -224,6 +223,11 @@ struct iommu_ops {/* Get the number of windows per domain */ u32 (*domain_get_windows)(struct iommu_domain *domain); +/** +* @of_xlate: +* +* Add OF master IDs to iommu grouping. +*/ int (*of_xlate)(struct device *dev,struct of_phandle_args *args);unsigned long pgsize_bitmap;
And add documentation for the error codes there.
If you want to ignore some errors returned from the add_device operation you should document it similarly, and in particular document which error check(s) need to be performed by of_xlate and which are the responsibility of add_device.
Quoted 27 lines
Fixes 7b07cbefb68d ("iommu: of: Handle IOMMU lookup failure with deferredprobing or error")
Reported-by Geert Uytterhoeven <[email protected]>Tested-by Magnus Damn <[email protected]>Signed-off-by Sricharan R <[email protected]>---
[V2] Corrected spelling/case in commit logdrivers/iommu/of_iommu.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/iommu/of_iommu.c +6
-0@@ -237,6 +237,12 @@ const struct iommu_ops *of_iommu_configure(structdevice *dev, ops =ERR_PTR (err); } +/* Ignore all other errors apart from EPROBE_DEFER */ +if (IS_ERR (ops) && (PTR_ERR (ops) != -EPROBE_DEFER )) { +dev_info (dev,"Adding to IOMMU failed: %ld\n" ,PTR_ERR (ops)); + ops =NULL ; + } +return ops; }
Signature · 3 lines
Regards, Laurent Pinchart
Hi Sricharan,
Thank you for the patch.
Quoted 9 lines · On Thursday 18 May 2017 15:37:08 Sricharan R wrote:
Now with IOMMU probe deferral, we return -EPROBE_DEFER for masters that are connected to an IOMMU which is not probed yet, but going to get probed, so that we can attach the correct
dma_ops. So while trying to defer the probe of the master, check if theof_iommunode that it is connected to is marked in DT as 'status=disabled', then the IOMMU is never is going to get probed. So simply return NULL and let the master work without an IOMMU.
Quoted 6 lines
Fixes 7b07cbefb68d ("iommu: of: Handle IOMMU lookup failure with deferredprobing or error") Signed-off-by: Sricharan R <[email protected]>
Reported-by Geert Uytterhoeven <[email protected]>Tested-by Will Deacon <[email protected]>Tested-by Magnus Damn <[email protected]>Acked-by Will Deacon <[email protected]>
Quoted 18 lines
---
[V2] Corrected spelling/case in commit logdrivers/iommu/of_iommu.c | 1 + 1 file changed, 1 insertion(+)drivers/iommu/of_iommu.c +1
-0@@ -118,6 +118,7 @@ static bool of_iommu_driver_present(struct device_node *np) ops =iommu_ops_from_fwnode (fwnode);if ((ops && !ops->of_xlate) || + !of_device_is_available (iommu_spec->np) || (!ops && !of_iommu_driver_present (iommu_spec->np)))return NULL ;
Signature · 3 lines
Regards, Laurent Pinchart
Re: [PATCH V2 2/3] iommu: of: Ignore all errors except EPROBE_DEFER
In reply to Laurent PinchartHi Laurent,
Quoted 13 lines · Laurent Pinchart
Hi Sricharan,
Thank you for the patch.
Quoted 7 lines · On Thursday 18 May 2017 15:37:09 Sricharan R wrote:
While deferring the probe of IOMMU masters, xlate and
add_devicecallback can pass back error values like-ENODEV, which means IOMMU cannot be connected with that master for real reasons. So rather than killing the master's probe for such errors, just ignore the errors and let the master work without an IOMMU.
Quoted 5 lines
I don't think this is a good idea. Why should we allow IOMMU drivers to return an error if we're always going to ignore the error value ? That will lead to drivers implementing slightly different behaviours, which will be painful the day we'll need to start acting based on the error value.
The of_iommu_configure interface, before this series, was returning either correct 'iommu_ops' or NULL. Also there was no return value from of_dma_configure which calls of_iommu_configure. This means that if we block only -ENODEV now and let the other errors, the probe of the master devices can be killed for reasons apart from deferring. This would be a change in behavior introduced. All of xlate, add_device, of_pci_map_rid and others can return values apart from -ENODEV. So was thinking that restoring the old behavior, except for returning EPROBE_DEFER was the better thing to do ?
Regards,
Sricharan
Quoted 67 lines
At the very least, if you want to give a specific meaning to
-ENODEV, you should check for that value specifically and not ignore all errors other than -EPROBE_DEFER. You also need to document the meaning of the error value. This can be done in the documentation of theof_xlateoperation ininclude/linux/iommu.h:include/linux/iommu.h +5
-1@@ -181,7 +181,6 @@ struct iommu_resv_region {* @domain_window_disable: Disable a particular window for a domain * @domain_set_windows: Set the number of windows for a domain * @domain_get_windows: Return the number of windows for a domain -* @of_xlate: add OF master IDs to iommu grouping * @pgsize_bitmap: bitmap of all possible supported page sizes */ struct iommu_ops {@@ -224,6 +223,11 @@ struct iommu_ops {/* Get the number of windows per domain */ u32 (*domain_get_windows)(struct iommu_domain *domain); +/** +* @of_xlate: +* +* Add OF master IDs to iommu grouping. +*/ int (*of_xlate)(struct device *dev,struct of_phandle_args *args);unsigned long pgsize_bitmap;And add documentation for the error codes there.
If you want to ignore some errors returned from the
add_deviceoperation you should document it similarly, and in particular document which error check(s) need to be performed byof_xlateand which are the responsibility ofadd_device.Quoted 27 lines
Fixes 7b07cbefb68d ("iommu: of: Handle IOMMU lookup failure with deferredprobing or error")
Reported-by Geert Uytterhoeven <[email protected]>Tested-by Magnus Damn <[email protected]>Signed-off-by Sricharan R <[email protected]>---
[V2] Corrected spelling/case in commit logdrivers/iommu/of_iommu.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/iommu/of_iommu.c +6
-0@@ -237,6 +237,12 @@ const struct iommu_ops *of_iommu_configure(structdevice *dev, ops =ERR_PTR (err); } +/* Ignore all other errors apart from EPROBE_DEFER */ +if (IS_ERR (ops) && (PTR_ERR (ops) != -EPROBE_DEFER )) { +dev_info (dev,"Adding to IOMMU failed: %ld\n" ,PTR_ERR (ops)); + ops =NULL ; + } +return ops; }
Signature · 1 line
"QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCH V2 2/3] iommu: of: Ignore all errors except EPROBE_DEFER
Hi Sricharan,
Quoted 16 lines · On Thursday 18 May 2017 17:26:14 Sricharan R wrote:
Quoted 9 lines · Laurent Pinchart
Quoted 7 lines · On Thursday 18 May 2017 15:37:09 Sricharan R wrote:
While deferring the probe of IOMMU masters, xlate and
add_devicecallback can pass back error values like-ENODEV, which means IOMMU cannot be connected with that master for real reasons. So rather than killing the master's probe for such errors, just ignore the errors and let the master work without an IOMMU.Quoted 5 lines
I don't think this is a good idea. Why should we allow IOMMU drivers to return an error if we're always going to ignore the error value ? That will lead to drivers implementing slightly different behaviours, which will be painful the day we'll need to start acting based on the error value.
Quoted 9 lines
The
of_iommu_configureinterface, before this series, was returning either correct 'iommu_ops' or NULL. Also there was no return value fromof_dma_configurewhich callsof_iommu_configure. This means that if we block only-ENODEVnow and let the other errors, the probe of the master devices can be killed for reasons apart from deferring. This would be a change in behavior introduced. All of xlate,add_device,of_pci_map_ridand others can return values apart from-ENODEV. So was thinking that restoring the old behavior, except for returningEPROBE_DEFERwas the better thing to do ?
We went from a situation where of_iommu_configure() could return either valid operations in the case the device was to be handled by the IOMMU or NULL otherwise, to a situation where we needed a third option for probe deferral. The way we've done this, through error pointers, allows lots of other errors to be returned as well from the of_xlate and add_device operations.
There is currently no use for returning error codes other than -EPROBE_DEFFER from of_iommu_configure(), so your proposal is to block errors returned from the of_xlate and add_device operations inside of_iommu_configure(). My point is that, by doing so, we allow IOMMU drivers to return random error codes that are then ignored. I believe this can cause problems in the future when we will need to extend the API and standardize more error codes, as by then IOMMU drivers will return random errors (they actually do so already to some extent).
For of_xlate I agree with you to some extent. v4.11 just checked whether of_xlate succeeded or not, and just didn't use the IOMMU in case it failed. The exact error value was ignored, and drivers already return random errors. Going back to the v4.11 behaviour is what we need to do in the short-term, even if I believe we should standardize the error values returned from of_xlate after v4.12.
For add_device, however, the situation is a bit different. The add_device operation is called from the IOMMU bus notifier, and the -ENODEV error is ignored by add_iommu_group(). Any other error will cause bus_set_iommu() to fail, which makes IOMMU probing fail for the drivers that check the return value of bus_set_iommu() (some of them don't :-/).
Fixing all this properly requires standardizing the error codes, and going through the existing IOMMU drivers to comply with the standardized behaviour. While this shouldn't be very difficult, it's likely not material for a v4.12- rc fix. We will thus likely need to merge this patch (or something very similar to it), but I'd really like to see this fixed properly for v4.13.
Quoted 66 lines
Quoted 66 lines
At the very least, if you want to give a specific meaning to
-ENODEV, you should check for that value specifically and not ignore all errors other than -EPROBE_DEFER. You also need to document the meaning of the error value. This can be done in the documentation of theof_xlateoperation ininclude/linux/iommu.h:include/linux/iommu.h +0
-1@@ -181,7 +181,6 @@ struct iommu_resv_region {* @domain_window_disable: Disable a particular window for a domain * @domain_set_windows: Set the number of windows for a domain * @domain_get_windows: Return the number of windows for a domain -* @of_xlate: add OF master IDs to iommu grouping * @pgsize_bitmap: bitmap of all possible supported page sizes */ struct
iommu_ops{include/linux/iommu.h +5
-0@@ -224,6 +223,11 @@ struct iommu_ops {/* Get the number of windows per domain */ u32 (*domain_get_windows)(struct iommu_domain *domain); +/** +* @of_xlate: +* +* Add OF master IDs to iommu grouping. +*/ int (*of_xlate)(struct device *dev,struct of_phandle_args *args);unsigned long pgsize_bitmap;And add documentation for the error codes there.
If you want to ignore some errors returned from the
add_deviceoperation you should document it similarly, and in particular document which error check(s) need to be performed byof_xlateand which are the responsibility ofadd_device.Quoted 27 lines
Fixes 7b07cbefb68d ("iommu: of: Handle IOMMU lookup failure withdeferred probing or error")
Reported-by Geert Uytterhoeven <[email protected]>Tested-by Magnus Damn <[email protected]>Signed-off-by Sricharan R <[email protected]>---
[V2] Corrected spelling/case in commit logdrivers/iommu/of_iommu.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/iommu/of_iommu.c +6
-0@@ -237,6 +237,12 @@ const struct iommu_ops *of_iommu_configure(structdevice *dev, ops =ERR_PTR (err); } +/* Ignore all other errors apart from EPROBE_DEFER */ +if (IS_ERR (ops) && (PTR_ERR (ops) != -EPROBE_DEFER )) { +dev_info (dev,"Adding to IOMMU failed: %ld\n" ,PTR_ERR (ops)); + ops =NULL ; + } +return ops; }
Signature · 3 lines
Regards, Laurent Pinchart
Re: [PATCH V2 2/3] iommu: of: Ignore all errors except EPROBE_DEFER
Hi Laurent,
Quoted 29 lines · Laurent Pinchart
Hi Sricharan,
Quoted 16 lines · On Thursday 18 May 2017 17:26:14 Sricharan R wrote:
Quoted 9 lines · Laurent Pinchart
Quoted 7 lines · On Thursday 18 May 2017 15:37:09 Sricharan R wrote:
While deferring the probe of IOMMU masters, xlate and
add_devicecallback can pass back error values like-ENODEV, which means IOMMU cannot be connected with that master for real reasons. So rather than killing the master's probe for such errors, just ignore the errors and let the master work without an IOMMU.Quoted 5 lines
I don't think this is a good idea. Why should we allow IOMMU drivers to return an error if we're always going to ignore the error value ? That will lead to drivers implementing slightly different behaviours, which will be painful the day we'll need to start acting based on the error value.
Quoted 9 lines
The
of_iommu_configureinterface, before this series, was returning either correct 'iommu_ops' or NULL. Also there was no return value fromof_dma_configurewhich callsof_iommu_configure. This means that if we block only-ENODEVnow and let the other errors, the probe of the master devices can be killed for reasons apart from deferring. This would be a change in behavior introduced. All of xlate,add_device,of_pci_map_ridand others can return values apart from-ENODEV. So was thinking that restoring the old behavior, except for returningEPROBE_DEFERwas the better thing to do ?
Quoted 6 lines
We went from a situation where
of_iommu_configure()could return either valid operations in the case the device was to be handled by the IOMMU or NULL otherwise, to a situation where we needed a third option for probe deferral. The way we've done this, through error pointers, allows lots of other errors to be returned as well from theof_xlateandadd_deviceoperations.
right, this was difference in the behavior now.
Quoted 16 lines
There is currently no use for returning error codes other than -EPROBE_DEFFER from
of_iommu_configure(), so your proposal is to block errors returned from theof_xlateandadd_deviceoperations insideof_iommu_configure(). My point is that, by doing so, we allow IOMMU drivers to return random error codes that are then ignored. I believe this can cause problems in the future when we will need to extend the API and standardize more error codes, as by then IOMMU drivers will return random errors (they actually do so already to some extent).For
of_xlateI agree with you to some extent. v4.11 just checked whetherof_xlatesucceeded or not, and just didn't use the IOMMU in case it failed. The exact error value was ignored, and drivers already return random errors. Going back to the v4.11 behaviour is what we need to do in the short-term, even if I believe we should standardize the error values returned fromof_xlateafter v4.12.
Quoted 8 lines
For
add_device, however, the situation is a bit different. Theadd_deviceoperation is called from the IOMMU bus notifier, and the-ENODEVerror is ignored byadd_iommu_group(). Any other error will causebus_set_iommu()to fail, which makes IOMMU probing fail for the drivers that check the return value ofbus_set_iommu()(some of them don't :-/).Fixing all this properly requires standardizing the error codes, and going through the existing IOMMU drivers to comply with the standardized behaviour.
I understand your concern on standardizing the error codes from xlate, add_device, others and handling them properly. As you said there are quite some errors returned from them today. Also another thing is standardizing the behavior of of_iommu_configure itself. So that API serves to connect a device to its correct iommu_ops. When that's not possible, what should be the output and how should that be handled by the caller. The current behavior is to either 1) connect to correct ops or 2) wait for it or 3) progress further with plain/default dma_ops. Anyways as you said standardizing the iommu api ops, would make the of_iommu_configure handling more specific. Having said that i think similar fix needs to be done for acpi's iort_iommu_configure as well.
Quoted 4 lines
While this shouldn't be very difficult, it's likely not material for a v4.12- rc fix. We will thus likely need to merge this patch (or something very similar to it), but I'd really like to see this fixed properly for v4.13.
When you say "merge this patch (or something similar)", is that about documenting the error values for of_xlate and add_device that you showed down below (or) about the patch in discussion ?
Regards,
Sricharan
Quoted 67 lines
Quoted 66 lines
Quoted 66 lines
At the very least, if you want to give a specific meaning to
-ENODEV, you should check for that value specifically and not ignore all errors other than -EPROBE_DEFER. You also need to document the meaning of the error value. This can be done in the documentation of theof_xlateoperation ininclude/linux/iommu.h:include/linux/iommu.h +0
-1@@ -181,7 +181,6 @@ struct iommu_resv_region {* @domain_window_disable: Disable a particular window for a domain * @domain_set_windows: Set the number of windows for a domain * @domain_get_windows: Return the number of windows for a domain -* @of_xlate: add OF master IDs to iommu grouping * @pgsize_bitmap: bitmap of all possible supported page sizes */ struct
iommu_ops{include/linux/iommu.h +5
-0@@ -224,6 +223,11 @@ struct iommu_ops {/* Get the number of windows per domain */ u32 (*domain_get_windows)(struct iommu_domain *domain); +/** +* @of_xlate: +* +* Add OF master IDs to iommu grouping. +*/ int (*of_xlate)(struct device *dev,struct of_phandle_args *args);unsigned long pgsize_bitmap;And add documentation for the error codes there.
If you want to ignore some errors returned from the
add_deviceoperation you should document it similarly, and in particular document which error check(s) need to be performed byof_xlateand which are the responsibility ofadd_device.Quoted 27 lines
Fixes 7b07cbefb68d ("iommu: of: Handle IOMMU lookup failure withdeferred probing or error")
Reported-by Geert Uytterhoeven <[email protected]>Tested-by Magnus Damn <[email protected]>Signed-off-by Sricharan R <[email protected]>---
[V2] Corrected spelling/case in commit logdrivers/iommu/of_iommu.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/iommu/of_iommu.c +6
-0@@ -237,6 +237,12 @@ const struct iommu_ops *of_iommu_configure(structdevice *dev, ops =ERR_PTR (err); } +/* Ignore all other errors apart from EPROBE_DEFER */ +if (IS_ERR (ops) && (PTR_ERR (ops) != -EPROBE_DEFER )) { +dev_info (dev,"Adding to IOMMU failed: %ld\n" ,PTR_ERR (ops)); + ops =NULL ; + } +return ops; }
Signature · 1 line
"QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
Re: [PATCH V2 2/3] iommu: of: Ignore all errors except EPROBE_DEFER
Hi Sricharan
Quoted 63 lines · On Thursday 18 May 2017 19:08:12 Sricharan R wrote:
Quoted 27 lines · Laurent Pinchart
Quoted 16 lines · On Thursday 18 May 2017 17:26:14 Sricharan R wrote:
Quoted 9 lines · Laurent Pinchart
Quoted 7 lines · On Thursday 18 May 2017 15:37:09 Sricharan R wrote:
While deferring the probe of IOMMU masters, xlate and
add_devicecallback can pass back error values like-ENODEV, which means IOMMU cannot be connected with that master for real reasons. So rather than killing the master's probe for such errors, just ignore the errors and let the master work without an IOMMU.Quoted 5 lines
I don't think this is a good idea. Why should we allow IOMMU drivers to return an error if we're always going to ignore the error value ? That will lead to drivers implementing slightly different behaviours, which will be painful the day we'll need to start acting based on the error value.
Quoted 9 lines
The
of_iommu_configureinterface, before this series, was returning either correct 'iommu_ops' or NULL. Also there was no return value fromof_dma_configurewhich callsof_iommu_configure. This means that if we block only-ENODEVnow and let the other errors, the probe of the master devices can be killed for reasons apart from deferring. This would be a change in behavior introduced. All of xlate,add_device,of_pci_map_ridand others can return values apart from-ENODEV. So was thinking that restoring the old behavior, except for returningEPROBE_DEFERwas the better thing to do ?Quoted 6 lines
We went from a situation where
of_iommu_configure()could return either valid operations in the case the device was to be handled by the IOMMU or NULL otherwise, to a situation where we needed a third option for probe deferral. The way we've done this, through error pointers, allows lots of other errors to be returned as well from theof_xlateandadd_deviceoperations.right, this was difference in the behavior now.
Quoted 22 lines
There is currently no use for returning error codes other than -EPROBE_DEFFER from
of_iommu_configure(), so your proposal is to block errors returned from theof_xlateandadd_deviceoperations insideof_iommu_configure(). My point is that, by doing so, we allow IOMMU drivers to return random error codes that are then ignored. I believe this can cause problems in the future when we will need to extend the API and standardize more error codes, as by then IOMMU drivers will return random errors (they actually do so already to some extent).For
of_xlateI agree with you to some extent. v4.11 just checked whetherof_xlatesucceeded or not, and just didn't use the IOMMU in case it failed. The exact error value was ignored, and drivers already return random errors. Going back to the v4.11 behaviour is what we need to do in the short-term, even if I believe we should standardize the error values returned fromof_xlateafter v4.12.For
add_device, however, the situation is a bit different. Theadd_deviceoperation is called from the IOMMU bus notifier, and the-ENODEVerror is ignored byadd_iommu_group(). Any other error will causebus_set_iommu()to fail, which makes IOMMU probing fail for the drivers that check the return value ofbus_set_iommu()(some of them don't :-/).Quoted 3 lines
Fixing all this properly requires standardizing the error codes, and going through the existing IOMMU drivers to comply with the standardized behaviour.
Quoted 11 lines
I understand your concern on standardizing the error codes from xlate,
add_device, others and handling them properly. As you said there are quite some errors returned from them today. Also another thing is standardizing the behavior ofof_iommu_configureitself. So that API serves to connect a device to its correctiommu_ops. When that's not possible, what should be the output and how should that be handled by the caller. The current behavior is to either 1) connect to correct ops or 2) wait for it or 3) progress further with plain/defaultdma_ops. Anyways as you said standardizing the iommu api ops, would make theof_iommu_configurehandling more specific. Having said that i think similar fix needs to be done for acpi'siort_iommu_configureas well.
I'm less knowledgeable about ACPI but I think you're right. Would you like to tackle this for v4.13 ? :-)
Quoted 8 lines
Quoted 4 lines
While this shouldn't be very difficult, it's likely not material for a v4.12- rc fix. We will thus likely need to merge this patch (or something very similar to it), but I'd really like to see this fixed properly for v4.13.
When you say "merge this patch (or something similar)", is that about documenting the error values for
of_xlateandadd_devicethat you showed down below (or) about the patch in discussion ?
I meant the patch we're discussing, "[PATCH V2 2/3] iommu: of: Ignore all errors except EPROBE_DEFER". Sorry for the confusion. One more comment about this below.
Quoted 58 lines
Quoted 58 lines
Quoted 58 lines
Quoted 58 lines
At the very least, if you want to give a specific meaning to
-ENODEV, you should check for that value specifically and not ignore all errors other than -EPROBE_DEFER. You also need to document the meaning of the error value. This can be done in the documentation of theof_xlateoperation ininclude/linux/iommu.h:include/linux/iommu.h +5
-1@@ -181,7 +181,6 @@ struct iommu_resv_region {* @domain_window_disable: Disable a particular window for a domain * @domain_set_windows: Set the number of windows for a domain * @domain_get_windows: Return the number of windows for a domain -* @of_xlate: add OF master IDs to iommu grouping * @pgsize_bitmap: bitmap of all possible supported page sizes */ struct iommu_ops {@@ -224,6 +223,11 @@ struct iommu_ops {/* Get the number of windows per domain */ u32 (*domain_get_windows)(struct iommu_domain *domain); +/** +* @of_xlate: +* +* Add OF master IDs to iommu grouping. +*/ int (*of_xlate)(struct device *dev,struct of_phandle_args *args);unsigned long pgsize_bitmap;And add documentation for the error codes there.
If you want to ignore some errors returned from the
add_deviceoperation you should document it similarly, and in particular document which error check(s) need to be performed byof_xlateand which are the responsibility ofadd_device.Quoted 20 lines
Fixes 7b07cbefb68d ("iommu: of: Handle IOMMU lookup failure withdeferred probing or error")
Reported-by Geert Uytterhoeven <[email protected]>Tested-by Magnus Damn <[email protected]>Signed-off-by Sricharan R <[email protected]>---
[V2] Corrected spelling/case in commit logdrivers/iommu/of_iommu.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/iommu/of_iommu.c +0
-0@@ -237,6 +237,12 @@ const struct iommu_ops *of_iommu_configure(structdevice *dev, ops =ERR_PTR (err); }
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
drivers/iommu/of_iommu.c const struct iommu_ops *of_iommu_configure(struct · line 243
Quoted 3 lines
drivers/iommu/of_iommu.c +3
-0+/* Ignore all other errors apart from EPROBE_DEFER */ +if (IS_ERR (ops) && (PTR_ERR (ops) != -EPROBE_DEFER )) { + dev_info(dev, "Adding to IOMMU failed: %ld\n",
PTR_ERR(ops));
I would turn this into a dev_dbg(), as at least the -ENODEV error returned from add_device has a defined meaning (see the comment in add_iommu_group()) and is in my opinion not a condition that should be reported in the kernel log by default.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
drivers/iommu/of_iommu.c +3
-0+ ops =NULL ; + } +return ops; }
Signature · 3 lines
Regards, Laurent Pinchart
Re: [PATCH V2 2/3] iommu: of: Ignore all errors except EPROBE_DEFER
Hi Laurent,
Quoted 78 lines · Laurent Pinchart
Hi Sricharan
Quoted 63 lines · On Thursday 18 May 2017 19:08:12 Sricharan R wrote:
Quoted 27 lines · Laurent Pinchart
Quoted 16 lines · On Thursday 18 May 2017 17:26:14 Sricharan R wrote:
Quoted 9 lines · Laurent Pinchart
Quoted 7 lines · On Thursday 18 May 2017 15:37:09 Sricharan R wrote:
While deferring the probe of IOMMU masters, xlate and
add_devicecallback can pass back error values like-ENODEV, which means IOMMU cannot be connected with that master for real reasons. So rather than killing the master's probe for such errors, just ignore the errors and let the master work without an IOMMU.Quoted 5 lines
I don't think this is a good idea. Why should we allow IOMMU drivers to return an error if we're always going to ignore the error value ? That will lead to drivers implementing slightly different behaviours, which will be painful the day we'll need to start acting based on the error value.
Quoted 9 lines
The
of_iommu_configureinterface, before this series, was returning either correct 'iommu_ops' or NULL. Also there was no return value fromof_dma_configurewhich callsof_iommu_configure. This means that if we block only-ENODEVnow and let the other errors, the probe of the master devices can be killed for reasons apart from deferring. This would be a change in behavior introduced. All of xlate,add_device,of_pci_map_ridand others can return values apart from-ENODEV. So was thinking that restoring the old behavior, except for returningEPROBE_DEFERwas the better thing to do ?Quoted 6 lines
We went from a situation where
of_iommu_configure()could return either valid operations in the case the device was to be handled by the IOMMU or NULL otherwise, to a situation where we needed a third option for probe deferral. The way we've done this, through error pointers, allows lots of other errors to be returned as well from theof_xlateandadd_deviceoperations.right, this was difference in the behavior now.
Quoted 22 lines
There is currently no use for returning error codes other than -EPROBE_DEFFER from
of_iommu_configure(), so your proposal is to block errors returned from theof_xlateandadd_deviceoperations insideof_iommu_configure(). My point is that, by doing so, we allow IOMMU drivers to return random error codes that are then ignored. I believe this can cause problems in the future when we will need to extend the API and standardize more error codes, as by then IOMMU drivers will return random errors (they actually do so already to some extent).For
of_xlateI agree with you to some extent. v4.11 just checked whetherof_xlatesucceeded or not, and just didn't use the IOMMU in case it failed. The exact error value was ignored, and drivers already return random errors. Going back to the v4.11 behaviour is what we need to do in the short-term, even if I believe we should standardize the error values returned fromof_xlateafter v4.12.For
add_device, however, the situation is a bit different. Theadd_deviceoperation is called from the IOMMU bus notifier, and the-ENODEVerror is ignored byadd_iommu_group(). Any other error will causebus_set_iommu()to fail, which makes IOMMU probing fail for the drivers that check the return value ofbus_set_iommu()(some of them don't :-/).Quoted 3 lines
Fixing all this properly requires standardizing the error codes, and going through the existing IOMMU drivers to comply with the standardized behaviour.
Quoted 11 lines
I understand your concern on standardizing the error codes from xlate,
add_device, others and handling them properly. As you said there are quite some errors returned from them today. Also another thing is standardizing the behavior ofof_iommu_configureitself. So that API serves to connect a device to its correctiommu_ops. When that's not possible, what should be the output and how should that be handled by the caller. The current behavior is to either 1) connect to correct ops or 2) wait for it or 3) progress further with plain/defaultdma_ops. Anyways as you said standardizing the iommu api ops, would make theof_iommu_configurehandling more specific. Having said that i think similar fix needs to be done for acpi'siort_iommu_configureas well.
Quoted 2 lines
I'm less knowledgeable about ACPI but I think you're right. Would you like to tackle this for v4.13 ? :-)
Will add the fix for ACPI now in this series. ok, Will really see if i can address the standardizing part for 4.13.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
While this shouldn't be very difficult, it's likely not material for a v4.12- rc fix. We will thus likely need to merge this patch (or something very similar to it), but I'd really like to see this fixed properly for v4.13.
Quoted 7 lines
Quoted 3 lines
When you say "merge this patch (or something similar)", is that about documenting the error values for
of_xlateandadd_devicethat you showed down below (or) about the patch in discussion ?I meant the patch we're discussing, "[PATCH V2 2/3] iommu: of: Ignore all errors except
EPROBE_DEFER". Sorry for the confusion. One more comment about this below.
ok.
Quoted 64 lines
Quoted 59 lines
Quoted 59 lines
Quoted 59 lines
Quoted 59 lines
At the very least, if you want to give a specific meaning to
-ENODEV, you should check for that value specifically and not ignore all errors other than -EPROBE_DEFER. You also need to document the meaning of the error value. This can be done in the documentation of theof_xlateoperation ininclude/linux/iommu.h:include/linux/iommu.h +5
-1@@ -181,7 +181,6 @@ struct iommu_resv_region {* @domain_window_disable: Disable a particular window for a domain * @domain_set_windows: Set the number of windows for a domain * @domain_get_windows: Return the number of windows for a domain -* @of_xlate: add OF master IDs to iommu grouping * @pgsize_bitmap: bitmap of all possible supported page sizes */ struct iommu_ops {@@ -224,6 +223,11 @@ struct iommu_ops {/* Get the number of windows per domain */ u32 (*domain_get_windows)(struct iommu_domain *domain); +/** +* @of_xlate: +* +* Add OF master IDs to iommu grouping. +*/ int (*of_xlate)(struct device *dev,struct of_phandle_args *args);unsigned long pgsize_bitmap;And add documentation for the error codes there.
If you want to ignore some errors returned from the
add_deviceoperation you should document it similarly, and in particular document which error check(s) need to be performed byof_xlateand which are the responsibility ofadd_device.Quoted 20 lines
Fixes 7b07cbefb68d ("iommu: of: Handle IOMMU lookup failure withdeferred probing or error")
Reported-by Geert Uytterhoeven <[email protected]>Tested-by Magnus Damn <[email protected]>Signed-off-by Sricharan R <[email protected]>---
[V2] Corrected spelling/case in commit logdrivers/iommu/of_iommu.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/iommu/of_iommu.c +0
-0@@ -237,6 +237,12 @@ const struct iommu_ops *of_iommu_configure(structdevice *dev, ops =ERR_PTR (err); }Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
drivers/iommu/of_iommu.c const struct iommu_ops *of_iommu_configure(struct · line 243
Quoted 3 lines
drivers/iommu/of_iommu.c +3
-0+/* Ignore all other errors apart from EPROBE_DEFER */ +if (IS_ERR (ops) && (PTR_ERR (ops) != -EPROBE_DEFER )) { + dev_info(dev, "Adding to IOMMU failed: %ld\n",
PTR_ERR(ops));
Quoted 5 lines
I would turn this into a
dev_dbg(), as at least the-ENODEVerror returned fromadd_devicehas a defined meaning (see the comment inadd_iommu_group()) and is in my opinion not a condition that should be reported in the kernel log by default.
ok, will turn this in to a dev_dbg.
Regards,
Sricharan
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
drivers/iommu/of_iommu.c +3
-0+ ops =NULL ; + } +return ops; }
Signature · 1 line
"QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation