iommu/arm-smmu: stalling support (v2)
Subsystems (10)
cover iommu/arm-smmu: stalling support (v2)
Now, as was previously discussed[1], now we have a dt property on the smmu, to indicate that the smmu supports stalling, as well as a flag on the domain (set when fault-handler is registered) to indicate that the domain supports stalling.
I ran into one snag, thanks to the auto-attached DMA domain, since that doesn't support stalling. I think when the driver is explicitly managing the iommu, we should just get rid of the DMA domain. (But maybe there is a cleaner way to do this than what I came up with, so suggestions welcome.)
[1] http://www.spinics.net/lists/linux-arm-msm/msg25357.html
Rob Clark (3):
iommu: introduce stall/resume support iommu/arm-smmu: Add support to opt-in to stalling iommu/arm-smmu: detach DMA domain if driver is managing iommu
.../devicetree/bindings/iommu/arm,smmu.txt | 3 + drivers/gpu/drm/etnaviv/etnaviv_mmu.c | 2 +- drivers/gpu/drm/msm/msm_iommu.c | 12 ++- drivers/infiniband/hw/usnic/usnic_uiom.c | 2 +- drivers/iommu/arm-smmu.c | 100 +++++++++++++++++++-- drivers/iommu/iommu.c | 24 ++++- drivers/remoteproc/remoteproc_core.c | 2 +- include/linux/iommu.h | 5 +- 8 files changed, 136 insertions(+), 14 deletions(-)
Signature · 1 line
2.9.3
A message that is not in the archive (sent off-list or not yet received): 20170201152341.29142-1-robdclark-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
RFC 1/3 iommu: introduce stall/resume support
A new flag when registering the fault handler indicates that the user supports stalling, and will call iommu_domain_resume() at some point later, potentially from a workqueue. (This would allow the user to do mm related operations that could not be done from irq context.)
--- drivers/gpu/drm/etnaviv/etnaviv_mmu.c | 2 +- drivers/gpu/drm/msm/msm_iommu.c | 12 +++++++++--- drivers/infiniband/hw/usnic/usnic_uiom.c | 2 +- drivers/iommu/iommu.c | 24 +++++++++++++++++++++++- drivers/remoteproc/remoteproc_core.c | 2 +- include/linux/iommu.h | 5 ++++- 6 files changed, 39 insertions(+), 8 deletions(-)
drivers/gpu/drm/etnaviv/etnaviv_mmu.c +1 -1
@@ -303,7 +303,7 @@ struct etnaviv_iommu *etnaviv_iommu_new(struct etnaviv_gpu *gpu) mmu->domain->geometry.aperture_end - mmu->domain->geometry.aperture_start + 1); -iommu_set_fault_handler (mmu->domain, etnaviv_fault_handler, gpu->dev); +iommu_set_fault_handler (mmu->domain, etnaviv_fault_handler, gpu->dev, false);return mmu; }
drivers/gpu/drm/msm/msm_iommu.c +9 -3
@@ -29,9 +29,15 @@ static int msm_fault_handler(struct iommu_domain *domain, struct device *dev, unsigned long iova, int flags, void *arg) { struct msm_iommu *iommu = arg; +int ret =0 ; +if (iommu->base.handler) -return iommu->base.handler (iommu->base.arg, iova, flags); -pr_warn_ratelimited ("*** fault: iova=%08lx, flags=%d\n" , iova, flags); + ret = iommu->base.handler (iommu->base.arg, iova, flags); +else +pr_warn_ratelimited ("*** fault: iova=%08lx, flags=%d\n" , iova, flags); + +iommu_domain_resume (domain, false); +return 0 ; }
@@ -172,7 +178,7 @@ struct msm_mmu *msm_iommu_new(struct device *dev, struct iommu_domain *domain) iommu->domain = domain;msm_mmu_init (&iommu->base, dev, &funcs); -iommu_set_fault_handler (domain, msm_fault_handler, iommu); +iommu_set_fault_handler (domain, msm_fault_handler, iommu, true);if (of_find_compatible_node (NULL ,NULL ,"qcom,msm-smmu-v2" ) ||of_find_compatible_node (NULL ,NULL ,"qcom,msm-mmu-500" ))
drivers/infiniband/hw/usnic/usnic_uiom.c +1 -1
@@ -476,7 +476,7 @@ struct usnic_uiom_pd *usnic_uiom_alloc_pd(void)return ERR_PTR (-ENOMEM ); } -iommu_set_fault_handler (pd->domain, usnic_uiom_dma_fault,NULL ); +iommu_set_fault_handler (pd->domain, usnic_uiom_dma_fault,NULL , false);spin_lock_init (&pd->lock);INIT_LIST_HEAD (&pd->devs);
drivers/iommu/iommu.c +23 -1
@@ -1029,6 +1029,7 @@ EXPORT_SYMBOL_GPL(iommu_capable);* @domain: iommu domain * @handler: fault handler * @token: user data, will be passed back to the fault handler +* @can_stall: the user can support stalling on iommu fault * * This function should be used by IOMMU users which want to be notified * whenever an IOMMU fault happens.
@@ -1038,12 +1039,14 @@ EXPORT_SYMBOL_GPL(iommu_capable);*/ void iommu_set_fault_handler (struct iommu_domain *domain,iommu_fault_handler_t handler, -void *token) +void *token, +bool can_stall) {BUG_ON (!domain); domain->handler = handler; domain->handler_token = token; + domain->can_stall = can_stall; }EXPORT_SYMBOL_GPL (iommu_set_fault_handler);
@@ -1546,6 +1549,25 @@ int iommu_domain_set_attr(struct iommu_domain *domain, }EXPORT_SYMBOL_GPL (iommu_domain_set_attr); +/** +* iommu_domain_resume() - resume a stalled transaction after fault +* @domain: iommu domain +* @resume: if true, resume the transaction, else abort it +* +* Users that pass can_stall=true to iommu_set_fault_handler() must +* call this function to resume (or terminate) the stalled iommu +* transaction. It may either be called directly from the fault +* handler, or at some point later from a thread context (ie. if the +* fault handler needs to do anything that cannot be done from atomic +* context, ie. use any mm related API) +*/ +void iommu_domain_resume (struct iommu_domain *domain,bool resume) +{ +if (domain->ops->domain_resume) + domain->ops->domain_resume (domain, resume); +} +EXPORT_SYMBOL_GPL (iommu_domain_resume); +void iommu_get_dm_regions (struct device *dev,struct list_head *list) {const struct iommu_ops *ops = dev->bus->iommu_ops;
drivers/remoteproc/remoteproc_core.c +1 -1
@@ -110,7 +110,7 @@ static int rproc_enable_iommu(struct rproc *rproc)return -ENOMEM ; } -iommu_set_fault_handler (domain, rproc_iommu_fault, rproc); +iommu_set_fault_handler (domain, rproc_iommu_fault, rproc, false); ret =iommu_attach_device (domain, dev);if (ret) {
include/linux/iommu.h +4 -1
@@ -82,6 +82,7 @@ struct iommu_domain {unsigned long pgsize_bitmap;/* Bitmap of page sizes in use */ iommu_fault_handler_t handler;void *handler_token; +bool can_stall;struct iommu_domain_geometry geometry;void *iova_cookie; };
@@ -183,6 +184,7 @@ struct iommu_ops { enum iommu_attr attr, void *data);int (*domain_set_attr)(struct iommu_domain *domain,enum iommu_attr attr,void *data); +void (*domain_resume)(struct iommu_domain *domain,bool resume);/* Request/Free a list of direct mapping requirements for a device */ void (*get_dm_regions)(struct device *dev,struct list_head *list);
@@ -231,7 +233,7 @@ extern size_t default_iommu_map_sg(struct iommu_domain *domain, unsigned long io int prot);extern phys_addr_t iommu_iova_to_phys (struct iommu_domain *domain,dma_addr_t iova);extern void iommu_set_fault_handler (struct iommu_domain *domain, -iommu_fault_handler_t handler,void *token); +iommu_fault_handler_t handler,void *token,bool can_stall);extern void iommu_get_dm_regions (struct device *dev,struct list_head *list);extern void iommu_put_dm_regions (struct device *dev,struct list_head *list);
@@ -266,6 +268,7 @@ extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr, void *data);extern int iommu_domain_set_attr (struct iommu_domain *domain,enum iommu_attr ,void *data); +extern void iommu_domain_resume (struct iommu_domain *domain,bool resume);struct device *iommu_device_create (struct device *parent,void *drvdata,const struct attribute_group **groups,const char *fmt, ...)__printf (4 ,5 );
Signature · 1 line
2.9.3
RFC 2/3 iommu/arm-smmu: Add support to opt-in to stalling
In reply to a message not in the archiveTODO maybe some dev_dbg() or some other way to tell if stalling is actually enabled?
--- .../devicetree/bindings/iommu/arm,smmu.txt | 3 + drivers/iommu/arm-smmu.c | 85 ++++++++++++++++++++-- 2 files changed, 82 insertions(+), 6 deletions(-)
Documentation/devicetree/bindings/iommu/arm,smmu.txt +3 -0
@@ -68,6 +68,9 @@ conditions. aliases of secure registers have to be used during SMMU configuration. +- arm,smmu-enable-stall : Enable stall mode to stall memory transactions + and resume after fault is handled + ** Deprecated properties: - mmu-masters (deprecated in favour of the generic "iommus" binding) :
drivers/iommu/arm-smmu.c +79 -6
173 lines not on this page · raw message
Signature · 1 line
2.9.3
RFC 3/3 iommu/arm-smmu: detach DMA domain if driver is managing iommu
In reply to Rob ClarkBefore the driver is probed, arm_smmu_add_device() helpfully attaches an IOMMU_DOMAIN_DMA domain. Which ofc does not support stalling, and when the driver later attaches a domain that can_stall to an smmu that can stall, the default _DMA domain prevents stalling from being enabled. (And will cause further problems later)
One simple way to deal with this is simply toss the default _DMA domain if the driver attaches it's own domain.
TODO maybe the tracking of list of attached domains should be done in iommu core, so the detach can happen outside of group->mutex.
--- drivers/iommu/arm-smmu.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/iommu/arm-smmu.c +15 -0
@@ -1323,6 +1323,21 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) smmu =fwspec_smmu (fwspec); +/* +* If driver is explicitly managing the iommu, detatch any previously +* attached _DMA domains. +* +* TODO maybe this logic should be in iommu_attach_device() so it can +* happen outside of holding group->mutex?? +*/ +if (domain->type !=IOMMU_DOMAIN_DMA ) { +struct arm_smmu_domain *other_domain, *n; + +list_for_each_entry_safe (other_domain, n, &smmu->domain_list, domain_node) +if (other_domain->domain.type ==IOMMU_DOMAIN_DMA ) +arm_smmu_detach_dev (&other_domain->domain, dev); + } +if (WARN_ON (!list_empty (&smmu_domain->domain_node)))return -EINVAL ;
Signature · 1 line
2.9.3
A message that is not in the archive (sent off-list or not yet received): 20170201152341.29142-4-robdclark-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Re: [RFC 3/3] iommu/arm-smmu: detach DMA domain if driver is managing iommu
Quoted 28 lines · Rob Clark
Before the driver is probed,
arm_smmu_add_device()helpfully attaches anIOMMU_DOMAIN_DMAdomain. Which ofc does not support stalling, and when the driver later attaches a domain thatcan_stallto an smmu that can stall, the default_DMAdomain prevents stalling from being enabled. (And will cause further problems later)One simple way to deal with this is simply toss the default
_DMAdomain if the driver attaches it's own domain.TODO maybe the tracking of list of attached domains should be done in iommu core, so the detach can happen outside of
group->mutex.Signed-off-by Rob Clark <[email protected]>--- drivers/iommu/arm-smmu.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/iommu/arm-smmu.c +3
-0@@ -1323,6 +1323,21 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) smmu =fwspec_smmu (fwspec); +/* +* If driver is explicitly managing the iommu, detatch any previously +* attached _DMA domains.
drivers/iommu/arm-smmu.c static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) · line 1338
Quoted 10 lines
drivers/iommu/arm-smmu.c +10
-0+* +* TODO maybe this logic should be in iommu_attach_device() so it can +* happen outside of holding group->mutex?? +*/ +if (domain->type !=IOMMU_DOMAIN_DMA ) { +struct arm_smmu_domain *other_domain, *n; + +list_for_each_entry_safe (other_domain, n, &smmu->domain_list, domain_node) +if (other_domain->domain.type ==IOMMU_DOMAIN_DMA ) +arm_smmu_detach_dev (&other_domain->domain, dev);
hmm, we might want to unhook dev->archdata.dma_ops here too..
I'm thinking maybe on arm64 __generic_dma_ops() should fall back to swiotlb ops instead of dummy_ops if archdata.dma_ops is NULL, so that we could just set it to NULL here?
(Is there really any purpose for having the dummy-ops??)
BR,
-R
Quoted 8 lines
drivers/iommu/arm-smmu.c +2
-1+ } +if (WARN_ON (!list_empty (&smmu_domain->domain_node)))return -EINVAL ; --2.9.3
A message that is not in the archive (sent off-list or not yet received): CAF6AEGvXc1dUaqQ6=yp9XCZTv6y1r7jNJohrksO0=EJ1+UFY5g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
RE: [RFC 3/3] iommu/arm-smmu: detach DMA domain if driver is managing iommu
Hi Rob,
Quoted 26 lines · Rob Clark
Quoted 25 lines
Before the driver is probed,
arm_smmu_add_device()helpfully attaches anIOMMU_DOMAIN_DMAdomain. Which ofc does not support stalling, and when the driver later attaches a domain thatcan_stallto an smmu that can stall, the default_DMAdomain prevents stalling from being enabled. (And will cause further problems later)One simple way to deal with this is simply toss the default
_DMAdomain if the driver attaches it's own domain.TODO maybe the tracking of list of attached domains should be done in iommu core, so the detach can happen outside of
group->mutex.Signed-off-by Rob Clark <[email protected]>--- drivers/iommu/arm-smmu.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/iommu/arm-smmu.c +0
-0@@ -1323,6 +1323,21 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) smmu =fwspec_smmu (fwspec);
Quoted 14 lines
drivers/iommu/arm-smmu.c static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) · line 1338
Quoted 13 lines
drivers/iommu/arm-smmu.c +13
-0+/* +* If driver is explicitly managing the iommu, detatch any previously +* attached _DMA domains. +* +* TODO maybe this logic should be in iommu_attach_device() so it can +* happen outside of holding group->mutex?? +*/ +if (domain->type !=IOMMU_DOMAIN_DMA ) { +struct arm_smmu_domain *other_domain, *n; + +list_for_each_entry_safe (other_domain, n, &smmu->domain_list, domain_node) +if (other_domain->domain.type ==IOMMU_DOMAIN_DMA ) +arm_smmu_detach_dev (&other_domain->domain, dev);
So the arm_smmu_detach_dev api is no more there and is removed now. Also this will be a problem when multiple devices share the iommu, we end up removing domains used by other devices. Should this not be done per-device which does not want to use the DMA domain ?
Quoted 6 lines
hmm, we might want to unhook
dev->archdata.dma_opshere too..I'm thinking maybe on arm64
__generic_dma_ops()should fall back to swiotlb ops instead ofdummy_opsif archdata.dma_ops is NULL, so that we could just set it to NULL here?
hmm, both not attaching the default dma domain and not setting the dma_ops is tried in this series as well [1]
[1] https://www.spinics.net/lists/arm-kernel/msg556081.html
Quoted 2 lines
(Is there really any purpose for having the dummy-ops??)
To enforce setting arch_setup_dma_ops for device so that the devices can do cache coherent transactions, otherwise disable the dma capability of the device. I see that this was introduced as a part of making ACPI_CCA_REQUIRED to be set in arm64 and later generalized.
Regards,
Sricharan
Quoted 11 lines
BR,
-RQuoted 8 lines
drivers/iommu/arm-smmu.c +2
-1+ } +if (WARN_ON (!list_empty (&smmu_domain->domain_node)))return -EINVAL ; --2.9.3
Re: [RFC 3/3] iommu/arm-smmu: detach DMA domain if driver is managing iommu
Quoted 43 lines · Sricharan
Hi Rob,
Quoted 26 lines · Rob Clark
Quoted 25 lines
Before the driver is probed,
arm_smmu_add_device()helpfully attaches anIOMMU_DOMAIN_DMAdomain. Which ofc does not support stalling, and when the driver later attaches a domain thatcan_stallto an smmu that can stall, the default_DMAdomain prevents stalling from being enabled. (And will cause further problems later)One simple way to deal with this is simply toss the default
_DMAdomain if the driver attaches it's own domain.TODO maybe the tracking of list of attached domains should be done in iommu core, so the detach can happen outside of
group->mutex.Signed-off-by Rob Clark <[email protected]>--- drivers/iommu/arm-smmu.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/iommu/arm-smmu.c +0
-0@@ -1323,6 +1323,21 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) smmu =fwspec_smmu (fwspec);Quoted 14 lines
drivers/iommu/arm-smmu.c static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) · line 1338
Quoted 13 lines
drivers/iommu/arm-smmu.c +13
-0+/* +* If driver is explicitly managing the iommu, detatch any previously +* attached _DMA domains. +* +* TODO maybe this logic should be in iommu_attach_device() so it can +* happen outside of holding group->mutex?? +*/ +if (domain->type !=IOMMU_DOMAIN_DMA ) { +struct arm_smmu_domain *other_domain, *n; + +list_for_each_entry_safe (other_domain, n, &smmu->domain_list, domain_node) +if (other_domain->domain.type ==IOMMU_DOMAIN_DMA ) +arm_smmu_detach_dev (&other_domain->domain, dev);
Quoted 4 lines
So the
arm_smmu_detach_devapi is no more there and is removed now. Also this will be a problem when multiple devices share the iommu, we end up removing domains used by other devices. Should this not be done per-device which does not want to use the DMA domain ?
I actually added the _detach fxn in the previous patch.
But I think managing the iommu explicitly is not going to work if multiple devices share the same pagetable. I think that is a scenario that should never happen with the gpu.
Quoted 11 lines
Quoted 6 lines
hmm, we might want to unhook
dev->archdata.dma_opshere too..I'm thinking maybe on arm64
__generic_dma_ops()should fall back to swiotlb ops instead ofdummy_opsif archdata.dma_ops is NULL, so that we could just set it to NULL here?hmm, both not attaching the default dma domain and not setting the
dma_opsis tried in this series as well [1]
I'll have a look at that series later this morning.
Quoted 8 lines
Quoted 2 lines
(Is there really any purpose for having the dummy-ops??)
To enforce setting
arch_setup_dma_opsfor device so that the devices can do cache coherent transactions, otherwise disable the dma capability of the device. I see that this was introduced as a part of makingACPI_CCA_REQUIREDto be set in arm64 and later generalized.
hmm, looks like all the dummy ops just fail. I'm not entirely sure how that can be useful. But I'll poke around the git history. Seems like just always falling back to swiotlb op's would be the equivalent to armv7 falling back to arm_dma_ops..
BR,
-R
Quoted 16 lines
Regards,
SricharanQuoted 11 lines
BR,
-RQuoted 8 lines
drivers/iommu/arm-smmu.c +2
-1+ } +if (WARN_ON (!list_empty (&smmu_domain->domain_node)))return -EINVAL ; --2.9.3
Re: [RFC 3/3] iommu/arm-smmu: detach DMA domain if driver is managing iommu
In reply to SricharanQuoted 60 lines · Sricharan
Hi Rob,
Quoted 26 lines · Rob Clark
Quoted 25 lines
Before the driver is probed,
arm_smmu_add_device()helpfully attaches anIOMMU_DOMAIN_DMAdomain. Which ofc does not support stalling, and when the driver later attaches a domain thatcan_stallto an smmu that can stall, the default_DMAdomain prevents stalling from being enabled. (And will cause further problems later)One simple way to deal with this is simply toss the default
_DMAdomain if the driver attaches it's own domain.TODO maybe the tracking of list of attached domains should be done in iommu core, so the detach can happen outside of
group->mutex.Signed-off-by Rob Clark <[email protected]>--- drivers/iommu/arm-smmu.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/iommu/arm-smmu.c +0
-0@@ -1323,6 +1323,21 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) smmu =fwspec_smmu (fwspec);Quoted 14 lines
drivers/iommu/arm-smmu.c static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) · line 1338
Quoted 13 lines
drivers/iommu/arm-smmu.c +13
-0+/* +* If driver is explicitly managing the iommu, detatch any previously +* attached _DMA domains. +* +* TODO maybe this logic should be in iommu_attach_device() so it can +* happen outside of holding group->mutex?? +*/ +if (domain->type !=IOMMU_DOMAIN_DMA ) { +struct arm_smmu_domain *other_domain, *n; + +list_for_each_entry_safe (other_domain, n, &smmu->domain_list, domain_node) +if (other_domain->domain.type ==IOMMU_DOMAIN_DMA ) +arm_smmu_detach_dev (&other_domain->domain, dev);So the
arm_smmu_detach_devapi is no more there and is removed now. Also this will be a problem when multiple devices share the iommu, we end up removing domains used by other devices. Should this not be done per-device which does not want to use the DMA domain ?Quoted 6 lines
hmm, we might want to unhook
dev->archdata.dma_opshere too..I'm thinking maybe on arm64
__generic_dma_ops()should fall back to swiotlb ops instead ofdummy_opsif archdata.dma_ops is NULL, so that we could just set it to NULL here?hmm, both not attaching the default dma domain and not setting the
dma_opsis tried in this series as well [1]
Quoted 8 lines
Quoted 2 lines
(Is there really any purpose for having the dummy-ops??)
To enforce setting
arch_setup_dma_opsfor device so that the devices can do cache coherent transactions, otherwise disable the dma capability of the device. I see that this was introduced as a part of makingACPI_CCA_REQUIREDto be set in arm64 and later generalized.
hmm, maybe fallback to swiotlb ops vs dummy ops should depend on whether it is an ACPI system or not..
BR,
-R