road blocks for using dma-iommu on more than arm64
Hi all,
are there any road blocks for using drivers/iommu/dma-iommu.c for more than just arm64, e.g. also for the Intel/AMD iommus? I've been trying to consolidate the giant amounts of duplicate code in the dma ops implementations for a while, and iommus that provide an iommu_api driver seems like an easy enough target. I remember we had discussions about such things a while ago, so I wonder if its just that no one got to this yet or if there are any known issues that need to be solved first.
Quoted 10 lines · Christoph Hellwig
Hi all,
are there any road blocks for using
drivers/iommu/dma-iommu.cfor more than just arm64, e.g. also for the Intel/AMD iommus? I've been trying to consolidate the giant amounts of duplicate code in the dma ops implementations for a while, and iommus that provide aniommu_apidriver seems like an easy enough target. I remember we had discussions about such things a while ago, so I wonder if its just that no one got to this yet or if there are any known issues that need to be solved first.
dma-iommu does require that the IOMMU driver allows iommu_{un,}map() to be called in atomic context, but last time I looked I think it was only the AMD driver that needs attention there. The main other concern, for x86 at least, was the performance overhead of the extra level of API abstraction and indirection. However, dma-iommu is now on a par with all the IOVA optimisations, and once we've got the new IOMMU API TLB maintenance calls implemented and hooked up, the difference will hopefully be small enough to make the conversion viable.
It's been on my to-do list for ages to convert 32-bit ARM over to dma-iommu as well - some of the blockers like DMA configuration order have now been resolved, but there are still some awkward dependencies between IOMMU and DRM drivers that I need to get a handle on.
Robin.
Quoted 8 lines · Robin Murphy
dma-iommu does require that the IOMMU driver allows iommu_{un,}map() to be called in atomic context, but last time I looked I think it was only the AMD driver that needs attention there. The main other concern, for x86 at least, was the performance overhead of the extra level of API abstraction and indirection. However, dma-iommu is now on a par with all the IOVA optimisations, and once we've got the new IOMMU API TLB maintenance calls implemented and hooked up, the difference will hopefully be small enough to make the conversion viable.
Thanks, that's good news.
Quoted 4 lines
It's been on my to-do list for ages to convert 32-bit ARM over to dma-iommu as well - some of the blockers like DMA configuration order have now been resolved, but there are still some awkward dependencies between IOMMU and DRM drivers that I need to get a handle on.
Talking about arm32 - do you remember why it uses it's own dmabounce subsystem instead of swiotlb?
Quoted 12 lines · Christoph Hellwig
Quoted 8 lines · Robin Murphy
dma-iommu does require that the IOMMU driver allows iommu_{un,}map() to be called in atomic context, but last time I looked I think it was only the AMD driver that needs attention there. The main other concern, for x86 at least, was the performance overhead of the extra level of API abstraction and indirection. However, dma-iommu is now on a par with all the IOVA optimisations, and once we've got the new IOMMU API TLB maintenance calls implemented and hooked up, the difference will hopefully be small enough to make the conversion viable.
Thanks, that's good news.
Quoted 7 lines
Quoted 4 lines
It's been on my to-do list for ages to convert 32-bit ARM over to dma-iommu as well - some of the blockers like DMA configuration order have now been resolved, but there are still some awkward dependencies between IOMMU and DRM drivers that I need to get a handle on.
Talking about arm32 - do you remember why it uses it's own dmabounce subsystem instead of swiotlb?
IIRC, dmabounce has to stay because it handles non-contiguous DMA masks, thanks to a StrongARM hardware erratum where the DMA controller could only access every other megabyte of memory (or something like that).
Robin.
Quoted 3 lines · Robin Murphy
IIRC, dmabounce has to stay because it handles non-contiguous DMA masks, thanks to a StrongARM hardware erratum where the DMA controller could only access every other megabyte of memory (or something like that).
Isn't that exactly the problem that xen-swiotlb solves?
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Robin Murphy
IIRC, dmabounce has to stay because it handles non-contiguous DMA masks, thanks to a StrongARM hardware erratum where the DMA controller could only access every other megabyte of memory (or something like that).
Isn't that exactly the problem that xen-swiotlb solves?
As far as I can see, not quite - swiotlb-xen appears to handle buffers which span non-contiguous host pages, but still assumes that DMA masks are of the full DMA_BIT_MASK() form such that checks like "addr <= mask" work. I suppose we *could* try just making the SWIOTLB users themselves robust against incomplete masks (e.g. [2]) if they need to be, but I'm not sure whether that would actually work in practice.
FWIW, I could have sworn I had an old mail from Russell about this but that seems to have vanished - after a bit more digging it's actually about avoiding a particular address line on the SA-1111 as documented in [1], and set up in sa1111_configure_smc() (arch/arm/common/sa1111.c).
Robin.
[1] https://wwwcip.informatik.uni-erlangen.de/~simigern/jornada-7xx/docs/sa1111-errata.pdf
[2]
arch/arm64/include/asm/dma-mapping.h +4 -1
@@ -66,10 +66,13 @@ static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t dev_addr)static inline bool dma_capable (struct device *dev,dma_addr_t addr,size_t size) { +dma_addr_t mask; +if (!dev->dma_mask)return false; -return addr + size -1 <= *dev->dma_mask; + mask = *dev->dma_mask; +return !(addr + size -1 & ~mask) && (size -1 <= mask ^ (mask +1 )); }static inline void dma_mark_clean (void *addr,size_t size)