Skip to contentSkip to conversation

road blocks for using dma-iommu on more than arm64

DISCUSSION Christoph Hellwig 2 participants 6 messages Oct 1, 2017 – Oct 2, 2017 5 messages not in the archive arm64

lore
Share & tools
lore
#0 lore raw

Hi all,

are there any road blocks for using drivers/iommu/dma-iommu.c for more than just arm64, e.g. also for the Intel/AMD iommus? I've been trying to consolidate the giant amounts of duplicate code in the dma ops implementations for a while, and iommus that provide an iommu_api driver seems like an easy enough target. I remember we had discussions about such things a while ago, so I wonder if its just that no one got to this yet or if there are any known issues that need to be solved first.

#2 lore raw
In reply to a message not in the archive
Quoted 10 lines · Christoph Hellwig

Hi all,

are there any road blocks for using drivers/iommu/dma-iommu.c for more than just arm64, e.g. also for the Intel/AMD iommus? I've been trying to consolidate the giant amounts of duplicate code in the dma ops implementations for a while, and iommus that provide an iommu_api driver seems like an easy enough target. I remember we had discussions about such things a while ago, so I wonder if its just that no one got to this yet or if there are any known issues that need to be solved first.

dma-iommu does require that the IOMMU driver allows iommu_{un,}map() to be called in atomic context, but last time I looked I think it was only the AMD driver that needs attention there. The main other concern, for x86 at least, was the performance overhead of the extra level of API abstraction and indirection. However, dma-iommu is now on a par with all the IOVA optimisations, and once we've got the new IOMMU API TLB maintenance calls implemented and hooked up, the difference will hopefully be small enough to make the conversion viable.

It's been on my to-do list for ages to convert 32-bit ARM over to dma-iommu as well - some of the blockers like DMA configuration order have now been resolved, but there are still some awkward dependencies between IOMMU and DRM drivers that I need to get a handle on.

Robin.

#4 lore raw
In reply to a message not in the archive
Quoted 8 lines · Robin Murphy

dma-iommu does require that the IOMMU driver allows iommu_{un,}map() to be called in atomic context, but last time I looked I think it was only the AMD driver that needs attention there. The main other concern, for x86 at least, was the performance overhead of the extra level of API abstraction and indirection. However, dma-iommu is now on a par with all the IOVA optimisations, and once we've got the new IOMMU API TLB maintenance calls implemented and hooked up, the difference will hopefully be small enough to make the conversion viable.

Thanks, that's good news.

Quoted 4 lines

It's been on my to-do list for ages to convert 32-bit ARM over to dma-iommu as well - some of the blockers like DMA configuration order have now been resolved, but there are still some awkward dependencies between IOMMU and DRM drivers that I need to get a handle on.

Talking about arm32 - do you remember why it uses it's own dmabounce subsystem instead of swiotlb?

#6 lore raw
In reply to a message not in the archive
Quoted 12 lines · Christoph Hellwig
Quoted 8 lines · Robin Murphy

dma-iommu does require that the IOMMU driver allows iommu_{un,}map() to be called in atomic context, but last time I looked I think it was only the AMD driver that needs attention there. The main other concern, for x86 at least, was the performance overhead of the extra level of API abstraction and indirection. However, dma-iommu is now on a par with all the IOVA optimisations, and once we've got the new IOMMU API TLB maintenance calls implemented and hooked up, the difference will hopefully be small enough to make the conversion viable.

Thanks, that's good news.

Quoted 7 lines
Quoted 4 lines

It's been on my to-do list for ages to convert 32-bit ARM over to dma-iommu as well - some of the blockers like DMA configuration order have now been resolved, but there are still some awkward dependencies between IOMMU and DRM drivers that I need to get a handle on.

Talking about arm32 - do you remember why it uses it's own dmabounce subsystem instead of swiotlb?

IIRC, dmabounce has to stay because it handles non-contiguous DMA masks, thanks to a StrongARM hardware erratum where the DMA controller could only access every other megabyte of memory (or something like that).

Robin.

#8 lore raw
In reply to a message not in the archive
Quoted 3 lines · Robin Murphy

IIRC, dmabounce has to stay because it handles non-contiguous DMA masks, thanks to a StrongARM hardware erratum where the DMA controller could only access every other megabyte of memory (or something like that).

Isn't that exactly the problem that xen-swiotlb solves?

#10 lore raw
In reply to a message not in the archive
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Robin Murphy

IIRC, dmabounce has to stay because it handles non-contiguous DMA masks, thanks to a StrongARM hardware erratum where the DMA controller could only access every other megabyte of memory (or something like that).

Isn't that exactly the problem that xen-swiotlb solves?

As far as I can see, not quite - swiotlb-xen appears to handle buffers which span non-contiguous host pages, but still assumes that DMA masks are of the full DMA_BIT_MASK() form such that checks like "addr <= mask" work. I suppose we *could* try just making the SWIOTLB users themselves robust against incomplete masks (e.g. [2]) if they need to be, but I'm not sure whether that would actually work in practice.

FWIW, I could have sworn I had an old mail from Russell about this but that seems to have vanished - after a bit more digging it's actually about avoiding a particular address line on the SA-1111 as documented in [1], and set up in sa1111_configure_smc() (arch/arm/common/sa1111.c).

Robin.

[1] https://wwwcip.informatik.uni-erlangen.de/~simigern/jornada-7xx/docs/sa1111-errata.pdf

[2]

arch/arm64/include/asm/dma-mapping.h +4 -1
index 0df756b24863..8db96d714517 100644
@@ -66,10 +66,13 @@ static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t dev_addr)
 
 static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
 {
+	dma_addr_t mask;
+
 	if (!dev->dma_mask)
 		return false;
 
-	return addr + size - 1 <= *dev->dma_mask;
+	mask = *dev->dma_mask;
+	return !(addr + size - 1 & ~mask) && (size - 1 <= mask ^ (mask + 1));
 }
 
 static inline void dma_mark_clean(void *addr, size_t size)