Skip to contentSkip to conversation

arm64: Allow for different DMA and CPU bus offsets

PATCH Superseded Alexander Graf 5 participants 14 messages May 18, 2016 – Jun 7, 2016 arm64

lore
Share & tools
lore

Alexander Graf

author
#0 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Signed-off-by Alexander Graf <[email protected]>

---

This patch may conflict with another patch titled "swiotlb: prefix dma_to_phys and phys_to_dma functions" which is in flight, but hasn't seen an update since March.

Since this patch is very small and isolated to arm64, I'd prefer to keep them separate rather than combine them. So if the other patch gets accepted first, I'm happy to rebase this patch on top of a topic branch that would address the rename.

v1 -> v2:

- Use PAGE_SHIFT explicitly rather than __pfn_to_phys and __phys_to_pfn

---
 arch/arm64/include/asm/dma-mapping.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
arch/arm64/include/asm/dma-mapping.h +15 -2
index ba437f0..c14e9cd 100644
@@ -66,12 +66,16 @@ static inline bool is_device_dma_coherent(struct device *dev)
 
 static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	return (dma_addr_t)paddr;
+	dma_addr_t dev_addr = (dma_addr_t)paddr;
+
+	return dev_addr - ((dma_addr_t)dev->dma_pfn_offset << PAGE_SHIFT);
 }
 
 static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t dev_addr)
 {
-	return (phys_addr_t)dev_addr;
+	phys_addr_t paddr = (phys_addr_t)dev_addr;
+
+	return paddr + ((phys_addr_t)dev->dma_pfn_offset << PAGE_SHIFT);
 }
 
 static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)

@@ -86,5 +90,14 @@ static inline void dma_mark_clean(void *addr, size_t size)
 {
 }
 
+/* Override for dma_max_pfn() */
+static inline unsigned long dma_max_pfn(struct device *dev)
+{
+	dma_addr_t dma_max = (dma_addr_t)*dev->dma_mask;
+
+	return (ulong)dma_to_phys(dev, dma_max) >> PAGE_SHIFT;
+}
+#define dma_max_pfn(dev) dma_max_pfn(dev)
+
 #endif	/* __KERNEL__ */
 #endif	/* __ASM_DMA_MAPPING_H */
Signature · 1 line
1.8.5.6
#1 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

Quoted 10 lines · On Wednesday 18 May 2016 14:53:30 Alexander Graf wrote:

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Signed-off-by Alexander Graf <[email protected]>
Acked-by Arnd Bergmann <[email protected]>
maintainer
#2 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

In reply to Alexander Graf
Quoted 8 lines · Alexander Graf

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Signed-off-by Alexander Graf <[email protected]>

We were aware about this issue but we've been waiting for an actual use case. I'll queue this for 4.8 (given that RPi3 is not supported by the arm64 kernel, I don't see the rush to merge it earlier).

Thanks.

Signature · 1 line
Catalin
author
#3 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

Quoted 4 lines · Catalin Marinas
Quoted 3 lines · Alexander Graf

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

Quoted 8 lines
Quoted 5 lines

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Signed-off-by Alexander Graf <[email protected]>

We were aware about this issue but we've been waiting for an actual use case. I'll queue this for 4.8 (given that RPi3 is not supported by the arm64 kernel, I don't see the rush to merge it earlier).

Sure, works for me. That way Eric can try to push the other few bits necessary to run an arm64 kernel on the RPi3 towards 4.8 as well :).

Alex

maintainer
#4 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

Quoted 9 lines · Alexander Graf
Quoted 8 lines · Catalin Marinas
Quoted 7 lines · Alexander Graf

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Signed-off-by Alexander Graf <[email protected]>

We were aware about this issue but we've been waiting for an actual use case. I'll queue this for 4.8 (given that RPi3 is not supported by the arm64 kernel, I don't see the rush to merge it earlier).

Sure, works for me. That way Eric can try to push the other few bits necessary to run an arm64 kernel on the RPi3 towards 4.8 as well :).

As long as the firmware is PSCI capable ;)

Signature · 1 line
Catalin
author
#5 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

Quoted 10 lines · Catalin Marinas
Quoted 9 lines · Alexander Graf
Quoted 8 lines · Catalin Marinas
Quoted 7 lines · Alexander Graf

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Signed-off-by Alexander Graf <[email protected]>

We were aware about this issue but we've been waiting for an actual use case. I'll queue this for 4.8 (given that RPi3 is not supported by the arm64 kernel, I don't see the rush to merge it earlier).

Sure, works for me. That way Eric can try to push the other few bits necessary to run an arm64 kernel on the RPi3 towards 4.8 as well :).

As long as the firmware is PSCI capable ;)

So spin tables won't work for you? IIRC we don't own EL3 on there.

Alex

maintainer
#6 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

Quoted 12 lines · Alexander Graf
Quoted 11 lines · Catalin Marinas
Quoted 10 lines · Alexander Graf
Quoted 9 lines · Catalin Marinas
Quoted 8 lines · Alexander Graf

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Signed-off-by Alexander Graf <[email protected]>
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines

We were aware about this issue but we've been waiting for an actual use case. I'll queue this for 4.8 (given that RPi3 is not supported by the arm64 kernel, I don't see the rush to merge it earlier).

Sure, works for me. That way Eric can try to push the other few bits necessary to run an arm64 kernel on the RPi3 towards 4.8 as well :).

As long as the firmware is PSCI capable ;)

So spin tables won't work for you?

They only work for booting, not cpuidle or hotplug.

Quoted 1 line

IIRC we don't own EL3 on there.

Then how do you switch to AArch64?

Signature · 1 line
Catalin
author
#7 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

Quoted 13 lines · Catalin Marinas
Quoted 12 lines · Alexander Graf
Quoted 11 lines · Catalin Marinas
Quoted 10 lines · Alexander Graf
Quoted 9 lines · Catalin Marinas
Quoted 8 lines · Alexander Graf

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Signed-off-by Alexander Graf <[email protected]>
Quoted 8 lines
Quoted 7 lines
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines

We were aware about this issue but we've been waiting for an actual use case. I'll queue this for 4.8 (given that RPi3 is not supported by the arm64 kernel, I don't see the rush to merge it earlier).

Sure, works for me. That way Eric can try to push the other few bits necessary to run an arm64 kernel on the RPi3 towards 4.8 as well :).

As long as the firmware is PSCI capable ;)

So spin tables won't work for you?

They only work for booting, not cpuidle or hotplug.

Well, booting is a good step forward I'd say :).

Quoted 3 lines
Quoted 2 lines

IIRC we don't own EL3 on there.

Then how do you switch to AArch64?

You write into config.txt that you want to boot into AArch64 EL2 and the raspberry pi firmware takes care of the rest.

Alex

#8 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

Quoted 8 lines · On Wednesday 18 May 2016 17:33:41 Alexander Graf wrote:
Quoted 3 lines
Quoted 2 lines

IIRC we don't own EL3 on there.

Then how do you switch to AArch64?

You write into config.txt that you want to boot into AArch64 EL2 and the raspberry pi firmware takes care of the rest.

Have you tried telling the firmware that you'd like to boot into AArch64 EL3? ;-)

Arnd

author
#9 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

Quoted 8 lines · Arnd Bergmann
Quoted 6 lines · On Wednesday 18 May 2016 17:33:41 Alexander Graf wrote:
Quoted 2 lines
Quoted 1 line

IIRC we don't own EL3 on there.

Then how do you switch to AArch64?

You write into config.txt that you want to boot into AArch64 EL2 and the raspberry pi firmware takes care of the rest.

Have you tried telling the firmware that you'd like to boot into AArch64 EL3? ;-)

Yes, but then you don't get any SMP initialization (so we'd have to redo that in U-Boot, or port ATF) and you lose the usual workflow on defining how the hat device tree overlays work.

Overall, I think it's an interesting goal and you're more than happy to be my guest, but I think that spin tables are good enough for the device for now :). At least we'd be able to move people away from 32bit code.

Alex

#10 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

In reply to Catalin Marinas
Quoted 8 lines · Catalin Marinas
Quoted 7 lines · Alexander Graf

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Quoted 5 lines
Quoted 1 line
Signed-off-by Alexander Graf <[email protected]>

We were aware about this issue but we've been waiting for an actual use case. I'll queue this for 4.8 (given that RPi3 is not supported by the arm64 kernel, I don't see the rush to merge it earlier).

Sounds good to me. I'm looking forward to getting the other pieces in
place for 4.8.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20160518/d6a194aa/attachment.sig>

maintainer
#11 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

In reply to Catalin Marinas
Quoted 8 lines · Catalin Marinas
Quoted 7 lines · Alexander Graf

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Quoted 5 lines
Quoted 1 line
Signed-off-by Alexander Graf <[email protected]>

We were aware about this issue but we've been waiting for an actual use case. I'll queue this for 4.8 (given that RPi3 is not supported by the arm64 kernel, I don't see the rush to merge it earlier).

In case this patch goes in via the arm-soc tree (together with other RPi3 patches):

Acked-by Catalin Marinas <[email protected]>
#12 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

Quoted 11 lines · Catalin Marinas
Quoted 10 lines · Catalin Marinas
Quoted 8 lines · Alexander Graf

On arm64, all SoCs we supported so far either have an IOMMU or have bus addresses equal to CPU addresses.

However, with the Raspberry Pi 3 coming up, this is no longer true. To allow DMA to work with an AArch64 kernel on those devices, let's allow devices to have DMA offsets again.

Signed-off-by Alexander Graf <[email protected]>
Quoted 8 lines
Quoted 3 lines

We were aware about this issue but we've been waiting for an actual use case. I'll queue this for 4.8 (given that RPi3 is not supported by the arm64 kernel, I don't see the rush to merge it earlier).

In case this patch goes in via the arm-soc tree (together with other RPi3 patches):

Acked-by Catalin Marinas <[email protected]>

Thanks! I've pulled this into my for-next branch.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20160607/2c4ab37a/attachment-0001.sig>

#13 lore raw

PATCH v2 1/1 arm64: Allow for different DMA and CPU bus offsets

In reply to Alexander Graf
Quoted 5 lines · Alexander Graf

---

This patch may conflict with another patch titled "swiotlb: prefix dma_to_phys and phys_to_dma functions" which is in flight, but hasn't seen an update since March.

Thanks for the heads up. Due to missing API in DMA API, I couldn't make progress with the series.

I need dma_map_resource API to get merged. After that one of the drivers that actually uses dma_to_phys need to be changed to use dma_map_resource.

Then the rest of the series can come.

Signature · 3 lines
Sinan Kaya
Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project