arm64: dts: marvell: armada-37xx: Increase PCIe IO size from 64 KiB to 1 MiB

Commit 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space") increased size of PCIe MEM to 127 MiB, which is the maximal possible size for allocated 128 MiB PCIe window. PCIe IO size in that commit was unchanged.

Armada 3720 PCIe controller supports 32-bit IO space mapping so it is possible to assign more than 64 KiB if address space for IO.

Currently controller has assigned 127 MiB + 64 KiB memory and therefore there is 960 KiB of unused memory. So assign it to IO space by increasing IO window from 64 KiB to 1 MiB.

DTS file armada-3720-turris-mox.dts already uses whole 128 MiB space, so only update comment about 32-bit IO space mapping.

Signed-off-by Pali Rohár <[email protected]>

Fixes 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe

MEM space")

---
 arch/arm64/boot/dts/marvell/armada-3720-turris-mox.dts | 2 +-
 arch/arm64/boot/dts/marvell/armada-37xx.dtsi           | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

arch/arm64/boot/dts/marvell/armada-3720-turris-mox.dts +1 -1

index dd01409d4bb7..23e1b07c060a 100644
@@ -153,7 +153,7 @@
 	 * 2 size cells and also expects that the second range starts at 16 MB offset. If these
 	 * conditions are not met then U-Boot crashes during loading kernel DTB file. PCIe address
 	 * space is 128 MB long, so the best split between MEM and IO is to use fixed 16 MB window
-	 * for IO and the rest 112 MB (64+32+16) for MEM, despite that maximal IO size is just 64 kB.
+	 * for IO and the rest 112 MB (64+32+16) for MEM. Controller supports 32-bit IO mapping.
 	 * This bug is not present in U-Boot ports for other Armada 3700 devices and is fixed in
 	 * U-Boot version 2021.07. See relevant U-Boot commits (the last one contains fix):
 	 * https://source.denx.de/u-boot/u-boot/-/commit/cb2ddb291ee6fcbddd6d8f4ff49089dfe580f5d7

arch/arm64/boot/dts/marvell/armada-37xx.dtsi +2 -2

index 1c74f02535c6..8d59eabadce6 100644
@@ -508,12 +508,12 @@
 			/*
 			 * The 128 MiB address range [0xe8000000-0xf0000000] is
 			 * dedicated for PCIe and can be assigned to 8 windows
-			 * with size a power of two. Use one 64 KiB window for
+			 * with size a power of two. Use one 1 MiB window for
 			 * IO at the end and the remaining seven windows
 			 * (totaling 127 MiB) for MEM.
 			 */
 			ranges = <0x82000000 0 0xe8000000   0 0xe8000000   0 0x07f00000   /* Port 0 MEM */
-				  0x81000000 0 0xefff0000   0 0xefff0000   0 0x00010000>; /* Port 0 IO */
+				  0x81000000 0 0xeff00000   0 0xeff00000   0 0x00100000>; /* Port 0 IO*/
 			interrupt-map-mask = <0 0 0 7>;
 			interrupt-map = <0 0 0 1 &pcie_intc 0>,
 					<0 0 0 2 &pcie_intc 1>,

Signature · 1 line

2.20.1

Commit 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space") increased size of PCIe MEM to 127 MiB, which is the maximal possible size for allocated 128 MiB PCIe window. PCIe IO size in that commit was unchanged.

Armada 3720 PCIe controller supports 32-bit IO space mapping so it is possible to assign more than 64 KiB if address space for IO.

Currently controller has assigned 127 MiB + 64 KiB memory and therefore there is 960 KiB of unused memory. So assign it to IO space by increasing IO window from 64 KiB to 1 MiB.

DTS file armada-3720-turris-mox.dts already uses whole 128 MiB space, so only update comment about 32-bit IO space mapping.

Signed-off-by Pali Rohár <[email protected]>

Fixes 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space")

Quoted 13 lines · Pali Rohár

Commit 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space") increased size of PCIe MEM to 127 MiB, which is the maximal possible size for allocated 128 MiB PCIe window. PCIe IO size in that commit was unchanged.

Armada 3720 PCIe controller supports 32-bit IO space mapping so it is possible to assign more than 64 KiB if address space for IO.

Currently controller has assigned 127 MiB + 64 KiB memory and therefore there is 960 KiB of unused memory. So assign it to IO space by increasing IO window from 64 KiB to 1 MiB.

Quoted 5 lines

DTS file armada-3720-turris-mox.dts already uses whole 128 MiB space, so only update comment about 32-bit IO space mapping.

Signed-off-by Pali Rohár <[email protected]>

Fixes 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space")

I just saw this is the fixes pull request, and it seems very odd. Does this fix an actual bug?

Note that Linux normally doesn't map more than 64KB
of I/O space per PCI domain, so it should not make a difference to us.

Also, note that having a high bus address for the I/O space (0xefff0000, as as the CPU physical address here) means that a lot of the older devices that actually require I/O space won't work, because they need a low bus address in the first few KB.

Is this mapping a requirement from a broken bootloader, or can you change the mapping of the I/O port window in the physical space to the usual bus address 0?

Arnd

Quoted 17 lines · On Friday 18 February 2022 17:43:04 Arnd Bergmann wrote:

Quoted 16 lines · Pali Rohár

Commit 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space") increased size of PCIe MEM to 127 MiB, which is the maximal possible size for allocated 128 MiB PCIe window. PCIe IO size in that commit was unchanged.

Armada 3720 PCIe controller supports 32-bit IO space mapping so it is possible to assign more than 64 KiB if address space for IO.

Currently controller has assigned 127 MiB + 64 KiB memory and therefore there is 960 KiB of unused memory. So assign it to IO space by increasing IO window from 64 KiB to 1 MiB.

DTS file armada-3720-turris-mox.dts already uses whole 128 MiB space, so only update comment about 32-bit IO space mapping.

Quoted 5 lines

Quoted 2 lines

Signed-off-by Pali Rohár <[email protected]>

Fixes 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space")

I just saw this is the fixes pull request, and it seems very odd. Does this fix an actual bug?

Do you mean this patch or commit 514ef1e62d65?

Quoted 2 lines

Note that Linux normally doesn't map more than 64KB
of I/O space per PCI domain, so it should not make a difference to us.

Last time I looked into ARM code, it can allocate more than 64 kB for IO.

Quoted 5 lines

Also, note that having a high bus address for the I/O space (0xefff0000, as as the CPU physical address here) means that a lot of the older devices that actually require I/O space won't work, because they need a low bus address in the first few KB.

Quoted 3 lines

Is this mapping a requirement from a broken bootloader, or can you change the mapping of the I/O port window in the physical space to the usual bus address 0?

At physical address 0x0 it is not possible as at this address is mapped DDR.

ARM Trusted-Firmware sets PCIe space to range [0xe8000000-0xf0000000]. This (default) configuration is specified in DTS file. Which parts of this range is used for IO and which MEM is up to the a3720 PCIe kernel driver and currently it configures it based on how sub-ranges are specified in DT.

In some cases (e.g. when board has 4 GB of RAM), TF-A relocates this PCIe range to different location (otherwise it cannot activate more than 2 GB of RAM) and U-Boot during loading of kernel DTB file, is patching it.

It could be possible to change TF-A code to move PCIe space to different location (from [0xe8000000-0xf0000000]) but not to 0x0. But changing it means to move other parts and invent mapping in which most of RAM can be mapped to...

Quoted 21 lines · Pali Rohár

Quoted 20 lines · On Friday 18 February 2022 17:43:04 Arnd Bergmann wrote:

Quoted 18 lines · Pali Rohár

Commit 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space") increased size of PCIe MEM to 127 MiB, which is the maximal possible size for allocated 128 MiB PCIe window. PCIe IO size in that commit was unchanged.

Armada 3720 PCIe controller supports 32-bit IO space mapping so it is possible to assign more than 64 KiB if address space for IO.

Currently controller has assigned 127 MiB + 64 KiB memory and therefore there is 960 KiB of unused memory. So assign it to IO space by increasing IO window from 64 KiB to 1 MiB.

DTS file armada-3720-turris-mox.dts already uses whole 128 MiB space, so only update comment about 32-bit IO space mapping.

Signed-off-by Pali Rohár <[email protected]>

Fixes 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space")

Quoted 4 lines

Quoted 2 lines

I just saw this is the fixes pull request, and it seems very odd. Does this fix an actual bug?

Do you mean this patch or commit 514ef1e62d65?

This one. 514ef1e62d65 looks fine.

Quoted 16 lines

Quoted 2 lines

Note that Linux normally doesn't map more than 64KB
of I/O space per PCI domain, so it should not make a difference to us.

Last time I looked into ARM code, it can allocate more than 64 kB for IO.

Quoted 8 lines

Also, note that having a high bus address for the I/O space (0xefff0000, as as the CPU physical address here) means that a lot of the older devices that actually require I/O space won't work, because they need a low bus address in the first few KB.

Is this mapping a requirement from a broken bootloader, or can you change the mapping of the I/O port window in the physical space to the usual bus address 0?

Quoted 2 lines

At physical address 0x0 it is not possible as at this address is mapped DDR.

I meant bus address 0, not CPU physical address 0 of course. We don't care where in physical space the I/O window is.

Quoted 11 lines

ARM Trusted-Firmware sets PCIe space to range [0xe8000000-0xf0000000]. This (default) configuration is specified in DTS file. Which parts of this range is used for IO and which MEM is up to the a3720 PCIe kernel driver and currently it configures it based on how sub-ranges are specified in DT.

In some cases (e.g. when board has 4 GB of RAM), TF-A relocates this PCIe range to different location (otherwise it cannot activate more than 2 GB of RAM) and U-Boot during loading of kernel DTB file, is patching it.

Quoted 4 lines

It could be possible to change TF-A code to move PCIe space to different location (from [0xe8000000-0xf0000000]) but not to 0x0. But changing it means to move other parts and invent mapping in which most of RAM can be mapped to...

Can't you change the mapping to have a bus address that is different the physical address?

Arnd

Quoted 13 lines · Pali Rohár

Commit 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space") increased size of PCIe MEM to 127 MiB, which is the maximal possible size for allocated 128 MiB PCIe window. PCIe IO size in that commit was unchanged.

Armada 3720 PCIe controller supports 32-bit IO space mapping so it is possible to assign more than 64 KiB if address space for IO.

Currently controller has assigned 127 MiB + 64 KiB memory and therefore there is 960 KiB of unused memory. So assign it to IO space by increasing IO window from 64 KiB to 1 MiB.

Quoted 5 lines

DTS file armada-3720-turris-mox.dts already uses whole 128 MiB space, so only update comment about 32-bit IO space mapping.

Signed-off-by Pali Rohár <[email protected]>

Fixes 514ef1e62d65 ("arm64: dts: marvell: armada-37xx: Extend PCIe MEM space")

I just saw this is the fixes pull request, and it seems very odd. Does this fix an actual bug? Note that Linux normally doesn't map more than 64KB of I/O space per PCI domain, so it should not make a difference to us.

Also, note that having a high bus address for the I/O space (0xefff0000, as as the CPU physical address here) means that a lot of the older devices that actually require I/O space won't work, because they need a low bus address in the first few KB.

Is this mapping a requirement from a broken bootloader, or can you change the mapping of the I/O port window in the physical space to the usual bus address 0?

Arnd