Skip to contentSkip to conversation

New config added to handle 64-bit systems with 32-bit DMA support

PATCH In review Alberto Secondi 3 participants 7 messages Jun 19, 2024

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 New config added to handle 64-bit systems with 32-bit DMA support

From: Alberto Secondi <[email protected]>

The kernel assumes that 64-bit systems have 64-bit DMA support through CONFIG_ARCH_DMA_ADDR_T_64BIT. This is not always true; for example, several iMX8 systems (verified on iMX8MM and iMX8MP) have DMA with only 32-bit support. This results in several drivers requesting DMA_BIT_MASK(64), which causes malfunctions, particularly when systems have more than 3GB of DRAM (verified with the lan743x driver and iMX8 systems with 4GB of DRAM). Therefore, a new config ARCH_64BIT_HAS_DMA32_ONLY was added to manage 64-bit systems with 32-bit DMA, which adjusts DMA_BIT_MASK(n) accordingly.

---
 include/linux/dma-mapping.h | 4 ++++
 kernel/dma/Kconfig          | 8 ++++++++
 2 files changed, 12 insertions(+)
include/linux/dma-mapping.h +4 -0
index f693aafe221f..629220a777e3 100644
@@ -74,7 +74,11 @@
  */
 #define DMA_MAPPING_ERROR		(~(dma_addr_t)0)
 
+#ifdef CONFIG_ARCH_64BIT_HAS_DMA32_ONLY
+#define DMA_BIT_MASK(n)	(((n) > 32) ? ((1ULL<<(32))-1) : ((1ULL<<(n))-1))
+#else
 #define DMA_BIT_MASK(n)	(((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
+#endif
 
 #ifdef CONFIG_DMA_API_DEBUG
 void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr);
kernel/dma/Kconfig +8 -0
index c06e56be0ca1..0a27eafed808 100644
@@ -36,6 +36,14 @@ config NEED_DMA_MAP_STATE
 config ARCH_DMA_ADDR_T_64BIT
 	def_bool 64BIT || PHYS_ADDR_T_64BIT
 
+config ARCH_64BIT_HAS_DMA32_ONLY
+        bool "64bit System has DMA32 only"
+        depends on ARCH_DMA_ADDR_T_64BIT
+        default n
+	help
+	  This enables forcing the maximum DMA_BIT_MASK to 32 bits for
+	  64-bit systems that have DMA support limited to 32 bits.
+
 config ARCH_HAS_DMA_SET_MASK
 	bool
 
Signature · 1 line
2.34.1
#1 lore raw

Hi Alberto

Quoted 24 lines · Alberto Secondi

From: Alberto Secondi <[email protected]>

The kernel assumes that 64-bit systems have 64-bit DMA support through CONFIG_ARCH_DMA_ADDR_T_64BIT. This is not always true; for example, several iMX8 systems (verified on iMX8MM and iMX8MP) have DMA with only 32-bit support. This results in several drivers requesting DMA_BIT_MASK(64), which causes malfunctions, particularly when systems have more than 3GB of DRAM (verified with the lan743x driver and iMX8 systems with 4GB of DRAM). Therefore, a new config ARCH_64BIT_HAS_DMA32_ONLY was added to manage 64-bit systems with 32-bit DMA, which adjusts DMA_BIT_MASK(n) accordingly.

---
 include/linux/dma-mapping.h | 4 ++++
 kernel/dma/Kconfig          | 8 ++++++++
 2 files changed, 12 insertions(+)
include/linux/dma-mapping.h +0 -0
index f693aafe221f..629220a777e3 100644

include/linux/dma-mapping.h line 82

Quoted 9 lines
include/linux/dma-mapping.h +4 -0
@@ -74,7 +74,11 @@
  */
 #define DMA_MAPPING_ERROR              (~(dma_addr_t)0)
 
+#ifdef CONFIG_ARCH_64BIT_HAS_DMA32_ONLY
+#define DMA_BIT_MASK(n)        (((n) > 32) ? ((1ULL<<(32))-1) : ((1ULL<<(n))-1))
+#else
 #define DMA_BIT_MASK(n)        (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
+#endif
 

How can this fit configuration where you want to have one Kernel image for several arm64 machine?

Michael

Quoted 25 lines
include/linux/dma-mapping.h +0 -0
 #ifdef CONFIG_DMA_API_DEBUG
 void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr);
kernel/dma/Kconfig +8 -1
index c06e56be0ca1..0a27eafed808 100644
@@ -36,6 +36,14 @@ config NEED_DMA_MAP_STATE
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT || PHYS_ADDR_T_64BIT
 
+config ARCH_64BIT_HAS_DMA32_ONLY
+        bool "64bit System has DMA32 only"
+        depends on ARCH_DMA_ADDR_T_64BIT
+        default n
+       help
+         This enables forcing the maximum DMA_BIT_MASK to 32 bits for
+         64-bit systems that have DMA support limited to 32 bits.
+
 config ARCH_HAS_DMA_SET_MASK
        bool
 
--

2.34.1

#2 lore raw

Hi Alberto

Quoted 32 lines · Michael Nazzareno Trimarchi

Hi Alberto

Quoted 27 lines · Alberto Secondi

From: Alberto Secondi <[email protected]>

The kernel assumes that 64-bit systems have 64-bit DMA support through CONFIG_ARCH_DMA_ADDR_T_64BIT. This is not always true; for example, several iMX8 systems (verified on iMX8MM and iMX8MP) have DMA with only 32-bit support. This results in several drivers requesting DMA_BIT_MASK(64), which causes malfunctions, particularly when systems have more than 3GB of DRAM (verified with the lan743x driver and iMX8 systems with 4GB of DRAM). Therefore, a new config ARCH_64BIT_HAS_DMA32_ONLY was added to manage 64-bit systems with 32-bit DMA, which adjusts DMA_BIT_MASK(n) accordingly.

---
 include/linux/dma-mapping.h | 4 ++++
 kernel/dma/Kconfig          | 8 ++++++++
 2 files changed, 12 insertions(+)
include/linux/dma-mapping.h +0 -0
index f693aafe221f..629220a777e3 100644
@@ -74,7 +74,11 @@
  */
 #define DMA_MAPPING_ERROR              (~(dma_addr_t)0)
 
Quoted 10 lines

include/linux/dma-mapping.h line 82

Quoted 6 lines
include/linux/dma-mapping.h +4 -0
+#ifdef CONFIG_ARCH_64BIT_HAS_DMA32_ONLY
+#define DMA_BIT_MASK(n)        (((n) > 32) ? ((1ULL<<(32))-1) : ((1ULL<<(n))-1))
+#else
 #define DMA_BIT_MASK(n)        (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
+#endif
 

How can this fit configuration where you want to have one Kernel image for several arm64 machine?

I'm not an expert here but:

@coherent_dma_mask: Like dma_mask, but for alloc_coherent mapping as not all
* hardware supports 64-bit addresses for consistent allocations
* such descriptors.

This looks more connected to what you are looking for.

Michael

Quoted 27 lines

Michael

Quoted 25 lines
include/linux/dma-mapping.h +0 -0
 #ifdef CONFIG_DMA_API_DEBUG
 void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr);
kernel/dma/Kconfig +8 -1
index c06e56be0ca1..0a27eafed808 100644
@@ -36,6 +36,14 @@ config NEED_DMA_MAP_STATE
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT || PHYS_ADDR_T_64BIT
 
+config ARCH_64BIT_HAS_DMA32_ONLY
+        bool "64bit System has DMA32 only"
+        depends on ARCH_DMA_ADDR_T_64BIT
+        default n
+       help
+         This enables forcing the maximum DMA_BIT_MASK to 32 bits for
+         64-bit systems that have DMA support limited to 32 bits.
+
 config ARCH_HAS_DMA_SET_MASK
        bool
 
--

2.34.1

reviewer
#3 lore raw
In reply to Alberto Secondi
Quoted 10 lines · Alberto Secondi

------ Tessian Warning ------

Be careful, the email's sending address "albertosecondi@gmail[.]com" has never been seen on your company's network before today

This warning message will be removed if you reply to or forward this email to a recipient outside of your organization.

---- Tessian Warning End ----

From: Alberto Secondi <[email protected]>

Quoted 8 lines

The kernel assumes that 64-bit systems have 64-bit DMA support through CONFIG_ARCH_DMA_ADDR_T_64BIT. This is not always true; for example, several iMX8 systems (verified on iMX8MM and iMX8MP) have DMA with only 32-bit support. This results in several drivers requesting DMA_BIT_MASK(64), which causes malfunctions, particularly when systems have more than 3GB of DRAM (verified with the lan743x driver and iMX8 systems with 4GB of DRAM). Therefore, a new config ARCH_64BIT_HAS_DMA32_ONLY was added to manage 64-bit systems with 32-bit DMA, which adjusts DMA_BIT_MASK(n) accordingly.

No. If a system has devices naturally capable of >32-bit DMA, and memory at >32-bit system physical addresses, but only a 32-bit interconnect in between, that needs to be described properly in Devicetree/ACPI, not hacked around with completely non-portable kernel bodges.

Thanks,
Robin.

Quoted 42 lines
---
  include/linux/dma-mapping.h | 4 ++++
  kernel/dma/Kconfig          | 8 ++++++++
  2 files changed, 12 insertions(+)
include/linux/dma-mapping.h +4 -0
index f693aafe221f..629220a777e3 100644
@@ -74,7 +74,11 @@
   */
  #define DMA_MAPPING_ERROR		(~(dma_addr_t)0)
 
+#ifdef CONFIG_ARCH_64BIT_HAS_DMA32_ONLY
+#define DMA_BIT_MASK(n)	(((n) > 32) ? ((1ULL<<(32))-1) : ((1ULL<<(n))-1))
+#else
  #define DMA_BIT_MASK(n)	(((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
+#endif
 
  #ifdef CONFIG_DMA_API_DEBUG
  void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr);
kernel/dma/Kconfig +8 -0
index c06e56be0ca1..0a27eafed808 100644
@@ -36,6 +36,14 @@ config NEED_DMA_MAP_STATE
  config ARCH_DMA_ADDR_T_64BIT
  	def_bool 64BIT || PHYS_ADDR_T_64BIT
 
+config ARCH_64BIT_HAS_DMA32_ONLY
+        bool "64bit System has DMA32 only"
+        depends on ARCH_DMA_ADDR_T_64BIT
+        default n
+	help
+	  This enables forcing the maximum DMA_BIT_MASK to 32 bits for
+	  64-bit systems that have DMA support limited to 32 bits.
+
  config ARCH_HAS_DMA_SET_MASK
  	bool
 
Reviewed-by Frieder Schrempf Tested-by Frieder Schrempf
#4 lore raw

Hi Robin

Quoted 21 lines · Robin Murphy
Quoted 10 lines · Alberto Secondi

------ Tessian Warning ------

Be careful, the email's sending address "albertosecondi@gmail[.]com" has never been seen on your company's network before today

This warning message will be removed if you reply to or forward this email to a recipient outside of your organization.

---- Tessian Warning End ----

From: Alberto Secondi <[email protected]>

Quoted 8 lines

The kernel assumes that 64-bit systems have 64-bit DMA support through CONFIG_ARCH_DMA_ADDR_T_64BIT. This is not always true; for example, several iMX8 systems (verified on iMX8MM and iMX8MP) have DMA with only 32-bit support. This results in several drivers requesting DMA_BIT_MASK(64), which causes malfunctions, particularly when systems have more than 3GB of DRAM (verified with the lan743x driver and iMX8 systems with 4GB of DRAM). Therefore, a new config ARCH_64BIT_HAS_DMA32_ONLY was added to manage 64-bit systems with 32-bit DMA, which adjusts DMA_BIT_MASK(n) accordingly.

Quoted 5 lines

No. If a system has devices naturally capable of >32-bit DMA, and memory at >32-bit system physical addresses, but only a 32-bit interconnect in between, that needs to be described properly in Devicetree/ACPI, not hacked around with completely non-portable kernel bodges.

commit 4251a3ac4de9625a284a9c046cc915487e9b2a5e

Author: Lucas Stach <[email protected]>
Date:   Tue May 4 10:20:51 2021 +0200

arm64: dts: imx8mm: specify dma-ranges

DMA addressing capabilities on i.MX8MM are limited by the interconnect, same as on i.MX8MQ. Add dma-ranges to the the peripheral bus to let the kernel know about this.

(quoted diff) +0 -0

b/arch/arm64/boot/dts/freescale/imx8mm.dtsi
index 64aa38fd2b6e0..e7648c3b83905 100644

arch/arm64/boot/dts/freescale/imx8mm.dtsi +1 -0
@@ -261,6 +261,7 @@ soc@0 {
                #address-cells = <1>;
                #size-cells = <1>;
                ranges = <0x0 0x0 0x0 0x3e000000>;
+               dma-ranges = <0x40000000 0x0 0x40000000 0xc0000000>;
                nvmem-cells = <&imx8mm_uid>;
                nvmem-cell-names = "soc_unique_id";
 

Somenthing like this should already do it?

Michael

Quoted 46 lines

Thanks,
Robin.

Quoted 42 lines
---
  include/linux/dma-mapping.h | 4 ++++
  kernel/dma/Kconfig          | 8 ++++++++
  2 files changed, 12 insertions(+)
include/linux/dma-mapping.h +4 -0
index f693aafe221f..629220a777e3 100644
@@ -74,7 +74,11 @@
   */
  #define DMA_MAPPING_ERROR           (~(dma_addr_t)0)
 
+#ifdef CONFIG_ARCH_64BIT_HAS_DMA32_ONLY
+#define DMA_BIT_MASK(n)      (((n) > 32) ? ((1ULL<<(32))-1) : ((1ULL<<(n))-1))
+#else
  #define DMA_BIT_MASK(n)     (((n) == 64) ? ~0ULL : ((1ULL<<(n))-1))
+#endif
 
  #ifdef CONFIG_DMA_API_DEBUG
  void debug_dma_mapping_error(struct device *dev, dma_addr_t dma_addr);
kernel/dma/Kconfig +8 -0
index c06e56be0ca1..0a27eafed808 100644
@@ -36,6 +36,14 @@ config NEED_DMA_MAP_STATE
  config ARCH_DMA_ADDR_T_64BIT
      def_bool 64BIT || PHYS_ADDR_T_64BIT
 
+config ARCH_64BIT_HAS_DMA32_ONLY
+        bool "64bit System has DMA32 only"
+        depends on ARCH_DMA_ADDR_T_64BIT
+        default n
+     help
+       This enables forcing the maximum DMA_BIT_MASK to 32 bits for
+       64-bit systems that have DMA support limited to 32 bits.
+
  config ARCH_HAS_DMA_SET_MASK
      bool
 
reviewer
#5 lore raw
Quoted 59 lines · Michael Nazzareno Trimarchi

Hi Robin

Quoted 21 lines · Robin Murphy
Quoted 10 lines · Alberto Secondi

------ Tessian Warning ------

Be careful, the email's sending address "albertosecondi@gmail[.]com" has never been seen on your company's network before today

This warning message will be removed if you reply to or forward this email to a recipient outside of your organization.

---- Tessian Warning End ----

From: Alberto Secondi <[email protected]>

Quoted 8 lines

The kernel assumes that 64-bit systems have 64-bit DMA support through CONFIG_ARCH_DMA_ADDR_T_64BIT. This is not always true; for example, several iMX8 systems (verified on iMX8MM and iMX8MP) have DMA with only 32-bit support. This results in several drivers requesting DMA_BIT_MASK(64), which causes malfunctions, particularly when systems have more than 3GB of DRAM (verified with the lan743x driver and iMX8 systems with 4GB of DRAM). Therefore, a new config ARCH_64BIT_HAS_DMA32_ONLY was added to manage 64-bit systems with 32-bit DMA, which adjusts DMA_BIT_MASK(n) accordingly.

Quoted 5 lines

No. If a system has devices naturally capable of >32-bit DMA, and memory at >32-bit system physical addresses, but only a 32-bit interconnect in between, that needs to be described properly in Devicetree/ACPI, not hacked around with completely non-portable kernel bodges.

commit 4251a3ac4de9625a284a9c046cc915487e9b2a5e

Author: Lucas Stach <[email protected]>
Date:   Tue May 4 10:20:51 2021 +0200

arm64: dts: imx8mm: specify dma-ranges

DMA addressing capabilities on i.MX8MM are limited by the interconnect, same as on i.MX8MQ. Add dma-ranges to the the peripheral bus to let the kernel know about this.

arch/arm64/boot/dts/freescale/imx8mm.dtsi +1 -0
index 64aa38fd2b6e0..e7648c3b83905 100644
@@ -261,6 +261,7 @@ soc@0 {
                 #address-cells = <1>;
                 #size-cells = <1>;
                 ranges = <0x0 0x0 0x0 0x3e000000>;
+               dma-ranges = <0x40000000 0x0 0x40000000 0xc0000000>;
                 nvmem-cells = <&imx8mm_uid>;
                 nvmem-cell-names = "soc_unique_id";
 
Quoted 1 line

Somenthing like this should already do it?

Hmm, indeed... so the question to dig into would be why that's apparently not getting picked up by of_dma_get_range() for that device.

Thanks,
Robin.

#6 lore raw

Hi Robin

Quoted 62 lines · Robin Murphy
Quoted 59 lines · Michael Nazzareno Trimarchi

Hi Robin

Quoted 21 lines · Robin Murphy
Quoted 10 lines · Alberto Secondi

------ Tessian Warning ------

Be careful, the email's sending address "albertosecondi@gmail[.]com" has never been seen on your company's network before today

This warning message will be removed if you reply to or forward this email to a recipient outside of your organization.

---- Tessian Warning End ----

From: Alberto Secondi <[email protected]>

Quoted 8 lines

The kernel assumes that 64-bit systems have 64-bit DMA support through CONFIG_ARCH_DMA_ADDR_T_64BIT. This is not always true; for example, several iMX8 systems (verified on iMX8MM and iMX8MP) have DMA with only 32-bit support. This results in several drivers requesting DMA_BIT_MASK(64), which causes malfunctions, particularly when systems have more than 3GB of DRAM (verified with the lan743x driver and iMX8 systems with 4GB of DRAM). Therefore, a new config ARCH_64BIT_HAS_DMA32_ONLY was added to manage 64-bit systems with 32-bit DMA, which adjusts DMA_BIT_MASK(n) accordingly.

Quoted 5 lines

No. If a system has devices naturally capable of >32-bit DMA, and memory at >32-bit system physical addresses, but only a 32-bit interconnect in between, that needs to be described properly in Devicetree/ACPI, not hacked around with completely non-portable kernel bodges.

commit 4251a3ac4de9625a284a9c046cc915487e9b2a5e

Author: Lucas Stach <[email protected]>
Date:   Tue May 4 10:20:51 2021 +0200

arm64: dts: imx8mm: specify dma-ranges

DMA addressing capabilities on i.MX8MM are limited by the interconnect, same as on i.MX8MQ. Add dma-ranges to the the peripheral bus to let the kernel know about this.

arch/arm64/boot/dts/freescale/imx8mm.dtsi +1 -0
index 64aa38fd2b6e0..e7648c3b83905 100644
@@ -261,6 +261,7 @@ soc@0 {
                 #address-cells = <1>;
                 #size-cells = <1>;
                 ranges = <0x0 0x0 0x0 0x3e000000>;
+               dma-ranges = <0x40000000 0x0 0x40000000 0xc0000000>;
                 nvmem-cells = <&imx8mm_uid>;
                 nvmem-cell-names = "soc_unique_id";
 
Quoted 5 lines
Quoted 1 line

Somenthing like this should already do it?

Hmm, indeed... so the question to dig into would be why that's apparently not getting picked up by of_dma_get_range() for that device.

The patch does not include what kernel version was tested. After we have this answer that
we can know

Michael

Quoted 2 lines

Thanks,
Robin.

Signature · 11 lines
Michael Nazzareno Trimarchi
Co-Founder & Chief Executive Officer
M. +39 347 913 2170
[email protected]
__________________________________

Amarula Solutions BV
Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
T. +31 (0)85 111 9172
[email protected]
www.amarulasolutions.com