of: Set the DMA mask to 64 bits on ARM LPAE systems
PATCH 1/1 of: Set the DMA mask to 64 bits on ARM LPAE systems
By default on ARM systems, the coherent DMA mask (lowest address) is set to ~0 or 0xFFFFFFFFFFFFFFFF. Currently, of_platform_device_create_pdata sets the coherent DMA mask to 32 bits. This prevents coherent dma allocations from working by default without clients setting the DMA mask. Rather than make every client on an LPAE system set the mask, set the mask to a 64 bit value on ARM LPAE systems.
--- drivers/of/platform.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)
drivers/of/platform.c +4 -0
@@ -214,7 +214,11 @@ struct platform_device *of_platform_device_create_pdata(#if defined(CONFIG_MICROBLAZE ) dev->archdata.dma_mask =0xffffffffUL ;#endif +#ifdef CONFIG_ARM_LPAE + dev->dev.coherent_dma_mask =DMA_BIT_MASK (64 ); +#else dev->dev.coherent_dma_mask =DMA_BIT_MASK (32 ); +#endif dev->dev.bus = &platform_bus_type; dev->dev.platform_data = platform_data;
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
PATCH 1/1 of: Set the DMA mask to 64 bits on ARM LPAE systems
Quoted 18 lines · Laura Abbott
By default on ARM systems, the coherent DMA mask (lowest address) is set to ~0 or
0xFFFFFFFFFFFFFFFF. Currently,of_platform_device_create_pdatasets the coherent DMA mask to 32 bits. This prevents coherent dma allocations from working by default without clients setting the DMA mask. Rather than make every client on an LPAE system set the mask, set the mask to a 64 bit value on ARM LPAE systems.Signed-off-by Laura Abbott <[email protected]>--- drivers/of/platform.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)drivers/of/platform.c +0
-0
drivers/of/platform.c struct platform_device *of_platform_device_create_pdata( · line 221
Quoted 8 lines
drivers/of/platform.c +4
-0@@ -214,7 +214,11 @@ struct platform_device *of_platform_device_create_pdata(#if defined(CONFIG_MICROBLAZE ) dev->archdata.dma_mask =0xffffffffUL ;#endif +#ifdef CONFIG_ARM_LPAE + dev->dev.coherent_dma_mask =DMA_BIT_MASK (64 ); +#else dev->dev.coherent_dma_mask =DMA_BIT_MASK (32 ); +#endif
I wouldn't add CONFIG_ARM_LPAE checks in here, you can use CONFIG_ARCH_DMA_ADDR_T_64BIT (types.h uses this for the dma_addr_t definition).
Signature · 1 line
Catalin
PATCH 1/1 of: Set the DMA mask to 64 bits on ARM LPAE systems
Quoted 23 lines · Catalin Marinas
Quoted 22 lines · Laura Abbott
By default on ARM systems, the coherent DMA mask (lowest address) is set to ~0 or
0xFFFFFFFFFFFFFFFF. Currently,of_platform_device_create_pdatasets the coherent DMA mask to 32 bits. This prevents coherent dma allocations from working by default without clients setting the DMA mask. Rather than make every client on an LPAE system set the mask, set the mask to a 64 bit value on ARM LPAE systems.Signed-off-by Laura Abbott <[email protected]>--- drivers/of/platform.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)drivers/of/platform.c +1
-0@@ -214,7 +214,11 @@ struct platform_device *of_platform_device_create_pdata(#if defined(CONFIG_MICROBLAZE ) dev->archdata.dma_mask =0xffffffffUL ;#endif +#ifdef CONFIG_ARM_LPAE
Quoted 9 lines
drivers/of/platform.c struct platform_device *of_platform_device_create_pdata( · line 221
Quoted 4 lines
drivers/of/platform.c +3
-0+ dev->dev.coherent_dma_mask =DMA_BIT_MASK (64 ); +#else dev->dev.coherent_dma_mask =DMA_BIT_MASK (32 ); +#endif I wouldn't add
CONFIG_ARM_LPAEchecks in here, you can useCONFIG_ARCH_DMA_ADDR_T_64BIT(types.huses this for thedma_addr_tdefinition).
I thought about this as well but in arch/arm/mm/mm.h:
#ifdef CONFIG_ZONE_DMA
extern phys_addr_t arm_dma_limit;
#else
#define arm_dma_limit ((phys_addr_t)~0)
#endif
arm_dma_limit is explicitly cast to phys_addr_t, which means that arm_dma_limit will be always be sizeof(phys_addr_t) regardless of sizeof(dma_addr_t). Is it safe to assume that CONFIG_ARCH_DMA_ADDR_T_64BIT will always be selected if sizeof(phys_addr_t) == 8? If not, we've defeated the point of the patch.
Alternatively, should the type of arm_dma_limit be dma_addr_t instead of phys_addr_t?
Thanks,
Laura
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
PATCH 1/1 of: Set the DMA mask to 64 bits on ARM LPAE systems
Quoted 11 lines · Laura Abbott
I thought about this as well but in
arch/arm/mm/mm.h:#ifdef
CONFIG_ZONE_DMA
externphys_addr_tarm_dma_limit;
#else
#definearm_dma_limit((phys_addr_t)~0)
#endif
arm_dma_limitis explicitly cast tophys_addr_t, which means thatarm_dma_limitwill be always besizeof(phys_addr_t)regardless ofsizeof(dma_addr_t).
This is the size of the DMA zone, which controls the _minimum_ DMA mask that can be used in the system. DMA masks must always be greater than this limit.
PATCH 1/1 of: Set the DMA mask to 64 bits on ARM LPAE systems
In reply to Laura AbbottQuoted 34 lines · Laura Abbott
Quoted 23 lines · Catalin Marinas
Quoted 22 lines · Laura Abbott
By default on ARM systems, the coherent DMA mask (lowest address) is set to ~0 or
0xFFFFFFFFFFFFFFFF. Currently,of_platform_device_create_pdatasets the coherent DMA mask to 32 bits. This prevents coherent dma allocations from working by default without clients setting the DMA mask. Rather than make every client on an LPAE system set the mask, set the mask to a 64 bit value on ARM LPAE systems.Signed-off-by Laura Abbott <[email protected]>--- drivers/of/platform.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)drivers/of/platform.c +1
-0@@ -214,7 +214,11 @@ struct platform_device *of_platform_device_create_pdata(#if defined(CONFIG_MICROBLAZE ) dev->archdata.dma_mask =0xffffffffUL ;#endif +#ifdef CONFIG_ARM_LPAE Quoted 9 lines
drivers/of/platform.c struct platform_device *of_platform_device_create_pdata( · line 221
Quoted 4 lines
drivers/of/platform.c +3
-0+ dev->dev.coherent_dma_mask =DMA_BIT_MASK (64 ); +#else dev->dev.coherent_dma_mask =DMA_BIT_MASK (32 ); +#endif I wouldn't add
CONFIG_ARM_LPAEchecks in here, you can useCONFIG_ARCH_DMA_ADDR_T_64BIT(types.huses this for thedma_addr_tdefinition).
Quoted 7 lines
I thought about this as well but in
arch/arm/mm/mm.h:#ifdef
CONFIG_ZONE_DMA
externphys_addr_tarm_dma_limit;
#else
#definearm_dma_limit((phys_addr_t)~0)
#endif
Russell replied already on the meaning of this variable, so I don't think its type is relevant here. It is also used when calling dma_contiguous_reserve() which takes a phys_addr_t.
Quoted 5 lines
arm_dma_limitis explicitly cast tophys_addr_t, which means thatarm_dma_limitwill be always besizeof(phys_addr_t)regardless ofsizeof(dma_addr_t). Is it safe to assume thatCONFIG_ARCH_DMA_ADDR_T_64BITwill always be selected ifsizeof(phys_addr_t)== 8? If not, we've defeated the point of the patch.
sizeof(dma_addr_t) <= sizeof(phys_addr_t). I had this discussion with Russell when upstreaming the LPAE patches and the conclusion was that while the phys_addr_t is 64-bit with LPAE, the dma_addr_t size is a feature of the SoC and must be selected independently. However, I think with multi-platform support and LPAE enabled, we should probably set the dma_addr_t to 64-bit.
Quoted 2 lines
Alternatively, should the type of
arm_dma_limitbedma_addr_tinstead ofphys_addr_t?
arm_dma_limit should be phys_addr_t since this is an address as seen by the CPU. dma_addr_t OTOH is a bus address as seen by the device. You can in theory have a DMA buffer with physical address beyond 32-bit (from a CPU perspective) but with a bus address withing 32-bit as seen by the device.
There is some confusion in the kernel (you can google for past discussions) around the meaning of the DMA mask. If your RAM starts at 0, it doesn't matter much and there are several such assumptions in the kernel.
So I still consider that you should use #ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT and select this option on your platform.
Signature · 1 line
Catalin
PATCH 1/1 of: Set the DMA mask to 64 bits on ARM LPAE systems
Quoted 69 lines · Catalin Marinas
Quoted 34 lines · Laura Abbott
Quoted 23 lines · Catalin Marinas
Quoted 22 lines · Laura Abbott
By default on ARM systems, the coherent DMA mask (lowest address) is set to ~0 or
0xFFFFFFFFFFFFFFFF. Currently,of_platform_device_create_pdatasets the coherent DMA mask to 32 bits. This prevents coherent dma allocations from working by default without clients setting the DMA mask. Rather than make every client on an LPAE system set the mask, set the mask to a 64 bit value on ARM LPAE systems.Signed-off-by Laura Abbott <[email protected]>--- drivers/of/platform.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)drivers/of/platform.c +1
-0@@ -214,7 +214,11 @@ struct platform_device *of_platform_device_create_pdata(#if defined(CONFIG_MICROBLAZE ) dev->archdata.dma_mask =0xffffffffUL ;#endif +#ifdef CONFIG_ARM_LPAE Quoted 9 lines
drivers/of/platform.c struct platform_device *of_platform_device_create_pdata( · line 221
Quoted 4 lines
drivers/of/platform.c +3
-0+ dev->dev.coherent_dma_mask =DMA_BIT_MASK (64 ); +#else dev->dev.coherent_dma_mask =DMA_BIT_MASK (32 ); +#endif I wouldn't add
CONFIG_ARM_LPAEchecks in here, you can useCONFIG_ARCH_DMA_ADDR_T_64BIT(types.huses this for thedma_addr_tdefinition).Quoted 7 lines
I thought about this as well but in
arch/arm/mm/mm.h:#ifdef
CONFIG_ZONE_DMA
externphys_addr_tarm_dma_limit;
#else
#definearm_dma_limit((phys_addr_t)~0)
#endifRussell replied already on the meaning of this variable, so I don't think its type is relevant here. It is also used when calling
dma_contiguous_reserve()which takes aphys_addr_t.Quoted 5 lines
arm_dma_limitis explicitly cast tophys_addr_t, which means thatarm_dma_limitwill be always besizeof(phys_addr_t)regardless ofsizeof(dma_addr_t). Is it safe to assume thatCONFIG_ARCH_DMA_ADDR_T_64BITwill always be selected ifsizeof(phys_addr_t)== 8? If not, we've defeated the point of the patch.
sizeof(dma_addr_t)<=sizeof(phys_addr_t). I had this discussion with Russell when upstreaming the LPAE patches and the conclusion was that while thephys_addr_tis 64-bit with LPAE, thedma_addr_tsize is a feature of the SoC and must be selected independently. However, I think with multi-platform support and LPAE enabled, we should probably set thedma_addr_tto 64-bit.Quoted 2 lines
Alternatively, should the type of
arm_dma_limitbedma_addr_tinstead ofphys_addr_t?
arm_dma_limitshould bephys_addr_tsince this is an address as seen by the CPU.dma_addr_tOTOH is a bus address as seen by the device. You can in theory have a DMA buffer with physical address beyond 32-bit (from a CPU perspective) but with a bus address withing 32-bit as seen by the device.
Quoted 8 lines
There is some confusion in the kernel (you can google for past discussions) around the meaning of the DMA mask. If your RAM starts at 0, it doesn't matter much and there are several such assumptions in the kernel.
So I still consider that you should use #ifdef
CONFIG_ARCH_DMA_ADDR_T_64BITand select this option on your platform.
Thanks for the clarification on all this. It's non-obvious but I see why things are set up the way they are. I'll re-submit with CONFIG_ARCH_DMA_ADDR_T_64BIT.
Laura
Signature · 2 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation