Skip to contentSkip to conversation

arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

PATCH Applied Jonathan Marek 6 participants 18 messages Nov 27, 2025 – Jan 16

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Fixes af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
---
 arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/qcom/x1e80100.dtsi +2 -2
index cff34d1c74b60..cd34ce5dfd63a 100644
@@ -792,8 +792,8 @@ soc: soc@0 {
 
 		#address-cells = <2>;
 		#size-cells = <2>;
-		dma-ranges = <0 0 0 0 0x10 0>;
-		ranges = <0 0 0 0 0x10 0>;
+		dma-ranges = <0 0 0 0 0x100 0>;
+		ranges = <0 0 0 0 0x100 0>;
 
 		gcc: clock-controller@100000 {
 			compatible = "qcom,x1e80100-gcc";
Signature · 1 line
2.51.0
#1 lore raw
Quoted 16 lines · Jonathan Marek

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Fixes af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
---
 arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/qcom/x1e80100.dtsi +0 -0
index cff34d1c74b60..cd34ce5dfd63a 100644

arch/arm64/boot/dts/qcom/x1e80100.dtsi soc: soc@0 { · line 797

Quoted 8 lines
arch/arm64/boot/dts/qcom/x1e80100.dtsi +2 -2
@@ -792,8 +792,8 @@ soc: soc@0 {
 
 		#address-cells = <2>;
 		#size-cells = <2>;
-		dma-ranges = <0 0 0 0 0x10 0>;
-		ranges = <0 0 0 0 0x10 0>;
+		dma-ranges = <0 0 0 0 0x100 0>;
+		ranges = <0 0 0 0 0x100 0>;
 

Could you clarify which "issues" (crashes?) you are referring to?

We need to distinguish two distinct use cases here, which are both (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor with the regular DTB and in EL2 with the x1-el2.dtbo applied.

# EL2 with x1-el2.dtbo

For EL2, I think the 40-bit dma-ranges should indeed work correctly, so we could add your proposed change inside x1-el2.dtso. I'm not sure which issues we are fixing with that though (besides correctness of the hardware description). In EL2, all DMA devices should be behind an IOMMU. In this case, the dma-ranges limit the size of the I/O virtual addresses (DMA addresses) that are given to the devices. The IOMMU maps the DMA buffers to arbitrary physical memory addresses (including outside of the 36-bit range, dma-ranges limits only the DMA address).

I would expect that applying your change effectively just enlarges the I/O virtual address space, which will then be 40-bit instead of just 36-bit. For most devices, even 32-bit of virtual address space should be enough. A larger address space will only be applied for drivers that explicitly request a larger DMA mask (e.g. the nvme driver).

We can make this change for correctness, but given that it is only about the IOVA space, there shouldn't be much functional difference.

# EL1 with Gunyah hypervisor

For EL1, the hypervisor firmware used on most retail laptops limits the usable DMA memory in the SMMUs to the physical 36-bit range. You are right that laptops with 64 GiB memory are essentially unusable in EL1 without disabling the physical memory outside the 36-bit range, but applying this patch would make it even worse.

There are two separate cases:

- For devices behind the SMMUv2, the situation should be the same as above. Increased IOVA space, but no effect on physical address range. This is what is currently causing crashes with 64 GiB RAM in EL1.

- Devices behind the SMMUv3 (PCIe) do not have an IOMMU assigned when running in EL1. In this case, the 36-bit dma-ranges prevents PCIe devices from using memory outside the 36-bit range. They will fall back to bounce buffers in that case. Applying your patch will disable that, making it even more likely to crash than before.

Given that x1e80100.dtsi / hamoa.dtsi primarily models the EL1 setup with Gunyah hypervisor, I don't think it makes sense to apply this patch as-is. It will just make it even more likely to crash than before. I suggest adding these overrides in x1-el2.dtso, with the expected limited effect I described above.

Thanks,
Stephan

#2 lore raw
Quoted 34 lines · Stephan Gerhold
Quoted 16 lines · Jonathan Marek

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Fixes af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
---
 arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/qcom/x1e80100.dtsi +0 -0
index cff34d1c74b60..cd34ce5dfd63a 100644

arch/arm64/boot/dts/qcom/x1e80100.dtsi soc: soc@0 { · line 797

Quoted 8 lines
arch/arm64/boot/dts/qcom/x1e80100.dtsi +2 -2
@@ -792,8 +792,8 @@ soc: soc@0 {
 
 		#address-cells = <2>;
 		#size-cells = <2>;
-		dma-ranges = <0 0 0 0 0x10 0>;
-		ranges = <0 0 0 0 0x10 0>;
+		dma-ranges = <0 0 0 0 0x100 0>;
+		ranges = <0 0 0 0 0x100 0>;
 

Could you clarify which "issues" (crashes?) you are referring to?

We need to distinguish two distinct use cases here, which are both (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor with the regular DTB and in EL2 with the x1-el2.dtbo applied.

# EL2 with x1-el2.dtbo

Quoted 8 lines

For EL2, I think the 40-bit dma-ranges should indeed work correctly, so we could add your proposed change inside x1-el2.dtso. I'm not sure which issues we are fixing with that though (besides correctness of the hardware description). In EL2, all DMA devices should be behind an IOMMU. In this case, the dma-ranges limit the size of the I/O virtual addresses (DMA addresses) that are given to the devices. The IOMMU maps the DMA buffers to arbitrary physical memory addresses (including outside of the 36-bit range, dma-ranges limits only the DMA address).

I've been carrying something similar in my working tree for quite some time too.. The USB4 PCIe controllers have BAR spaces in the >36b region, so this will be necessary anyway.

As for the broken-firmware laptops, there's only so much we can do. A fix for this has been *long* released, but it's up to the OEMs to pull it in.

I'm not fully sure, but I think certain subsystems still have the 36b address limitation (camera?), so it would be good to know whether that needs to be accounted for

Konrad

#3 lore raw
Quoted 53 lines · Konrad Dybcio
Quoted 34 lines · Stephan Gerhold
Quoted 16 lines · Jonathan Marek

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Fixes af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
---
  arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/qcom/x1e80100.dtsi +0 -0
index cff34d1c74b60..cd34ce5dfd63a 100644

arch/arm64/boot/dts/qcom/x1e80100.dtsi soc: soc@0 { · line 797

Quoted 8 lines
arch/arm64/boot/dts/qcom/x1e80100.dtsi +2 -2
@@ -792,8 +792,8 @@ soc: soc@0 {
 
  		#address-cells = <2>;
  		#size-cells = <2>;
-		dma-ranges = <0 0 0 0 0x10 0>;
-		ranges = <0 0 0 0 0x10 0>;
+		dma-ranges = <0 0 0 0 0x100 0>;
+		ranges = <0 0 0 0 0x100 0>;
 

Could you clarify which "issues" (crashes?) you are referring to?

We need to distinguish two distinct use cases here, which are both (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor with the regular DTB and in EL2 with the x1-el2.dtbo applied.

# EL2 with x1-el2.dtbo

Quoted 8 lines

For EL2, I think the 40-bit dma-ranges should indeed work correctly, so we could add your proposed change inside x1-el2.dtso. I'm not sure which issues we are fixing with that though (besides correctness of the hardware description). In EL2, all DMA devices should be behind an IOMMU. In this case, the dma-ranges limit the size of the I/O virtual addresses (DMA addresses) that are given to the devices. The IOMMU maps the DMA buffers to arbitrary physical memory addresses (including outside of the 36-bit range, dma-ranges limits only the DMA address).

I've been carrying something similar in my working tree for quite some time too.. The USB4 PCIe controllers have BAR spaces in the >36b region, so this will be necessary anyway.

As for the broken-firmware laptops, there's only so much we can do. A fix for this has been *long* released, but it's up to the OEMs to pull it in.

Quoted 6 lines

I'm not fully sure, but I think certain subsystems still have the 36b address limitation (camera?), so it would be good to know whether that needs to be accounted for

Konrad

Most devices only support 32-bit address space, and use a 32-bit DMA mask (which is the default, I think?) to only get 32-bit virtual addresses. Camera driver can set a 36-bit DMA mask if it wants to use its whole range.

This patch is about the physical addresses, not virtual. Every device can access the full range (without this, the iommu dma driver thinks buffers with physical addresses outside 36-bit range are not accessible, and tries to use bounce buffers)

Reviewed-by
#4 lore raw
Quoted 53 lines · Jonathan Marek
Quoted 52 lines · Konrad Dybcio
Quoted 33 lines · Stephan Gerhold
Quoted 22 lines · Jonathan Marek

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Fixes af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")

---
  arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--

  1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/qcom/x1e80100.dtsi +2 -2
index cff34d1c74b60..cd34ce5dfd63a 100644
@@ -792,8 +792,8 @@ soc: soc@0 {
             #address-cells = <2>;
           #size-cells = <2>;
-        dma-ranges = <0 0 0 0 0x10 0>;
-        ranges = <0 0 0 0 0x10 0>;
+        dma-ranges = <0 0 0 0 0x100 0>;
+        ranges = <0 0 0 0 0x100 0>;
Quoted 1 line

Could you clarify which "issues" (crashes?) you are referring to?

We need to distinguish two distinct use cases here, which are both (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor with the regular DTB and in EL2 with the x1-el2.dtbo applied.

# EL2 with x1-el2.dtbo

Quoted 8 lines

For EL2, I think the 40-bit dma-ranges should indeed work correctly, so we could add your proposed change inside x1-el2.dtso. I'm not sure which issues we are fixing with that though (besides correctness of the hardware description). In EL2, all DMA devices should be behind an IOMMU. In this case, the dma-ranges limit the size of the I/O virtual addresses (DMA addresses) that are given to the devices. The IOMMU maps the DMA buffers to arbitrary physical memory addresses (including outside of the 36-bit range, dma-ranges limits only the DMA address).

I've been carrying something similar in my working tree for quite some time too.. The USB4 PCIe controllers have BAR spaces in the >36b region, so this will be necessary anyway.

As for the broken-firmware laptops, there's only so much we can do. A fix for this has been *long* released, but it's up to the OEMs to pull it in.

Quoted 8 lines
Quoted 6 lines

I'm not fully sure, but I think certain subsystems still have the 36b address limitation (camera?), so it would be good to know whether that needs to be accounted for

Konrad

Most devices only support 32-bit address space, and use a 32-bit DMA mask (which is the default, I think?) to only get 32-bit virtual addresses. Camera driver can set a 36-bit DMA mask if it wants to use its whole range.

Right

Quoted 1 line

This patch is about the physical addresses, not virtual. Every device can access the full range (without this, the iommu dma driver thinks buffers with physical addresses outside 36-bit range are not accessible, and tries to use bounce buffers)

Yeah this definitely checks out

With discussions about some broken firmware devices becoming more obviously broken in mind:

Konrad

#5 lore raw
In reply to Stephan Gerhold
Quoted 72 lines · Stephan Gerhold
Quoted 16 lines · Jonathan Marek

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Fixes af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
---
  arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/qcom/x1e80100.dtsi +0 -0
index cff34d1c74b60..cd34ce5dfd63a 100644

arch/arm64/boot/dts/qcom/x1e80100.dtsi soc: soc@0 { · line 797

Quoted 8 lines
arch/arm64/boot/dts/qcom/x1e80100.dtsi +2 -2
@@ -792,8 +792,8 @@ soc: soc@0 {
 
  		#address-cells = <2>;
  		#size-cells = <2>;
-		dma-ranges = <0 0 0 0 0x10 0>;
-		ranges = <0 0 0 0 0x10 0>;
+		dma-ranges = <0 0 0 0 0x100 0>;
+		ranges = <0 0 0 0 0x100 0>;
 

Could you clarify which "issues" (crashes?) you are referring to?

We need to distinguish two distinct use cases here, which are both (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor with the regular DTB and in EL2 with the x1-el2.dtbo applied.

# EL2 with x1-el2.dtbo

For EL2, I think the 40-bit dma-ranges should indeed work correctly, so we could add your proposed change inside x1-el2.dtso. I'm not sure which issues we are fixing with that though (besides correctness of the hardware description). In EL2, all DMA devices should be behind an IOMMU. In this case, the dma-ranges limit the size of the I/O virtual addresses (DMA addresses) that are given to the devices. The IOMMU maps the DMA buffers to arbitrary physical memory addresses (including outside of the 36-bit range, dma-ranges limits only the DMA address).

I would expect that applying your change effectively just enlarges the I/O virtual address space, which will then be 40-bit instead of just 36-bit. For most devices, even 32-bit of virtual address space should be enough. A larger address space will only be applied for drivers that explicitly request a larger DMA mask (e.g. the nvme driver).

We can make this change for correctness, but given that it is only about the IOVA space, there shouldn't be much functional difference.

# EL1 with Gunyah hypervisor

For EL1, the hypervisor firmware used on most retail laptops limits the usable DMA memory in the SMMUs to the physical 36-bit range. You are right that laptops with 64 GiB memory are essentially unusable in EL1 without disabling the physical memory outside the 36-bit range, but applying this patch would make it even worse.

There are two separate cases:

- For devices behind the SMMUv2, the situation should be the same as above. Increased IOVA space, but no effect on physical address range. This is what is currently causing crashes with 64 GiB RAM in EL1.

- Devices behind the SMMUv3 (PCIe) do not have an IOMMU assigned when running in EL1. In this case, the 36-bit dma-ranges prevents PCIe devices from using memory outside the 36-bit range. They will fall back to bounce buffers in that case. Applying your patch will disable that, making it even more likely to crash than before.

Quoted 9 lines

Given that x1e80100.dtsi / hamoa.dtsi primarily models the EL1 setup with Gunyah hypervisor, I don't think it makes sense to apply this patch as-is. It will just make it even more likely to crash than before. I suggest adding these overrides in x1-el2.dtso, with the expected limited effect I described above.

Thanks,
Stephan

I am using EL2.

Without this patch, DMA buffers allocated in the upper 36-bit physical range will try to use bounce buffers. The dma range from the dts is compared against the physical address, not the virtual address.

The crash I see is display driver crashes/freezes once a buffer is allocated in the upper 36-bit range and it tries to use bounce buffers. This can happens very quickly under load.

The same crash would happen for EL1 as well. I wasn't aware of the EL1 broken firmware when I sent this patch, but instead of display freezing I guess the behavior would a hard reset now, which is a bit worse but still unusable unles display/gpu driver is disabled.

This patch is correct and should be applied regardless of broken-firmware EL1 cases (where 64GB isn't usable anyway), but I guess the Fixes tag can/should be dropped.

#6 lore raw
Quoted 81 lines · Jonathan Marek
Quoted 70 lines · Stephan Gerhold
Quoted 16 lines · Jonathan Marek

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Fixes af16b00578a7 ("arm64: dts: qcom: Add base X1E80100 dtsi and the QCP dts")
---
  arch/arm64/boot/dts/qcom/x1e80100.dtsi | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/qcom/x1e80100.dtsi +0 -0
index cff34d1c74b60..cd34ce5dfd63a 100644

arch/arm64/boot/dts/qcom/x1e80100.dtsi soc: soc@0 { · line 795

Quoted 6 lines
arch/arm64/boot/dts/qcom/x1e80100.dtsi +2 -2
@@ -792,8 +792,8 @@ soc: soc@0 {
  		#address-cells = <2>;
  		#size-cells = <2>;
-		dma-ranges = <0 0 0 0 0x10 0>;
-		ranges = <0 0 0 0 0x10 0>;
+		dma-ranges = <0 0 0 0 0x100 0>;
+		ranges = <0 0 0 0 0x100 0>;

Could you clarify which "issues" (crashes?) you are referring to?

We need to distinguish two distinct use cases here, which are both (somewhat) supported upstream: Running in EL1 with the Gunyah hypervisor with the regular DTB and in EL2 with the x1-el2.dtbo applied.

# EL2 with x1-el2.dtbo

For EL2, I think the 40-bit dma-ranges should indeed work correctly, so we could add your proposed change inside x1-el2.dtso. I'm not sure which issues we are fixing with that though (besides correctness of the hardware description). In EL2, all DMA devices should be behind an IOMMU. In this case, the dma-ranges limit the size of the I/O virtual addresses (DMA addresses) that are given to the devices. The IOMMU maps the DMA buffers to arbitrary physical memory addresses (including outside of the 36-bit range, dma-ranges limits only the DMA address).

I would expect that applying your change effectively just enlarges the I/O virtual address space, which will then be 40-bit instead of just 36-bit. For most devices, even 32-bit of virtual address space should be enough. A larger address space will only be applied for drivers that explicitly request a larger DMA mask (e.g. the nvme driver).

We can make this change for correctness, but given that it is only about the IOVA space, there shouldn't be much functional difference.

# EL1 with Gunyah hypervisor

For EL1, the hypervisor firmware used on most retail laptops limits the usable DMA memory in the SMMUs to the physical 36-bit range. You are right that laptops with 64 GiB memory are essentially unusable in EL1 without disabling the physical memory outside the 36-bit range, but applying this patch would make it even worse.

There are two separate cases:

- For devices behind the SMMUv2, the situation should be the same as above. Increased IOVA space, but no effect on physical address range. This is what is currently causing crashes with 64 GiB RAM in EL1.

- Devices behind the SMMUv3 (PCIe) do not have an IOMMU assigned when running in EL1. In this case, the 36-bit dma-ranges prevents PCIe devices from using memory outside the 36-bit range. They will fall back to bounce buffers in that case. Applying your patch will disable that, making it even more likely to crash than before.

Quoted 9 lines

Given that x1e80100.dtsi / hamoa.dtsi primarily models the EL1 setup with Gunyah hypervisor, I don't think it makes sense to apply this patch as-is. It will just make it even more likely to crash than before. I suggest adding these overrides in x1-el2.dtso, with the expected limited effect I described above.

Thanks,
Stephan

Quoted 5 lines

I am using EL2.

Without this patch, DMA buffers allocated in the upper 36-bit physical range will try to use bounce buffers. The dma range from the dts is compared against the physical address, not the virtual address.

I don't think this is the case for the dma-iommu layer. I debugged a crash caused by USB in EL1 on a 64 GiB device earlier this year and it was happily using buffers above the 36-bit physical range without using bounce buffers. There is some code inside dma-iommu for using swiotlb, but it's used only for "untrusted" PCI devices and some edge cases with unaligned/small buffers.

Quoted 5 lines

The crash I see is display driver crashes/freezes once a buffer is allocated in the upper 36-bit range and it tries to use bounce buffers. This can happens very quickly under load.

You could be right about the MSM display driver though, since that bypasses dma-iommu and manages the IOMMU itself. I stared at the code a bit and I'm not immediately seeing where it would end up calling into swiotlb, but it might be hidden somewhere in the endless nesting.

Quoted 9 lines

The same crash would happen for EL1 as well. I wasn't aware of the EL1 broken firmware when I sent this patch, but instead of display freezing I guess the behavior would a hard reset now, which is a bit worse but still unusable unles display/gpu driver is disabled.

This patch is correct and should be applied regardless of broken-firmware EL1 cases (where 64GB isn't usable anyway), but I guess the Fixes tag can/should be dropped.

Please clarify the commit message a bit and mention the two separate use cases (EL1 and EL2). I'll leave it up to Bjorn/Konrad to decide whether to merge it. At the end you are right and using 64 GiB RAM in EL1 is kind of a lost cause anyway.

Thanks,
Stephan

#7 lore raw
Quoted 2 lines · Stephan Gerhold
Quoted 1 line · Jonathan Marek

On 11/28/25 5:26 AM, Stephan Gerhold wrote:

...

Quoted 20 lines
Quoted 6 lines

I am using EL2.

Without this patch, DMA buffers allocated in the upper 36-bit physical range will try to use bounce buffers. The dma range from the dts is compared against the physical address, not the virtual address.

I don't think this is the case for the dma-iommu layer. I debugged a crash caused by USB in EL1 on a 64 GiB device earlier this year and it was happily using buffers above the 36-bit physical range without using bounce buffers. There is some code inside dma-iommu for using swiotlb, but it's used only for "untrusted" PCI devices and some edge cases with unaligned/small buffers.

Quoted 5 lines

The crash I see is display driver crashes/freezes once a buffer is allocated in the upper 36-bit range and it tries to use bounce buffers. This can happens very quickly under load.

Quoted 5 lines

You could be right about the MSM display driver though, since that bypasses dma-iommu and manages the IOMMU itself. I stared at the code a bit and I'm not immediately seeing where it would end up calling into swiotlb, but it might be hidden somewhere in the endless nesting.

Looks like you are right about this, MSM driver ends up going through dma_direct_map_phys(), which decides to use bounce buffers. I didn't try to see if other drivers end up using bounce buffers, but it would make sense that only MSM driver is affected.

Quoted 18 lines
Quoted 9 lines

The same crash would happen for EL1 as well. I wasn't aware of the EL1 broken firmware when I sent this patch, but instead of display freezing I guess the behavior would a hard reset now, which is a bit worse but still unusable unles display/gpu driver is disabled.

This patch is correct and should be applied regardless of broken-firmware EL1 cases (where 64GB isn't usable anyway), but I guess the Fixes tag can/should be dropped.

Please clarify the commit message a bit and mention the two separate use cases (EL1 and EL2). I'll leave it up to Bjorn/Konrad to decide whether to merge it. At the end you are right and using 64 GiB RAM in EL1 is kind of a lost cause anyway.

Thanks,
Stephan

#8 lore raw

Hi,

Sorry to hijack this thread, but I've been carrying this patch to workaround the buggy firmware from Stephan for my Lenovo T14s with 64gb RAM which seems to make Linux boot & work just fine with systemd-boot.
Of course, it's a hack and not ready for upstream.

On another topic, perhaps we need to add this to
https://gitlab.com/Linaro/arm64-laptops/linux , what do you think Stephan ?

Chris

commit 16e21600c6a65ace0c75b5a722e27ced8e1bfc48

Author: Stephan Gerhold <[email protected]>
Date:   Thu Mar 6 17:41:15 2025 +0000

HACK: efi-stub: Exclude 40-bit memory

(quoted diff) +0 -0

b/drivers/firmware/efi/libstub/efi-stub.c
index 874f63b4a383..355a9b2c0818 100644

drivers/firmware/efi/libstub/efi-stub.c +0 -0
@@ -106,6 +106,17 @@ efi_status_t

efi_handle_cmdline(efi_loaded_image_t *image, char **cmdline_ptr)

 {
        char *cmdline __free(efi_pool) = NULL;
        efi_status_t status;
+       efi_physical_addr_t addr;
+       unsigned long pages;
+
+       addr = 0x0000008800000000;
+       pages = 0x800000000 / EFI_PAGE_SIZE;
+       status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS,

+ EFI_RESERVED_TYPE,
pages, &addr);
+ if (status != EFI_SUCCESS) {
+ efi_err("failed to reserve upper 32 GiB (40-bit memory)\n");
+ return EFI_OUT_OF_RESOURCES;
+ }

/*
* Get the command line from EFI, using the LOADED_IMAGE

Cheers!

Chris

Quoted 57 lines · Jonathan Marek
Quoted 2 lines · Stephan Gerhold
Quoted 1 line · Jonathan Marek

On 11/28/25 5:26 AM, Stephan Gerhold wrote:

...

Quoted 20 lines
Quoted 6 lines

I am using EL2.

Without this patch, DMA buffers allocated in the upper 36-bit physical range will try to use bounce buffers. The dma range from the dts is compared against the physical address, not the virtual address.

I don't think this is the case for the dma-iommu layer. I debugged a crash caused by USB in EL1 on a 64 GiB device earlier this year and it was happily using buffers above the 36-bit physical range without using bounce buffers. There is some code inside dma-iommu for using swiotlb, but it's used only for "untrusted" PCI devices and some edge cases with unaligned/small buffers.

Quoted 5 lines

The crash I see is display driver crashes/freezes once a buffer is allocated in the upper 36-bit range and it tries to use bounce buffers. This can happens very quickly under load.

Quoted 5 lines

You could be right about the MSM display driver though, since that bypasses dma-iommu and manages the IOMMU itself. I stared at the code a bit and I'm not immediately seeing where it would end up calling into swiotlb, but it might be hidden somewhere in the endless nesting.

Looks like you are right about this, MSM driver ends up going through dma_direct_map_phys(), which decides to use bounce buffers. I didn't try to see if other drivers end up using bounce buffers, but it would make sense that only MSM driver is affected.

Quoted 18 lines
Quoted 9 lines

The same crash would happen for EL1 as well. I wasn't aware of the EL1 broken firmware when I sent this patch, but instead of display freezing I guess the behavior would a hard reset now, which is a bit worse but still unusable unles display/gpu driver is disabled.

This patch is correct and should be applied regardless of broken-firmware EL1 cases (where 64GB isn't usable anyway), but I guess the Fixes tag can/should be dropped.

Please clarify the commit message a bit and mention the two separate use cases (EL1 and EL2). I'll leave it up to Bjorn/Konrad to decide whether to merge it. At the end you are right and using 64 GiB RAM in EL1 is kind of a lost cause anyway.

Thanks,
Stephan

#9 lore raw

Hi,

Quoted 9 lines · Christopher Obbard

Hi,

Sorry to hijack this thread, but I've been carrying this patch to workaround the buggy firmware from Stephan for my Lenovo T14s with 64gb RAM which seems to make Linux boot & work just fine with systemd-boot.
Of course, it's a hack and not ready for upstream.

Quoted 3 lines

On another topic, perhaps we need to add this to
https://gitlab.com/Linaro/arm64-laptops/linux , what do you think Stephan ?

Small update, I create an issue to track this bug:
https://gitlab.com/Linaro/arm64-laptops/linux/-/issues/23
and opened a merge request with Stephan's patch:
https://gitlab.com/Linaro/arm64-laptops/linux/-/merge_requests/1

Cheers!

Chris

Quoted 102 lines

Chris

commit 16e21600c6a65ace0c75b5a722e27ced8e1bfc48

Author: Stephan Gerhold <[email protected]>
Date:   Thu Mar 6 17:41:15 2025 +0000

HACK: efi-stub: Exclude 40-bit memory

drivers/firmware/efi/libstub/efi-stub.c +11 -0
index 874f63b4a383..355a9b2c0818 100644
@@ -106,6 +106,17 @@ efi_status_t
 efi_handle_cmdline(efi_loaded_image_t *image, char **cmdline_ptr)
 {
        char *cmdline __free(efi_pool) = NULL;
        efi_status_t status;
+       efi_physical_addr_t addr;
+       unsigned long pages;
+
+       addr = 0x0000008800000000;
+       pages = 0x800000000 / EFI_PAGE_SIZE;
+       status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS,
+                                                EFI_RESERVED_TYPE,
 pages, &addr);
+       if (status != EFI_SUCCESS) {
+               efi_err("failed to reserve upper 32 GiB (40-bit memory)\n");
+               return EFI_OUT_OF_RESOURCES;
+       }
 

/*
* Get the command line from EFI, using the LOADED_IMAGE

Cheers!

Chris

Quoted 57 lines · Jonathan Marek
Quoted 2 lines · Stephan Gerhold
Quoted 1 line · Jonathan Marek

On 11/28/25 5:26 AM, Stephan Gerhold wrote:

...

Quoted 20 lines
Quoted 6 lines

I am using EL2.

Without this patch, DMA buffers allocated in the upper 36-bit physical range will try to use bounce buffers. The dma range from the dts is compared against the physical address, not the virtual address.

I don't think this is the case for the dma-iommu layer. I debugged a crash caused by USB in EL1 on a 64 GiB device earlier this year and it was happily using buffers above the 36-bit physical range without using bounce buffers. There is some code inside dma-iommu for using swiotlb, but it's used only for "untrusted" PCI devices and some edge cases with unaligned/small buffers.

Quoted 5 lines

The crash I see is display driver crashes/freezes once a buffer is allocated in the upper 36-bit range and it tries to use bounce buffers. This can happens very quickly under load.

Quoted 5 lines

You could be right about the MSM display driver though, since that bypasses dma-iommu and manages the IOMMU itself. I stared at the code a bit and I'm not immediately seeing where it would end up calling into swiotlb, but it might be hidden somewhere in the endless nesting.

Looks like you are right about this, MSM driver ends up going through dma_direct_map_phys(), which decides to use bounce buffers. I didn't try to see if other drivers end up using bounce buffers, but it would make sense that only MSM driver is affected.

Quoted 18 lines
Quoted 9 lines

The same crash would happen for EL1 as well. I wasn't aware of the EL1 broken firmware when I sent this patch, but instead of display freezing I guess the behavior would a hard reset now, which is a bit worse but still unusable unles display/gpu driver is disabled.

This patch is correct and should be applied regardless of broken-firmware EL1 cases (where 64GB isn't usable anyway), but I guess the Fixes tag can/should be dropped.

Please clarify the commit message a bit and mention the two separate use cases (EL1 and EL2). I'll leave it up to Bjorn/Konrad to decide whether to merge it. At the end you are right and using 64 GiB RAM in EL1 is kind of a lost cause anyway.

Thanks,
Stephan

#10 lore raw
In reply to Christopher Obbard

FYI, for a less hacky solution you can use CONFIG_ARM64_VA_BITS_39, arm64 linux as-is refuses to use any memory it can't identity map (and the extra 32GB of memory is outside the 39-bit range).

Quoted 55 lines · Christopher Obbard

Hi,

Sorry to hijack this thread, but I've been carrying this patch to workaround the buggy firmware from Stephan for my Lenovo T14s with 64gb RAM which seems to make Linux boot & work just fine with systemd-boot.
Of course, it's a hack and not ready for upstream.

On another topic, perhaps we need to add this to
https://gitlab.com/Linaro/arm64-laptops/linux , what do you think Stephan ?

Chris

commit 16e21600c6a65ace0c75b5a722e27ced8e1bfc48

Author: Stephan Gerhold <[email protected]>
Date:   Thu Mar 6 17:41:15 2025 +0000

HACK: efi-stub: Exclude 40-bit memory

drivers/firmware/efi/libstub/efi-stub.c +11 -0
index 874f63b4a383..355a9b2c0818 100644
@@ -106,6 +106,17 @@ efi_status_t
 efi_handle_cmdline(efi_loaded_image_t *image, char **cmdline_ptr)
  {
         char *cmdline __free(efi_pool) = NULL;
         efi_status_t status;
+       efi_physical_addr_t addr;
+       unsigned long pages;
+
+       addr = 0x0000008800000000;
+       pages = 0x800000000 / EFI_PAGE_SIZE;
+       status = efi_bs_call(allocate_pages, EFI_ALLOCATE_ADDRESS,
+                                                EFI_RESERVED_TYPE,
 pages, &addr);
+       if (status != EFI_SUCCESS) {
+               efi_err("failed to reserve upper 32 GiB (40-bit memory)\n");
+               return EFI_OUT_OF_RESOURCES;
+       }
 

/*
* Get the command line from EFI, using the LOADED_IMAGE

Cheers!

Chris

#11 lore raw

Hi All,

On my Thinkpad T14S OLED with 64GB, with this patch applied, I am unable to boot the laptop anymore. It doesn't matter whether I have mem=30GB, nor if I set CONFIG_ARM64_VA_BITS_39, or not limiting the ram - shortly after loading the device tree, the laptop just resets.

I have tried over 20 boots with it applied, as well as trying both booting a working kernel, and then rebooting as well as cold boots.

-- steev

#12 lore raw

I was curious about this so I tried booting EL1
- 32GB CRD boots fine
- 64GB T14S boots with CONFIG_ARM64_VA_BITS_39 (except display driver didn't probe, because it depends on adsp firmware, lol). without CONFIG_ARM64_VA_BITS_39 it resets

Anyway making things worse for "64GB+EL1+broken firmware" shouldn't block this patch, because "64GB+EL1+broken firmware" needs workarounds anyway and this change is needed to use 64GB without issues otherwise.

Quoted 7 lines · Steev Klimaszewski

Hi All,

On my Thinkpad T14S OLED with 64GB, with this patch applied, I am unable to boot the laptop anymore. It doesn't matter whether I have mem=30GB, nor if I set CONFIG_ARM64_VA_BITS_39, or not limiting the ram - shortly after loading the device tree, the laptop just resets.

Quoted 6 lines

I have tried over 20 boots with it applied, as well as trying both booting a working kernel, and then rebooting as well as cold boots.

-- steev

.

#13 lore raw

Hi Jonathan,

Quoted 6 lines · Jonathan Marek

I was curious about this so I tried booting EL1
- 32GB CRD boots fine
- 64GB T14S boots with CONFIG_ARM64_VA_BITS_39 (except display driver didn't probe, because it depends on adsp firmware, lol). without CONFIG_ARM64_VA_BITS_39 it resets

Weird, I definitely have CONFIG_ARM64_VA_BITS_39 set here.

Quoted 4 lines

Anyway making things worse for "64GB+EL1+broken firmware" shouldn't block this patch, because "64GB+EL1+broken firmware" needs workarounds anyway and this change is needed to use 64GB without issues otherwise.

I'm not sure I follow this logic; most users when they first get the system will be stuck in el1 until they get el2 set up, which also requires workarounds in order to set up. I don't at all disagree that the fix is more correct, but I don't like users having broken things just because the vendor hasn't fixed their firmware yet. Additionally, distro kernels will default to BITS_52 for arm64 kernels, not 39 because most of them run more places than just Qualcomm laptops.

Quoted 16 lines · Steev Klimaszewski
Quoted 7 lines

Hi All,

On my Thinkpad T14S OLED with 64GB, with this patch applied, I am unable to boot the laptop anymore. It doesn't matter whether I have mem=30GB, nor if I set CONFIG_ARM64_VA_BITS_39, or not limiting the ram - shortly after loading the device tree, the laptop just resets.

Quoted 6 lines

I have tried over 20 boots with it applied, as well as trying both booting a working kernel, and then rebooting as well as cold boots.

-- steev

.

maintainer Applied
#14 lore raw
In reply to Jonathan Marek
Quoted 6 lines · Jonathan Marek

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Applied, thanks!

[1/1] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

      commit: b38dd256e11a4c8bd5a893e11fc42d493939c907

Best regards,

Signature · 1 line
Bjorn Andersson <[email protected]>
#15 lore raw

It turns out this change will make things worse for the (unfortunately common) EL1+64GB+brokenfirmware case.

Because of that I think the Fixes: tag and "(fix 64GB models)" should be dropped from the commit message. (I can also send a v2 with extra info in the commit message if needed)

Quoted 16 lines · Bjorn Andersson
Quoted 6 lines · Jonathan Marek

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Applied, thanks!

[1/1] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

       commit: b38dd256e11a4c8bd5a893e11fc42d493939c907

Best regards,

maintainer
#16 lore raw
Quoted 7 lines · Jonathan Marek

It turns out this change will make things worse for the (unfortunately common) EL1+64GB+brokenfirmware case.

Because of that I think the Fixes: tag and "(fix 64GB models)" should be dropped from the commit message. (I can also send a v2 with extra info in the commit message if needed)

It seems to me that neither of these to actions will affect the impact of the patch. What does "make things worse" imply?

Are we better off dropping this patch?

Regards,
Bjorn

Quoted 17 lines · Bjorn Andersson
Quoted 16 lines · Jonathan Marek
Quoted 6 lines

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Applied, thanks!

[1/1] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

       commit: b38dd256e11a4c8bd5a893e11fc42d493939c907

Best regards,

#17 lore raw
Quoted 4 lines · Bjorn Andersson
Quoted 3 lines · Jonathan Marek

It turns out this change will make things worse for the (unfortunately common) EL1+64GB+brokenfirmware case.

Quoted 8 lines
Quoted 4 lines

Because of that I think the Fixes: tag and "(fix 64GB models)" should be dropped from the commit message. (I can also send a v2 with extra info in the commit message if needed)

It seems to me that neither of these to actions will affect the impact of the patch. What does "make things worse" imply?

From what Stephan wrote [1]: Without this change, devices that use SMMUv3 (PCIe) use bounce buffers for memory outside the 36-bit range. With this change they won't use SWIOTLB and it will crash instead. (it would still crash for other reasons without this change)

[1] https://patchwork.kernel.org/comment/26681014/

Quoted 1 line

Are we better off dropping this patch?

This patch gets EL2+64GB (and presumably EL1 as well if OEM provides updated EL2 firmware) working without crashes. That's better than having all 64GB cases be broken.

Quoted 21 lines

Regards,
Bjorn

Quoted 17 lines · Bjorn Andersson
Quoted 16 lines · Jonathan Marek
Quoted 6 lines

Unlike the phone SoCs this was copied from, x1e has a 40-bit physical bus. The upper address space is used to support more than 32GB of memory.

This fixes issues when DMA buffers are allocated outside the 36-bit range.

Applied, thanks!

[1/1] arm64: dts: qcom: x1e: bus is 40-bits (fix 64GB models)

        commit: b38dd256e11a4c8bd5a893e11fc42d493939c907

Best regards,