Skip to contentSkip to conversation

arm64: dts: qcom: sm6125-xiaomi-laurel-sprout: reserve firmware-owned DRAM

PATCH In review Roman Linev 3 participants 7 messages Aug 21 – Nov 8

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 arm64: dts: qcom: sm6125-xiaomi-laurel-sprout: reserve firmware-owned DRAM

sm6125.dtsi sizes reserved_mem1@46200000 at 0x2d00000 (45 MiB), so it ends at 0x48f00000. On laurel-sprout the firmware owns 0x4900000 (73 MiB) at that base: the downstream device tree overrides the SoC region with

	removed_region@46200000 { reg = <0 0x46200000 0 0x4900000>; };

which ends exactly at 0x4ab00000, where mainline places camera_mem. The 28 MiB between 0x48f00000 and 0x4ab00000 is therefore firmware-owned on this board while mainline leaves it as free System RAM and hands it to the page allocator.

On Qualcomm a "removed_region" is XPU-protected, so this is not a matter of reading stale data: the region is not the kernel's to allocate.

The observed symptom is hard lockups under sustained buffered reads, where page-cache pressure eventually reaches the disputed range. Before this change the device wedged at a rate of roughly one lockup per 59 MiB of buffered reads from the SD card, reproducibly enough to fire on demand. With the region reserved, the same workload has run 691 GiB of buffered reads with zero lockups.

Reserve the delta as a board-level region abutting camera_mem rather than resizing the SoC-level reserved_mem1, since other SM6125 boards may legitimately have a different split; sm6125-xiaomi-ginkgo, for instance, has its own corrected reserved-memory layout.

---
 .../boot/dts/qcom/sm6125-xiaomi-laurel-sprout.dts  | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
arch/arm64/boot/dts/qcom/sm6125-xiaomi-laurel-sprout.dts +14 -0
index 139f2b401..a6ee6ee80 100644
@@ -37,6 +37,20 @@ framebuffer0: framebuffer@5c000000 {
 	};
 
 	reserved-memory {
+		/*
+		 * The SoC dtsi sizes reserved_mem1 at 0x2d00000 (45 MiB), ending
+		 * at 0x48f00000. This device's firmware owns 0x4900000 (73 MiB)
+		 * there -- downstream laurel_sprout carves out
+		 * removed_region@46200000 with that size, ending exactly at
+		 * camera_mem's 0x4ab00000. The 28 MiB in between is XPU-owned
+		 * on this board and mainline currently hands it to the buddy
+		 * allocator.
+		 */
+		reserved_mem1_laurel: memory@48f00000 {
+			reg = <0x0 0x48f00000 0x0 0x1c00000>;
+			no-map;
+		};
+
 		debug_mem: debug@ffb00000 {
 			reg = <0x0 0xffb00000 0x0 0xc0000>;
 			no-map;
Signature · 1 line
2.55.0
#1 lore raw
Quoted 6 lines · Roman Linev

sm6125.dtsi sizes reserved_mem1@46200000 at 0x2d00000 (45 MiB), so it ends at 0x48f00000. On laurel-sprout the firmware owns 0x4900000 (73 MiB) at that base: the downstream device tree overrides the SoC region with

	removed_region@46200000 { reg = <0 0x46200000 0 0x4900000>; };
Quoted 7 lines

which ends exactly at 0x4ab00000, where mainline places camera_mem. The 28 MiB between 0x48f00000 and 0x4ab00000 is therefore firmware-owned on this board while mainline leaves it as free System RAM and hands it to the page allocator.

On Qualcomm a "removed_region" is XPU-protected, so this is not a matter of reading stale data: the region is not the kernel's to allocate.

Not necessarily, XPU protection is a specialized mechanism, there may be other owners that don't like tampering with their memory (hyp, tz..)

Quoted 6 lines

The observed symptom is hard lockups under sustained buffered reads, where page-cache pressure eventually reaches the disputed range. Before this change the device wedged at a rate of roughly one lockup per 59 MiB of buffered reads from the SD card, reproducibly enough to fire on demand. With the region reserved, the same workload has run 691 GiB of buffered reads with zero lockups.

CONFIG_MEMTEST=y and memtest=1 in cmdline is good to run to confirm your memory map checks out

[...]

arch/arm64/boot/dts/qcom/sm6125-xiaomi-laurel-sprout.dts framebuffer0: framebuffer@5c000000 { · line 49

Quoted 10 lines
arch/arm64/boot/dts/qcom/sm6125-xiaomi-laurel-sprout.dts +10 -0
+		/*
+		 * The SoC dtsi sizes reserved_mem1 at 0x2d00000 (45 MiB), ending
+		 * at 0x48f00000. This device's firmware owns 0x4900000 (73 MiB)
+		 * there -- downstream laurel_sprout carves out
+		 * removed_region@46200000 with that size, ending exactly at
+		 * camera_mem's 0x4ab00000. The 28 MiB in between is XPU-owned
+		 * on this board and mainline currently hands it to the buddy
+		 * allocator.
+		 */
+		reserved_mem1_laurel: memory@48f00000 {

The label is unused, you can drop it. I think you should just do something like this:

/* This carveout is larger than on other devices */

&reserved_mem1 {
	reg = <0x0 0x46200000 0x0 foo>;
};

Konrad

#2 lore raw

PATCH v2 1/1 arm64: dts: qcom: sm6125-xiaomi-laurel-sprout: enlarge reserved_mem1

In reply to Roman Linev

sm6125.dtsi sizes reserved_mem1@46200000 at 0x2d00000 (45 MiB), so it ends at 0x48f00000. On laurel-sprout the firmware owns more than that:
the downstream device tree describes the same base as

  removed_region@46200000 { reg = <0 0x46200000 0 0x4900000>; };

i.e. 0x4900000 (73 MiB), ending exactly at 0x4ab00000 where mainline places camera_mem. The 28 MiB between 0x48f00000 and 0x4ab00000 is therefore owned by firmware on this board while mainline leaves it as free System RAM and hands it to the page allocator.

The observed symptom is hard lockups under sustained buffered reads, where page cache pressure eventually reaches the disputed range. Before this change the device wedged at a rate of roughly one lockup per 59 MiB of buffered reads from the SD card, reproducibly enough to fire on demand. With the region enlarged, the same workload has run 691 GiB of buffered reads with zero lockups.

Booting with CONFIG_MEMTEST=y and memtest=1 confirms the resulting map:
17 free ranges are swept with no bad ranges reported, and the swept list skips 0x46200000..0x4ab00000 entirely, so none of the firmware carveout is in the page allocator's pool any more.

---
Changes in v2 (all from Konrad's review of v1):
- Resize the existing &reserved_mem1 instead of adding a second, board-level region for the delta. Same bytes reserved, one node.
- Drop the now-unused label.
- Do not claim the range is XPU-protected. As Konrad pointed out it may equally be claimed by hyp or TZ; either way it is not the kernel's to allocate, so the commit message now says only that.
- Add the CONFIG_MEMTEST=y / memtest=1 confirmation Konrad asked for.

v1: https://lore.kernel.org/r/[email protected]

---
 .../boot/dts/qcom/sm6125-xiaomi-laurel-sprout.dts    | 12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/arm64/boot/dts/qcom/sm6125-xiaomi-laurel-sprout.dts +12 -0
index 139f2b4..027c009 100644
@@ -324,6 +324,18 @@
 	status = "okay";
 };
 
+/*
+ * The firmware carveout at this base is larger on this device than the
+ * SoC dtsi assumes: downstream laurel_sprout describes it as
+ * removed_region@46200000 with a size of 0x4900000, ending exactly where
+ * camera_mem begins. The 28 MiB past the dtsi's 0x2d00000 is not the
+ * kernel's to allocate, and handing it to the buddy allocator wedges the
+ * machine under memory pressure.
+ */
+&reserved_mem1 {
+	reg = <0x0 0x46200000 0x0 0x4900000>;
+};
+
 &rpm_requests {
 	regulators-0 {
 		compatible = "qcom,rpm-pm6125-regulators";
Signature · 1 line
2.55.0
#3 lore raw

Re: [PATCH v2] arm64: dts: qcom: sm6125-xiaomi-laurel-sprout: enlarge reserved_mem1

Quoted 3 lines · Roman Linev

sm6125.dtsi sizes reserved_mem1@46200000 at 0x2d00000 (45 MiB), so it ends at 0x48f00000. On laurel-sprout the firmware owns more than that:
the downstream device tree describes the same base as

[...]

arch/arm64/boot/dts/qcom/sm6125-xiaomi-laurel-sprout.dts line 337

Quoted 11 lines
arch/arm64/boot/dts/qcom/sm6125-xiaomi-laurel-sprout.dts +11 -0
+/*
+ * The firmware carveout at this base is larger on this device than the
+ * SoC dtsi assumes: downstream laurel_sprout describes it as
+ * removed_region@46200000 with a size of 0x4900000, ending exactly where
+ * camera_mem begins. The 28 MiB past the dtsi's 0x2d00000 is not the
+ * kernel's to allocate, and handing it to the buddy allocator wedges the
+ * machine under memory pressure.
+ */
+&reserved_mem1 {
+	reg = <0x0 0x46200000 0x0 0x4900000>;
+};

Please update the common DTSI, it matches the reference fw map and I'm very surprised the sony phone didn't crash without that reservation

Konrad

#4 lore raw

PATCH v3 1/1 arm64: dts: qcom: sm6125: enlarge reserved_mem1

In reply to Roman Linev

reserved_mem1@46200000 is sized 0x2d00000 (45 MiB) in the SoC dtsi, so it ends at 0x48f00000, while camera_mem begins at 0x4ab00000. The reference firmware memory map for sm6125 reserves 0x4900000 (73 MiB) at this base, ending exactly where camera_mem starts, so the 28 MiB in between belongs to firmware and not to the kernel. The dtsi nevertheless leaves that span as normal System RAM and hands it to the page allocator.

Every sm6125 board inherits this reservation and none of them overrides it or places another region in the gap, so size it for the firmware map in the common dtsi.

On laurel-sprout the mismatch is observable: page-cache pressure eventually reaches the span, and a store into it never completes, which hard-locks the machine. Before this change the board wedged at roughly one lockup per 59 MiB of buffered reads; after enlarging the region the same workload ran 691 GiB of buffered reads with no lockups. Booting with CONFIG_MEMTEST=y and memtest=1 sweeps 17 free ranges with no bad ranges reported and skips 0x46200000..0x4ab00000 entirely, confirming the carveout is out of the page allocator's pool.

The downstream laurel_sprout device tree describes the same base as removed_region@46200000 with a size of 0x4900000, so the reserved bytes are not a new claim about this hardware family.

---
Changes in v3 (from Konrad's review of v2):
- Move the reservation into the common SoC dtsi, as the size follows the reference firmware map and applies to every sm6125 board. The v2 board-level &reserved_mem1 override is dropped; v3 touches no board file.
- State that no sm6125 board overrides this region or places another region in 0x48f00000..0x4ab00000, and that all four affected boards build with the enlarged span ending exactly at camera_mem.

v2: https://lore.kernel.org/r/[email protected]
v1: https://lore.kernel.org/r/[email protected]

---
 arch/arm64/boot/dts/qcom/sm6125.dtsi | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/boot/dts/qcom/sm6125.dtsi +1 -1
index 86de156..fb126db 100644
@@ -282,7 +282,7 @@ smem_mem: memory@46000000 {
 		};
 
 		reserved_mem1: memory@46200000 {
-			reg = <0x0 0x46200000 0x0 0x2d00000>;
+			reg = <0x0 0x46200000 0x0 0x4900000>;
 			no-map;
 		};
 
Reviewed-by on v3 1/1
#5 lore raw

Re: [PATCH v3] arm64: dts: qcom: sm6125: enlarge reserved_mem1

Quoted 20 lines · Roman Linev

reserved_mem1@46200000 is sized 0x2d00000 (45 MiB) in the SoC dtsi, so it ends at 0x48f00000, while camera_mem begins at 0x4ab00000. The reference firmware memory map for sm6125 reserves 0x4900000 (73 MiB) at this base, ending exactly where camera_mem starts, so the 28 MiB in between belongs to firmware and not to the kernel. The dtsi nevertheless leaves that span as normal System RAM and hands it to the page allocator.

Every sm6125 board inherits this reservation and none of them overrides it or places another region in the gap, so size it for the firmware map in the common dtsi.

On laurel-sprout the mismatch is observable: page-cache pressure eventually reaches the span, and a store into it never completes, which hard-locks the machine. Before this change the board wedged at roughly one lockup per 59 MiB of buffered reads; after enlarging the region the same workload ran 691 GiB of buffered reads with no lockups. Booting with CONFIG_MEMTEST=y and memtest=1 sweeps 17 free ranges with no bad ranges reported and skips 0x46200000..0x4ab00000 entirely, confirming the carveout is out of the page allocator's pool.

Quoted 6 lines

The downstream laurel_sprout device tree describes the same base as removed_region@46200000 with a size of 0x4900000, so the reserved bytes are not a new claim about this hardware family.

---

Konrad

maintainer Applied
#6 lore raw

Re: [PATCH v3] arm64: dts: qcom: sm6125: enlarge reserved_mem1

In reply to Roman Linev
Quoted 8 lines · Roman Linev

reserved_mem1@46200000 is sized 0x2d00000 (45 MiB) in the SoC dtsi, so it ends at 0x48f00000, while camera_mem begins at 0x4ab00000. The reference firmware memory map for sm6125 reserves 0x4900000 (73 MiB) at this base, ending exactly where camera_mem starts, so the 28 MiB in between belongs to firmware and not to the kernel. The dtsi nevertheless leaves that span as normal System RAM and hands it to the page allocator.

[...]

Applied, thanks!

[1/1] arm64: dts: qcom: sm6125: enlarge reserved_mem1

      commit: 0733c22353a775b2b48e13a396ae8434cb17dc7a

Best regards,

Signature · 1 line
Bjorn Andersson <[email protected]>