Skip to contentSkip to conversation

spl, rk3399: fix FIT image loadingg out-of-range

PATCH Superseded Jerome Forissier 3 participants 7 messages Jun 9, 2022 – Jun 29, 2022 Arm (32-bit)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover spl, rk3399: fix FIT image loadingg out-of-range

The patches in this series address an issue I met when trying to enable FIT signature verification by SPL on a RockPi4B board.

- The first patch avoids a buffer overflow when writing to INTMEM1 SRAM
- The second one addresses reliability issues I had with back-to-back MMC reads from the microSD card. By allowing DMA operation, the issue is gone and generally speaking DMA is preferred over FIFO mode anyways.

Jerome Forissier (2):
spl: fit: add config option for temporary buffer when loading image
rockchip: rk3399: enable spl-fifo-mode for sdmmc only when needed

 arch/arm/dts/rk3399-u-boot.dtsi |  2 +
 common/spl/Kconfig              | 45 +++++++++++++++++++
 common/spl/spl_fit.c            | 79 ++++++++++++++++++++++++++++++---
 3 files changed, 120 insertions(+), 6 deletions(-)
Signature · 1 line
2.34.1
#2 lore raw

PATCH 2/2 rockchip: rk3399: enable spl-fifo-mode for sdmmc only when needed

Commit 5c606ca35c42 ("rockchip: rk3399: enable spl-fifo-mode for sdmmc") mentions that the RK3399 SoC can't do DMA between SDMMC and SRAM. According to the TRM "7.3.2 Embedded SRAM access path" [1], only the 8KB SRAM at 0xff3b0000 (INTMEM1) is in this situation. The 192KB SRAM can be accessed by both DMA controllers.

Assuming the only use case for writing from MMC to INTMEM1 is loading a FIT image, and with the introduction of a temporary buffer for that purpose (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE, which is required anyways to ensure the destination boundaries are enforced), then spl-fifo-mode is not needed anymore and DMA can be enabled safely.

---
 arch/arm/dts/rk3399-u-boot.dtsi | 2 ++
 1 file changed, 2 insertions(+)
arch/arm/dts/rk3399-u-boot.dtsi +2 -0
index 716b9a433a..a1b6d6f007 100644
@@ -124,8 +124,10 @@
 &sdmmc {
 	u-boot,dm-pre-reloc;
 
+#ifndef CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE
 	/* mmc to sram can't do dma, prevent aborts transferring TF-A parts */
 	u-boot,spl-fifo-mode;
+#endif
 };
 
 &spi1 {
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/2 spl: fit: add config option for temporary buffer when loading image

In reply to Jerome Forissier

When the load address of a FIT image isn't properly aligned, spl_load_fit_image() may write past the end of the destination buffer. It is not an issue in many cases because the memory happens to be writeable and nothing important is present in the overflow. On RockPi4 however there is a configuration where a TF-A image (bl31_0xff3b0000.bin) has to be loaded into a 8K range of SRAM memory, between 0xff3b0000 and 0xff3b2000. The end address is a hard limit, because due to the way the hardware is wired, the addresses wrap and any overflow gets written back to 0xff3b0000 thus overwriting previous data.

To address this problem, introduce a helper function which loads data using a temporary buffer allocated on the stack. The size of the buffer is defined by SPL_LOAD_FIT_IMAGE_BUFFER_SIZE (default 0x0 or disabled).

---
 common/spl/Kconfig   | 45 +++++++++++++++++++++++++
 common/spl/spl_fit.c | 79 ++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 118 insertions(+), 6 deletions(-)
common/spl/Kconfig +45 -0
index 50ff113cab..d95141a6f6 100644
@@ -1339,6 +1339,51 @@ config SPL_OPENSBI_LOAD_ADDR
 	help
 	  Load address of the OpenSBI binary.
 
+config SPL_LOAD_FIT_IMAGE_BUFFER_SIZE
+	hex "Read unaligned external FIT images to a temporary buffer in SPL"
+	default 0x0
+	depends on SPL_LOAD_FIT
+	help
+	  An aligned FIT image is such that it starts at the beginning
+	  of a block in media and has a load_addr in its FIT header
+	  that is DMA aligned in RAM. These aligned images can be read
+	  directly from media to RAM. Unaligned external FIT images
+	  are those that need reading some extra data before and/or after
+	  the image because they don't occupy fully the blocks they're
+	  in in media, or their destination is not DMA aligned and must
+	  be read somewhere aligned before copying them to load_addr.
+
+	  With this option set to 0x0 full blocks will just be read in
+	  the closest DMA aligned address, and the unaligned image
+	  inside those read blocks will later be copied to
+	  load_addr. Meanwhile memory outside [load_addr,
+	  load_addr+length) will have been written. That's no problem
+	  when the block length, image size and load_addr have been
+	  taken into account when laying out memory.
+
+	  But in some cases not all memory is writable, or undesired
+	  effects arise when writing outside [load_addr,
+	  load_addr+length). For instance, in RK3399, one of the
+	  images is ATF2, of size 8KiB which should be loaded into
+	  INTMEM1, at address 0xff3b0000. This address is DMA aligned
+	  but tight. It maps to a 8KiB SRAM area. If the image on
+	  media is not block aligned some extra bytes get read before
+	  and after, and everything extra written in
+	  0xff3b2000-0xff3bffff corrupts SRAM at
+	  0xff3b0000-0xff3b1fff before the image is memcpyied in place.
+
+	  With this option set to a nonzero value a DMA aligned buffer
+	  will be allocated on the stack and the image will be read
+	  in chuncks of blocks to this buffer, with each chunk being
+	  memcpyied to [load_addr,load_addr+length), so never writing
+	  outside the destination area.
+
+	  The advantage of enabling this option is safety, and the
+	  disadvantage is more stack use and slower image load (one read
+	  per chunk instead of just one).
+
+	  The default is 0x0 to replicate previous behaviour.
+
 config TPL
 	bool
 	depends on SUPPORT_TPL
common/spl/spl_fit.c +73 -6

106 lines not on this page · raw message

Signature · 1 line
2.34.1
#3 lore raw

Re: [PATCH 2/2] rockchip: rk3399: enable spl-fifo-mode for sdmmc only when needed

In reply to Jerome Forissier
Quoted 24 lines · Jerome Forissier

Commit 5c606ca35c42 ("rockchip: rk3399: enable spl-fifo-mode for sdmmc") mentions that the RK3399 SoC can't do DMA between SDMMC and SRAM. According to the TRM "7.3.2 Embedded SRAM access path" [1], only the 8KB SRAM at 0xff3b0000 (INTMEM1) is in this situation. The 192KB SRAM can be accessed by both DMA controllers.

Assuming the only use case for writing from MMC to INTMEM1 is loading a FIT image, and with the introduction of a temporary buffer for that purpose (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE, which is required anyways to ensure the destination boundaries are enforced), then spl-fifo-mode is not needed anymore and DMA can be enabled safely.

---
 arch/arm/dts/rk3399-u-boot.dtsi | 2 ++
 1 file changed, 2 insertions(+)
arch/arm/dts/rk3399-u-boot.dtsi +0 -0
index 716b9a433a..a1b6d6f007 100644

arch/arm/dts/rk3399-u-boot.dtsi line 127

Quoted 4 lines
arch/arm/dts/rk3399-u-boot.dtsi +1 -0
@@ -124,8 +124,10 @@
 &sdmmc {
 	u-boot,dm-pre-reloc;
 
+#ifndef CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE

Oops, that should rather be:

+#if (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE == 0)

Quoted 6 lines
arch/arm/dts/rk3399-u-boot.dtsi +1 -0
 	/* mmc to sram can't do dma, prevent aborts transferring TF-A parts */
 	u-boot,spl-fifo-mode;
+#endif
 };
 
 &spi1 {
Reviewed-by Tested-by
#4 lore raw

Re: [PATCH 2/2] rockchip: rk3399: enable spl-fifo-mode for sdmmc only when needed

El Tue, Jun 14, 2022 at 11:16:42AM -0700, Jerome Forissier deia:

Quoted 4 lines

Oops, that should rather be:

+#if (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE == 0)

I tested with this change, not that my opinion counts much, but anyway:

This changes reduce some 0.2 s the boot time of my Rock Pi 4B.

Before this series, just with u-boot/master from today
c18e5fb055 dtoc: Update test_src_scan.py for new tegra compatibles
plus the patches I sent to this list (I can't boot from MMC without them)
https://lists.denx.de/pipermail/u-boot/2022-June/485497.html
and bootstage configured, I get :

Timer summary in microseconds (10 records):

       Mark    Elapsed  Stage
  1,903,436  1,903,436  board_init_f
  1,903,436          0  board_init_f
  2,900,331    996,895  board_init_r
  4,091,657  1,191,326  id=64
  4,930,650    838,993  id=65
  4,930,827        177  main_loop
  7,946,715  3,015,888  bootm_start
  9,010,637  1,063,922  id=15
  9,010,639          2  start_kernel

Accumulated time:

                22,700  dm_r
               479,397  dm_f

With that plus 1/2 in this series and CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE=0x3000
I get something similar (slightly slower because of >=3 correct reads instead of 1 overwriting read per image):

Timer summary in microseconds (10 records):

       Mark    Elapsed  Stage
  1,979,414  1,979,414  board_init_f
  1,979,414          0  board_init_f
  2,976,429    997,015  board_init_r
  4,166,623  1,190,194  id=64
  5,005,623    839,000  id=65
  5,005,800        177  main_loop
  8,020,791  3,014,991  bootm_start
  9,084,116  1,063,325  id=15
  9,084,118          2  start_kernel

Accumulated time:

                22,699  dm_r
               479,480  dm_f

With that plus this 2/2 it's faster (while safer) than initially.

Timer summary in microseconds (10 records):

       Mark    Elapsed  Stage
  1,709,384  1,709,384  board_init_f
  1,709,384          0  board_init_f
  2,706,192    996,808  board_init_r
  3,895,269  1,189,077  id=64
  4,733,786    838,517  id=65
  4,733,963        177  main_loop
  7,751,063  3,017,100  bootm_start
  8,814,449  1,063,386  id=15
  8,814,451          2  start_kernel

Accumulated time:

                22,703  dm_r
               479,520  dm_f

With this change your 2/2 patch becomes

---

From: Jerome Forissier <[email protected]>
Date: Thu, 9 Jun 2022 17:23:22 +0200
Subject: [PATCH] rockchip: rk3399: enable spl-fifo-mode for sdmmc only when
needed

Commit 5c606ca35c42 ("rockchip: rk3399: enable spl-fifo-mode for sdmmc") mentions that the RK3399 SoC can't do DMA between SDMMC and SRAM. According to the TRM "7.3.2 Embedded SRAM access path" [1], only the 8KB SRAM at 0xff3b0000 (INTMEM1) is in this situation. The 192KB SRAM can be accessed by both DMA controllers.

Assuming the only use case for writing from MMC to INTMEM1 is loading a FIT image, and with the introduction of a temporary buffer for that purpose (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE, which is required anyways to ensure the destination boundaries are enforced), then spl-fifo-mode is not needed anymore and DMA can be enabled safely.

---
 arch/arm/dts/rk3399-u-boot.dtsi | 2 ++
 1 file changed, 2 insertions(+)
arch/arm/dts/rk3399-u-boot.dtsi +2 -0
index 716b9a433a..e0bb230022 100644
@@ -124,9 +124,10 @@
 &sdmmc {
 	u-boot,dm-pre-reloc;
 
+#if (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE == 0)
 	/* mmc to sram can't do dma, prevent aborts transferring TF-A parts */
 	u-boot,spl-fifo-mode;
+#endif
 };
 
 &spi1 {
Signature · 1 line
2.20.1
Reviewed-by on v1 2/2
#5 lore raw

Re: [PATCH 2/2] rockchip: rk3399: enable spl-fifo-mode for sdmmc only when needed

In reply to Jerome Forissier
Quoted 12 lines · Jerome Forissier

Commit 5c606ca35c42 ("rockchip: rk3399: enable spl-fifo-mode for sdmmc") mentions that the RK3399 SoC can't do DMA between SDMMC and SRAM. According to the TRM "7.3.2 Embedded SRAM access path" [1], only the 8KB SRAM at 0xff3b0000 (INTMEM1) is in this situation. The 192KB SRAM can be accessed by both DMA controllers.

Assuming the only use case for writing from MMC to INTMEM1 is loading a FIT image, and with the introduction of a temporary buffer for that purpose (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE, which is required anyways to ensure the destination boundaries are enforced), then spl-fifo-mode is not needed anymore and DMA can be enabled safely.

Quoted 3 lines

Thanks,

- Kever

Quoted 19 lines
---
  arch/arm/dts/rk3399-u-boot.dtsi | 2 ++
  1 file changed, 2 insertions(+)
arch/arm/dts/rk3399-u-boot.dtsi +2 -0
index 716b9a433a..a1b6d6f007 100644
@@ -124,8 +124,10 @@
  &sdmmc {
  	u-boot,dm-pre-reloc;
 
+#ifndef CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE
  	/* mmc to sram can't do dma, prevent aborts transferring TF-A parts */
  	u-boot,spl-fifo-mode;
+#endif
  };
 
  &spi1 {
#6 lore raw

Re: [PATCH 2/2] rockchip: rk3399: enable spl-fifo-mode for sdmmc only when needed

Quoted 14 lines · Kever Yang
Quoted 12 lines · Jerome Forissier

Commit 5c606ca35c42 ("rockchip: rk3399: enable spl-fifo-mode for sdmmc") mentions that the RK3399 SoC can't do DMA between SDMMC and SRAM. According to the TRM "7.3.2 Embedded SRAM access path" [1], only the 8KB SRAM at 0xff3b0000 (INTMEM1) is in this situation. The 192KB SRAM can be accessed by both DMA controllers.

Assuming the only use case for writing from MMC to INTMEM1 is loading a FIT image, and with the introduction of a temporary buffer for that purpose (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE, which is required anyways to ensure the destination boundaries are enforced), then spl-fifo-mode is not needed anymore and DMA can be enabled safely.

Quoted 5 lines
Quoted 3 lines

Thanks. I will post a v2 with the corrected conditional [#if (xxx == 0)].

Signature · 1 line
Jerome
Quoted 22 lines

Thanks,

- Kever

Quoted 17 lines

---
  arch/arm/dts/rk3399-u-boot.dtsi | 2 ++

  1 file changed, 2 insertions(+)
arch/arm/dts/rk3399-u-boot.dtsi +0 -0
index 716b9a433a..a1b6d6f007 100644
@@ -124,8 +124,10 @@
   &sdmmc {
       u-boot,dm-pre-reloc;

  +#ifndef CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE
      /* mmc to sram can't do dma, prevent aborts transferring TF-A parts */ u-boot,spl-fifo-mode;
+#endif
  };
    &spi1 {