Skip to contentSkip to conversation

qemu-arm64: Allow booting via Trusted Firmware

PATCH New Andre Przywara 4 participants 17 messages Sep 30, 2020 – Oct 9, 2020 Arm (32-bit)

lore
Share & tools
lore
#0 lore raw

v2 cover qemu-arm64: Allow booting via Trusted Firmware

U-Boot on QEMU-arm64 can be used in two configurations: Loaded directly via QEMU's -bios option, or as a non-secure payload (BL33) via ARM Trusted Firmware-A (TF-A).
In the latter case we need to define CONFIG_TFABOOT, to accommodate the first flash bank being secure only, and manually set SYS_TEXT_BASE to the address configured in TF-A (currently 0x60000000).

To avoid this poorly documented adventure, we enable a position independent build, and also let the flash regions be always detected through the DTB. This results in a single build to work under both scenarios, and also allows to move the BL33 load address in TF-A to something lower in the future.

For this to work, we have to first make PIE work when booted from ROM. While writing to ROM should not hurt, it might trigger CFI flash sequences, and indeed crashes for me in the middle of the fixup routine. This is covered by patch 2/6, which skips the whole fixup routine if the offset is actually 0 (as it is in our case). To support older toolchains (including the popular Linaro builds), we need to ensure we do the static RELA fixups, even with PIE enabled (patch 1/6).
Also we have to decouple the relative initial stack pointer from the PIE option, as we always need to use the fixed version, pointing to RAM (patch 3/6).
Patch 4/6 drops the hard-coded flash address, instead U-Boot can already read all required information from QEMU's DTB.
Patch 5/6 is a cleanup, while the last patch enables the PIE build.

With this series the very same u-boot.bin file works when directly loaded from the QEMU command line (-bios), but also when embedded into TF-A's fip.bin, removing the need for case-specific build options.

Please have a look!

Cheers,
Andre

Changelog v1 .. v2:
- Always do STATIC_RELA static fixups (new first patch)
- Reword commit messages for 3/6 and 5/6

Andre Przywara (5):
arm64: PIE: Skip fixups if distance is zero
arm64: PIE: Allow fixed stack pointer
qemu-arm: Remove need to specify flash banks
qemu: Drop ARCH_SUPPORT_TFABOOT
qemu/arm64: Enable POSITION_INDEPENDENT

 arch/arm/Kconfig             | 4 ++--
 arch/arm/cpu/armv8/start.S   | 3 ++-
 configs/qemu_arm64_defconfig | 1 +
 include/configs/qemu-arm.h   | 8 +-------
 4 files changed, 6 insertions(+), 10 deletions(-)
Signature · 21 lines
2.17.5


*** BLURB HERE ***

Andre Przywara (6):
  arm64: PIE: Do not skip static relocation
  arm64: PIE: Skip fixups if distance is zero
  arm64: PIE: Allow fixed stack pointer
  qemu-arm: Remove need to specify flash banks
  qemu-arm: Drop ARCH_SUPPORT_TFABOOT
  qemu-arm64: Enable POSITION_INDEPENDENT

 arch/arm/Kconfig             | 6 +++---
 arch/arm/cpu/armv8/start.S   | 3 ++-
 configs/qemu_arm64_defconfig | 1 +
 include/configs/qemu-arm.h   | 8 +-------
 4 files changed, 7 insertions(+), 11 deletions(-)

-- 
2.17.5
#1 lore raw

PATCH v2 1/6 arm64: PIE: Do not skip static relocation

When we build an arm64 target and enable POSITION_INDEPENDENT, we were skipping our build-time dynamic relocation fixup routine (STATIC_RELA).

This was probably done because we didn't need it in this case, as the PIE fixup routine in start.S would take care of that at runtime.

However when we now skip this routine (upon detecting that the fixup offset is 0), this might lead to uninitialised pointers.

Remove the exception, so that we always do the build-time relocation.

NOTE: GNU binutils starting with v2.27.1 do this build-time relocation automatically, to be in-line with other architecures. So on newer toolchains our manual fixup is actually not needed. It doesn't hurt to have it, though, so that we keep compatibility with the popular Linaro toolchains, which lack this feature.

---
 arch/arm/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm/Kconfig +1 -1
index 80702c23d34..b6fb276b6f8 100644
@@ -76,7 +76,7 @@ config GIC_V3_ITS
 
 config STATIC_RELA
 	bool
-	default y if ARM64 && !POSITION_INDEPENDENT
+	default y if ARM64
 
 config DMA_ADDR_T_64BIT
 	bool
Signature · 1 line
2.17.5
#2 lore raw

PATCH v2 1/6 arm64: PIE: Do not skip static relocation

Quoted 9 lines · Andre Przywara

When we build an arm64 target and enable POSITION_INDEPENDENT, we were skipping our build-time dynamic relocation fixup routine (STATIC_RELA).

This was probably done because we didn't need it in this case, as the PIE fixup routine in start.S would take care of that at runtime.

However when we now skip this routine (upon detecting that the fixup offset is 0), this might lead to uninitialised pointers.

Quoted 7 lines

Remove the exception, so that we always do the build-time relocation.

NOTE: GNU binutils starting with v2.27.1 do this build-time relocation automatically, to be in-line with other architecures. So on newer toolchains our manual fixup is actually not needed. It doesn't hurt to have it, though, so that we keep compatibility with the popular Linaro toolchains, which lack this feature.

#3 lore raw

PATCH v2 1/6 arm64: PIE: Do not skip static relocation

In reply to Andre Przywara

Hi,

Quoted 7 lines

NOTE: GNU binutils starting with v2.27.1 do this build-time relocation automatically, to be in-line with other architecures. So on newer toolchains our manual fixup is actually not needed. It doesn't hurt to have it, though, so that we keep compatibility with the popular Linaro toolchains, which lack this feature.

Quoted 8 lines

---

With this, U-Boot now runs well from ROM when compiled with "gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu".

Thanks
-Amit

#4 lore raw

PATCH v2 1/6 arm64: PIE: Do not skip static relocation

In reply to Andre Przywara
Quoted 11 lines · Andre Przywara

When we build an arm64 target and enable POSITION_INDEPENDENT, we were skipping our build-time dynamic relocation fixup routine (STATIC_RELA).

This was probably done because we didn't need it in this case, as the PIE fixup routine in start.S would take care of that at runtime.

However when we now skip this routine (upon detecting that the fixup offset is 0), this might lead to uninitialised pointers.

Remove the exception, so that we always do the build-time relocation.

Quoted 8 lines

NOTE: GNU binutils starting with v2.27.1 do this build-time relocation automatically, to be in-line with other architecures. So on newer toolchains our manual fixup is actually not needed. It doesn't hurt to have it, though, so that we keep compatibility with the popular Linaro toolchains, which lack this feature.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20201009/b6df8103/attachment.sig>
#5 lore raw

PATCH v2 2/6 arm64: PIE: Skip fixups if distance is zero

In reply to Andre Przywara

When the actual offset between link and runtime address is zero, there is no need for patching up U-Boot early when running with CONFIG_POSITION_INDEPENDENT.

Skip the whole routine when the distance is 0.

This helps when U-Boot is loaded into ROM, or in otherwise sensitive memory locations.

---
 arch/arm/cpu/armv8/start.S | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/arm/cpu/armv8/start.S +2 -1
index 002698b501c..02b952bb328 100644
@@ -66,7 +66,8 @@ save_boot_params_ret:
 pie_fixup:
 	adr	x0, _start		/* x0 <- Runtime value of _start */
 	ldr	x1, _TEXT_BASE		/* x1 <- Linked value of _start */
-	sub	x9, x0, x1		/* x9 <- Run-vs-link offset */
+	subs	x9, x0, x1		/* x9 <- Run-vs-link offset */
+	beq	pie_fixup_done
 	adr	x2, __rel_dyn_start	/* x2 <- Runtime &__rel_dyn_start */
 	adr	x3, __rel_dyn_end	/* x3 <- Runtime &__rel_dyn_end */
 pie_fix_loop:
Signature · 1 line
2.17.5
#6 lore raw

PATCH v2 2/6 arm64: PIE: Skip fixups if distance is zero

Quoted 11 lines · Andre Przywara

When the actual offset between link and runtime address is zero, there is no need for patching up U-Boot early when running with CONFIG_POSITION_INDEPENDENT.

Skip the whole routine when the distance is 0.

This helps when U-Boot is loaded into ROM, or in otherwise sensitive memory locations.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20201009/3697064f/attachment.sig>
#7 lore raw

PATCH v2 3/6 arm64: PIE: Allow fixed stack pointer

In reply to Andre Przywara

Currently selecting CONFIG_POSITION_INDEPENDENT also forces us to use an initial stack pointer relative to the beginning of the BSS section. This makes some sense, because this should be writable memory anyway.

However the BSS section is not cleared or used until later in the setup process (after relocation), so memory nearby might not be available early enough to host the initial stack. This is an issue if U-Boot is loaded from (Flash-)ROM, for instance.

Allow CONFIG_INIT_SP_RELATIVE to be turned off by a board's config, to be able to select a fixed stack pointer, for instance in known good DRAM.

This will help QEMU utilising PIE, when it's loaded to (Flash-)ROM.

---
 arch/arm/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm/Kconfig +1 -1
index b6fb276b6f8..486141478ce 100644
@@ -12,7 +12,6 @@ config ARM64
 if ARM64
 config POSITION_INDEPENDENT
 	bool "Generate position-independent pre-relocation code"
-	select INIT_SP_RELATIVE
 	help
 	  U-Boot expects to be linked to a specific hard-coded address, and to
 	  be loaded to and run from that address. This option lifts that
@@ -23,6 +22,7 @@ config POSITION_INDEPENDENT
 
 config INIT_SP_RELATIVE
 	bool "Specify the early stack pointer relative to the .bss section"
+	default y if POSITION_INDEPENDENT
 	help
 	  U-Boot typically uses a hard-coded value for the stack pointer
 	  before relocation. Enable this option to instead calculate the
Signature · 1 line
2.17.5
#8 lore raw

PATCH v2 3/6 arm64: PIE: Allow fixed stack pointer

Quoted 9 lines · Andre Przywara

Currently selecting CONFIG_POSITION_INDEPENDENT also forces us to use an initial stack pointer relative to the beginning of the BSS section. This makes some sense, because this should be writable memory anyway.

However the BSS section is not cleared or used until later in the setup process (after relocation), so memory nearby might not be available early enough to host the initial stack. This is an issue if U-Boot is loaded from (Flash-)ROM, for instance.

Quoted 8 lines

Allow CONFIG_INIT_SP_RELATIVE to be turned off by a board's config, to be able to select a fixed stack pointer, for instance in known good DRAM.

This will help QEMU utilising PIE, when it's loaded to (Flash-)ROM.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20201009/c0440675/attachment.sig>
#9 lore raw

PATCH v2 4/6 qemu-arm: Remove need to specify flash banks

In reply to Andre Przywara

Currently we hard-code the number and initial addresses of QEMU's flash banks, even though our code is perfectly able to gather the same information from the DTB provided by QEMU.
This is especially annoying, since we have two slightly different U-Boot configurations ("bare-metal" vs. loaded via Arm Trusted Firmware), which need to be selected at build time.

Drop the two hard coded alternatives, and use CONFIG_SYS_MAX_FLASH_BANKS_DETECT instead, which relies on the DTB to figure out the actual flash configuration at runtime.

---
 include/configs/qemu-arm.h | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
include/configs/qemu-arm.h +1 -7
index bc8b7c5c123..273fa1a7d7b 100644
@@ -45,13 +45,7 @@
 #define CONFIG_SYS_CBSIZE 512
 
 #define CONFIG_SYS_MONITOR_BASE		CONFIG_SYS_TEXT_BASE
-#ifdef CONFIG_TFABOOT
-#define CONFIG_SYS_FLASH_BASE		0x4000000
-#define CONFIG_SYS_MAX_FLASH_BANKS	1
-#else
-#define CONFIG_SYS_FLASH_BASE		0x0
-#define CONFIG_SYS_MAX_FLASH_BANKS	2
-#endif
+#define CONFIG_SYS_MAX_FLASH_BANKS_DETECT	2
 #define CONFIG_SYS_MAX_FLASH_SECT	256 /* Sector: 256K, Bank: 64M */
 #define CONFIG_CFI_FLASH_USE_WEAK_ACCESSORS
 
Signature · 1 line
2.17.5
#10 lore raw

PATCH v2 4/6 qemu-arm: Remove need to specify flash banks

Quoted 8 lines · Andre Przywara

Currently we hard-code the number and initial addresses of QEMU's flash

banks, even though our code is perfectly able to gather the same information from the DTB provided by QEMU.
This is especially annoying, since we have two slightly different U-Boot configurations ("bare-metal" vs. loaded via Arm Trusted Firmware), which need to be selected at build time.

Quoted 5 lines

Drop the two hard coded alternatives, and use CONFIG_SYS_MAX_FLASH_BANKS_DETECT instead, which relies on the DTB to figure out the actual flash configuration at runtime.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20201009/67d613bf/attachment.sig>
#11 lore raw

PATCH v2 5/6 qemu-arm: Drop ARCH_SUPPORT_TFABOOT

In reply to Andre Przywara

CONFIG_ARCH_SUPPORT_TFABOOT was used on the qemu-arm64 platform to guard a tweak to the flash bank configuration. U-Boot now reads the current flash setup from the devicetree, so there is no need for this option anymore.

---
 arch/arm/Kconfig | 1 -
 1 file changed, 1 deletion(-)
arch/arm/Kconfig +0 -1
index 486141478ce..5ffa84c1d98 100644
@@ -929,7 +929,6 @@ config ARCH_OWL
 
 config ARCH_QEMU
 	bool "QEMU Virtual Platform"
-	select ARCH_SUPPORT_TFABOOT
 	select DM
 	select DM_SERIAL
 	select OF_CONTROL
Signature · 1 line
2.17.5
#12 lore raw

PATCH v2 5/6 qemu-arm: Drop ARCH_SUPPORT_TFABOOT

Quoted 6 lines · Andre Przywara

CONFIG_ARCH_SUPPORT_TFABOOT was used on the qemu-arm64 platform to guard a tweak to the flash bank configuration. U-Boot now reads the current flash setup from the devicetree, so there is no need for this option anymore.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200930/7d64645b/attachment.sig>
#13 lore raw

PATCH v2 5/6 qemu-arm: Drop ARCH_SUPPORT_TFABOOT

In reply to Andre Przywara
Quoted 7 lines · Andre Przywara

CONFIG_ARCH_SUPPORT_TFABOOT was used on the qemu-arm64 platform to guard a tweak to the flash bank configuration. U-Boot now reads the current flash setup from the devicetree, so there is no need for this option anymore.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20201009/0032ff4b/attachment.sig>
#14 lore raw

PATCH v2 6/6 qemu-arm64: Enable POSITION_INDEPENDENT

In reply to Andre Przywara

Now that PIE works when U-Boot is started from ROM, let's enable CONFIG_POSITION_INDEPENDENT, which allows to load U-Boot also via ARM Trusted-Firmware's fip.bin to DRAM, without tweaking the configuration.

To get a writable initial stack, we need to keep the fixed initial stack pointer, which points to DRAM in our case.

---
 arch/arm/Kconfig             | 1 +
 configs/qemu_arm64_defconfig | 1 +
 2 files changed, 2 insertions(+)
arch/arm/Kconfig +1 -0
index 5ffa84c1d98..8ba73680699 100644
@@ -22,6 +22,7 @@ config POSITION_INDEPENDENT
 
 config INIT_SP_RELATIVE
 	bool "Specify the early stack pointer relative to the .bss section"
+	default n if ARCH_QEMU
 	default y if POSITION_INDEPENDENT
 	help
 	  U-Boot typically uses a hard-coded value for the stack pointer
configs/qemu_arm64_defconfig +1 -0
index 31ea2d342fc..4450e7ced42 100644
@@ -1,6 +1,7 @@
 CONFIG_ARM=y
 CONFIG_ARCH_QEMU=y
 CONFIG_NR_DRAM_BANKS=1
+CONFIG_POSITION_INDEPENDENT=y
 CONFIG_ENV_SIZE=0x40000
 CONFIG_ENV_SECT_SIZE=0x40000
 CONFIG_AHCI=y
Signature · 1 line
2.17.5
#15 lore raw

PATCH v2 6/6 qemu-arm64: Enable POSITION_INDEPENDENT

Quoted 10 lines · Andre Przywara

Now that PIE works when U-Boot is started from ROM, let's enable CONFIG_POSITION_INDEPENDENT, which allows to load U-Boot also via ARM Trusted-Firmware's fip.bin to DRAM, without tweaking the configuration.

To get a writable initial stack, we need to keep the fixed initial stack pointer, which points to DRAM in our case.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20201009/035af7f6/attachment.sig>
#16 lore raw

v2 cover qemu-arm64: Allow booting via Trusted Firmware

In reply to Andre Przywara
Quoted 27 lines · Andre Przywara

U-Boot on QEMU-arm64 can be used in two configurations: Loaded directly via QEMU's -bios option, or as a non-secure payload (BL33) via ARM Trusted Firmware-A (TF-A).
In the latter case we need to define CONFIG_TFABOOT, to accommodate the first flash bank being secure only, and manually set SYS_TEXT_BASE to the address configured in TF-A (currently 0x60000000).

To avoid this poorly documented adventure, we enable a position independent build, and also let the flash regions be always detected through the DTB. This results in a single build to work under both scenarios, and also allows to move the BL33 load address in TF-A to something lower in the future.

For this to work, we have to first make PIE work when booted from ROM. While writing to ROM should not hurt, it might trigger CFI flash sequences, and indeed crashes for me in the middle of the fixup routine. This is covered by patch 2/6, which skips the whole fixup routine if the offset is actually 0 (as it is in our case). To support older toolchains (including the popular Linaro builds), we need to ensure we do the static RELA fixups, even with PIE enabled (patch 1/6).
Also we have to decouple the relative initial stack pointer from the PIE option, as we always need to use the fixed version, pointing to RAM (patch 3/6).
Patch 4/6 drops the hard-coded flash address, instead U-Boot can already read all required information from QEMU's DTB.
Patch 5/6 is a cleanup, while the last patch enables the PIE build.

Quoted 5 lines

With this series the very same u-boot.bin file works when directly loaded from the QEMU command line (-bios), but also when embedded into TF-A's fip.bin, removing the need for case-specific build options.

Please have a look!

Can you please also update doc/board/emulation/qemu-arm.rst with instructions / example of using this configuration? Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200930/ac3f1c5b/attachment.sig>