Skip to contentSkip to conversation

Support position independent execution

PATCH Superseded Vincent Chen 4 participants 10 messages Mar 5, 2021 – Mar 16, 2021

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Support position independent execution

This patch set enables OpenSBI to support position-independent execution so that one OpenSBI generic firmware can run on multiple platforms, even if they have different memory base addresses. However, the PIC code will decrease a bit of performance.

The performance loss of PIC comes from two additional operations. One is from the symbol relocation in the beginning stage, and another one is from the GOT reference (auipc + load GOT) in the OpenSBI lifetime. The former only affects the boot time, but the latter will affect the performance of the OpenSBI payload such as the Linux kernel. After analyzing the fw_payload.elf of the generic platform, 27 symbols use the GOT reference method to get their address. Fortunately, 17 of these symbols are accessed once during the initialized step. Only the remaining 10 symbols may be called repeatedly during the lifetime, which are
1. hartid_to_domain_table
2. hartid_to_scratch_table
3. sbi_hart_expected_trap
4. sbi_tlb_local_hfence_vvma
5. sbi_tlb_local_hfence_gvma
6. sbi_tlb_local_sfence_vma_asid
7. sbi_tlb_local_fence_i
8. sbi_tlb_local_hfence_gvma
9. sbi_tlb_local_hfence_vvma
10. sbi_tlb_local_sfence_vma
From these 10 symbols, I think the IPI handler may be one of the critical and commonly used paths affected by the PIC mode. Therefore, I create two kernel modules, One keeps issuing the REMOTE_FENCE_I request and the other one keeps issuing REMOTE_SFENCE_VMA request. These two modules are executed on the unleashed board to measure the consumed time of the sbi_tlb_request(). I found the PIC causes a 1.6% and 4.1% performance loss in addressing REMOTE_FENCE_I and REMOTE_SFENCE_VMA, respectively. In addition, to observe the impact of PIC mode in the general usage cases, I use SPEC2006 to benchmark performance. However, I could not observe any performance impact from the results. Based on the above two results, I think the performance impact of the PIC may not be obvious in general usage. Just in case, this patch still creates an FW_PIC option for users to configure the build options.

Changes from v1 patch:
1. Add -fpic to ASFLAGS and then remove the ".option pic" from fw_base.S
2. Append the required flags to $(firmware-genflags-y), $(firmware-cflags-y) and $(firmware-ldflags-y) instead of modifying top Makefile.
3. Rearrange the patches

Vincent Chen (2):
firmware: Use lla to access all global symbols
firmware: Support position independent execution

 Makefile                      |   2 +-
 firmware/fw_base.S            | 147 +++++++++++++++++++++++++++++-------------
 firmware/fw_base.ldS          |  13 ++++
 firmware/fw_dynamic.S         |  18 +++---
 firmware/fw_jump.S            |  12 +++-
 firmware/fw_payload.S         |   8 ++-
 firmware/objects.mk           |   7 ++
 firmware/payloads/test_head.S |  18 +++---
 include/sbi/riscv_elf.h       |   9 +++
 9 files changed, 168 insertions(+), 66 deletions(-)
 create mode 100644 include/sbi/riscv_elf.h
Signature · 1 line
2.7.4
#1 lore raw

PATCH v2 1/2 firmware: Use lla to access all global symbols

When OpenSBI is compiled as fPIE mode, the assembler will translate "la" to GOT reference pattern. It will cause to cost an additional load instruction when obtaining the symbol address. However, if the symbol locates within the positive or negative 2GB region, we can use "lla" instead of "la" to avoid unneeded GOT references. This patch assumes that the OpenSBI image excluding the payload does not exceed 2GB. Based on this assumption, all "la" instructions are replaced by "lla" to avoid performance degradation when compiling as fPIE mode.

---
 firmware/fw_base.S            | 88 +++++++++++++++++++++----------------------
 firmware/fw_dynamic.S         | 18 ++++-----
 firmware/fw_jump.S            |  2 +-
 firmware/fw_payload.S         |  2 +-
 firmware/payloads/test_head.S | 18 ++++-----
 5 files changed, 64 insertions(+), 64 deletions(-)
firmware/fw_base.S +44 -44
index ab33e11..6cc5f88 100644
@@ -57,39 +57,39 @@ _start:
 	bne	a0, a6, _wait_relocate_copy_done
 _try_lottery:
 	/* Jump to relocation wait loop if we don't get relocation lottery */
-	la	a6, _relocate_lottery
+	lla	a6, _relocate_lottery
 	li	a7, 1
 	amoadd.w a6, a7, (a6)
 	bnez	a6, _wait_relocate_copy_done
 
 	/* Save load address */
-	la	t0, _load_start
-	la	t1, _start
+	lla	t0, _load_start
+	lla	t1, _start
 	REG_S	t1, 0(t0)
 
 	/* Relocate if load address != link address */
 _relocate:
-	la	t0, _link_start
+	lla	t0, _link_start
 	REG_L	t0, 0(t0)
-	la	t1, _link_end
+	lla	t1, _link_end
 	REG_L	t1, 0(t1)
-	la	t2, _load_start
+	lla	t2, _load_start
 	REG_L	t2, 0(t2)
 	sub	t3, t1, t0
 	add	t3, t3, t2
 	beq	t0, t2, _relocate_done
-	la	t4, _relocate_done
+	lla	t4, _relocate_done
 	sub	t4, t4, t2
 	add	t4, t4, t0
 	blt	t2, t0, _relocate_copy_to_upper
 _relocate_copy_to_lower:
 	ble	t1, t2, _relocate_copy_to_lower_loop
-	la	t3, _relocate_lottery
+	lla	t3, _relocate_lottery
 	BRANGE	t2, t1, t3, _start_hang
-	la	t3, _boot_status
+	lla	t3, _boot_status
 	BRANGE	t2, t1, t3, _start_hang
-	la	t3, _relocate
-	la	t5, _relocate_done
+	lla	t3, _relocate
+	lla	t5, _relocate_done
 	BRANGE	t2, t1, t3, _start_hang
 	BRANGE	t2, t1, t5, _start_hang
 	BRANGE  t3, t5, t2, _start_hang
@@ -102,12 +102,12 @@ _relocate_copy_to_lower_loop:
 	jr	t4
 _relocate_copy_to_upper:
 	ble	t3, t0, _relocate_copy_to_upper_loop
-	la	t2, _relocate_lottery
+	lla	t2, _relocate_lottery
 	BRANGE	t0, t3, t2, _start_hang
-	la	t2, _boot_status
+	lla	t2, _boot_status
 	BRANGE	t0, t3, t2, _start_hang
-	la	t2, _relocate
-	la	t5, _relocate_done
+	lla	t2, _relocate
+	lla	t5, _relocate_done
 	BRANGE	t0, t3, t2, _start_hang
 	BRANGE	t0, t3, t5, _start_hang
 	BRANGE	t2, t5, t0, _start_hang
@@ -119,12 +119,12 @@ _relocate_copy_to_upper_loop:
 	blt	t0, t1, _relocate_copy_to_upper_loop
 	jr	t4
 _wait_relocate_copy_done:
-	la	t0, _start
-	la	t1, _link_start
+	lla	t0, _start
+	lla	t1, _link_start
 	REG_L	t1, 0(t1)
 	beq	t0, t1, _wait_for_boot_hart
-	la	t2, _boot_status
-	la	t3, _wait_for_boot_hart
+	lla	t2, _boot_status
+	lla	t3, _wait_for_boot_hart
 	sub	t3, t3, t0
 	add	t3, t3, t1
 1:
@@ -143,10 +143,10 @@ _relocate_done:
 	 * Mark relocate copy done
 	 * Use _boot_status copy relative to the load address
 	 */
-	la	t0, _boot_status
-	la	t1, _link_start
+	lla	t0, _boot_status
+	lla	t1, _link_start
 	REG_L	t1, 0(t1)
-	la	t2, _load_start
+	lla	t2, _load_start
 	REG_L	t2, 0(t2)
 	sub	t0, t0, t1
 	add	t0, t0, t2
@@ -161,19 +161,19 @@ _relocate_done:
 	call	_reset_regs
 
 	/* Zero-out BSS */
-	la	s4, _bss_start
-	la	s5, _bss_end
+	lla	s4, _bss_start
+	lla	s5, _bss_end
 _bss_zero:
 	REG_S	zero, (s4)
 	add	s4, s4, __SIZEOF_POINTER__
 	blt	s4, s5, _bss_zero
 
 	/* Setup temporary trap handler */
-	la	s4, _start_hang
+	lla	s4, _start_hang
 	csrw	CSR_MTVEC, s4
 
 	/* Setup temporary stack */
-	la	s4, _fw_end
+	lla	s4, _fw_end
 	li	s5, (SBI_SCRATCH_SIZE * 2)
 	add	sp, s4, s5
 
@@ -184,7 +184,7 @@ _bss_zero:
 
 #ifdef FW_FDT_PATH
 	/* Override previous arg1 */
-	la	a1, fw_fdt_bin
+	lla	a1, fw_fdt_bin
 #endif
 
 	/*
@@ -202,7 +202,7 @@ _bss_zero:
 	 * s7 -> HART Count
 	 * s8 -> HART Stack Size
 	 */
-	la	a4, platform
+	lla	a4, platform
 #if __riscv_xlen == 64
 	lwu	s7, SBI_PLATFORM_HART_COUNT_OFFSET(a4)
 	lwu	s8, SBI_PLATFORM_HART_STACK_SIZE_OFFSET(a4)
@@ -212,7 +212,7 @@ _bss_zero:
 #endif
 
 	/* Setup scratch space for all the HARTs*/
-	la	tp, _fw_end
+	lla	tp, _fw_end
 	mul	a5, s7, s8
 	add	tp, tp, a5
 	/* Keep a copy of tp */
@@ -230,8 +230,8 @@ _scratch_init:
 
 	/* Initialize scratch space */
 	/* Store fw_start and fw_size in scratch space */
-	la	a4, _fw_start
-	la	a5, _fw_end
+	lla	a4, _fw_start
+	lla	a5, _fw_end
 	mul	t0, s7, s8
 	add	a5, a5, t0
 	sub	a5, a5, a4
@@ -253,16 +253,16 @@ _scratch_init:
 	REG_S	a0, SBI_SCRATCH_NEXT_MODE_OFFSET(tp)
 	MOV_3R	a0, s0, a1, s1, a2, s2
 	/* Store warm_boot address in scratch space */
-	la	a4, _start_warm
+	lla	a4, _start_warm
 	REG_S	a4, SBI_SCRATCH_WARMBOOT_ADDR_OFFSET(tp)
 	/* Store platform address in scratch space */
-	la	a4, platform
+	lla	a4, platform
 	REG_S	a4, SBI_SCRATCH_PLATFORM_ADDR_OFFSET(tp)
 	/* Store hartid-to-scratch function address in scratch space */
-	la	a4, _hartid_to_scratch
+	lla	a4, _hartid_to_scratch
 	REG_S	a4, SBI_SCRATCH_HARTID_TO_SCRATCH_OFFSET(tp)
 	/* Store trap-exit function address in scratch space */
-	la	a4, _trap_exit
+	lla	a4, _trap_exit
 	REG_S	a4, SBI_SCRATCH_TRAP_EXIT_OFFSET(tp)
 	/* Clear tmp0 in scratch space */
 	REG_S	zero, SBI_SCRATCH_TMP0_OFFSET(tp)
@@ -343,7 +343,7 @@ _fdt_reloc_done:
 
 	/* mark boot hart done */
 	li	t0, BOOT_STATUS_BOOT_HART_DONE
-	la	t1, _boot_status
+	lla	t1, _boot_status
 	REG_S	t0, 0(t1)
 	fence	rw, rw
 	j	_start_warm
@@ -351,7 +351,7 @@ _fdt_reloc_done:
 	/* waiting for boot hart to be done (_boot_status == 2) */
 _wait_for_boot_hart:
 	li	t0, BOOT_STATUS_BOOT_HART_DONE
-	la	t1, _boot_status
+	lla	t1, _boot_status
 	REG_L	t1, 0(t1)
 	/* Reduce the bus traffic so that boot hart may proceed faster */
 	nop
@@ -369,7 +369,7 @@ _start_warm:
 	csrw	CSR_MIP, zero
 
 	/* Find HART count and HART stack size */
-	la	a4, platform
+	lla	a4, platform
 #if __riscv_xlen == 64
 	lwu	s7, SBI_PLATFORM_HART_COUNT_OFFSET(a4)
 	lwu	s8, SBI_PLATFORM_HART_STACK_SIZE_OFFSET(a4)
@@ -400,7 +400,7 @@ _start_warm:
 3:	bge	s6, s7, _start_hang
 
 	/* Find the scratch space based on HART index */
-	la	tp, _fw_end
+	lla	tp, _fw_end
 	mul	a5, s7, s8
 	add	tp, tp, a5
 	mul	a5, s8, s6
@@ -415,13 +415,13 @@ _start_warm:
 	add	sp, tp, zero
 
 	/* Setup trap handler */
-	la	a4, _trap_handler
+	lla	a4, _trap_handler
 #if __riscv_xlen == 32
 	csrr	a5, CSR_MISA
 	srli	a5, a5, ('H' - 'A')
 	andi	a5, a5, 0x1
 	beq	a5, zero, _skip_trap_handler_rv32_hyp
-	la	a4, _trap_handler_rv32_hyp
+	lla	a4, _trap_handler_rv32_hyp
 _skip_trap_handler_rv32_hyp:
 #endif
 	csrw	CSR_MTVEC, a4
@@ -432,7 +432,7 @@ _skip_trap_handler_rv32_hyp:
 	srli	a5, a5, ('H' - 'A')
 	andi	a5, a5, 0x1
 	beq	a5, zero, _skip_trap_exit_rv32_hyp
-	la	a4, _trap_exit_rv32_hyp
+	lla	a4, _trap_exit_rv32_hyp
 	csrr	a5, CSR_MSCRATCH
 	REG_S	a4, SBI_SCRATCH_TRAP_EXIT_OFFSET(a5)
 _skip_trap_exit_rv32_hyp:
@@ -468,7 +468,7 @@ _hartid_to_scratch:
 	 * t1 -> HART Stack End
 	 * t2 -> Temporary
 	 */
-	la	t2, platform
+	lla	t2, platform
 #if __riscv_xlen == 64
 	lwu	t0, SBI_PLATFORM_HART_STACK_SIZE_OFFSET(t2)
 	lwu	t2, SBI_PLATFORM_HART_COUNT_OFFSET(t2)
@@ -478,7 +478,7 @@ _hartid_to_scratch:
 #endif
 	sub	t2, t2, a1
 	mul	t2, t2, t0
-	la	t1, _fw_end
+	lla	t1, _fw_end
 	add	t1, t1, t2
 	li	t2, SBI_SCRATCH_SIZE
 	sub	a0, t1, t2
firmware/fw_dynamic.S +9 -9
index 8b56947..0705e63 100644
@@ -54,7 +54,7 @@ fw_boot_hart:
 	 */
 fw_save_info:
 	/* Save next arg1 in 'a1' */
-	la	a4, _dynamic_next_arg1
+	lla	a4, _dynamic_next_arg1
 	REG_S	a1, (a4)
 
 	/* Sanity checks */
@@ -66,13 +66,13 @@ fw_save_info:
 	bgt	a3, a4, _bad_dynamic_info
 
 	/* Save version == 0x1 fields */
-	la	a4, _dynamic_next_addr
+	lla	a4, _dynamic_next_addr
 	REG_L	a3, FW_DYNAMIC_INFO_NEXT_ADDR_OFFSET(a2)
 	REG_S	a3, (a4)
-	la	a4, _dynamic_next_mode
+	lla	a4, _dynamic_next_mode
 	REG_L	a3, FW_DYNAMIC_INFO_NEXT_MODE_OFFSET(a2)
 	REG_S	a3, (a4)
-	la	a4, _dynamic_options
+	lla	a4, _dynamic_options
 	REG_L	a3, FW_DYNAMIC_INFO_OPTIONS_OFFSET(a2)
 	REG_S	a3, (a4)
 
@@ -80,7 +80,7 @@ fw_save_info:
 	li	a4, 0x2
 	REG_L	a3, FW_DYNAMIC_INFO_VERSION_OFFSET(a2)
 	blt	a3, a4, 2f
-	la	a4, _dynamic_boot_hart
+	lla	a4, _dynamic_boot_hart
 	REG_L	a3, FW_DYNAMIC_INFO_BOOT_HART_OFFSET(a2)
 	REG_S	a3, (a4)
 2:
@@ -96,7 +96,7 @@ fw_save_info:
 	 * The next arg1 should be returned in 'a0'.
 	 */
 fw_next_arg1:
-	la	a0, _dynamic_next_arg1
+	lla	a0, _dynamic_next_arg1
 	REG_L	a0, (a0)
 	ret
 
@@ -108,7 +108,7 @@ fw_next_arg1:
 	 * The next address should be returned in 'a0'.
 	 */
 fw_next_addr:
-	la	a0, _dynamic_next_addr
+	lla	a0, _dynamic_next_addr
 	REG_L	a0, (a0)
 	ret
 
@@ -120,7 +120,7 @@ fw_next_addr:
 	 * The next address should be returned in 'a0'
 	 */
 fw_next_mode:
-	la	a0, _dynamic_next_mode
+	lla	a0, _dynamic_next_mode
 	REG_L	a0, (a0)
 	ret
 
@@ -133,7 +133,7 @@ fw_next_mode:
 	 * The next address should be returned in 'a0'.
 	 */
 fw_options:
-	la	a0, _dynamic_options
+	lla	a0, _dynamic_options
 	REG_L	a0, (a0)
 	ret
 
firmware/fw_jump.S +1 -1
index 8553f8c..5b24f8b 100644
@@ -59,7 +59,7 @@ fw_next_arg1:
 	 * The next address should be returned in 'a0'.
 	 */
 fw_next_addr:
-	la	a0, _jump_addr
+	lla	a0, _jump_addr
 	REG_L	a0, (a0)
 	ret
 
firmware/fw_payload.S +1 -1
index 1ef121e..c53a3bb 100644
@@ -59,7 +59,7 @@ fw_next_arg1:
 	 * The next address should be returned in 'a0'.
 	 */
 fw_next_addr:
-	la	a0, payload_bin
+	lla	a0, payload_bin
 	ret
 
 	.section .entry, "ax", %progbits
firmware/payloads/test_head.S +9 -9
index 840013e..4852f71 100644
@@ -28,20 +28,20 @@
 	.globl _start
 _start:
 	/* Pick one hart to run the main boot sequence */
-	la	a3, _hart_lottery
+	lla	a3, _hart_lottery
 	li	a2, 1
 	amoadd.w a3, a2, (a3)
 	bnez	a3, _start_hang
 
 	/* Save a0 and a1 */
-	la	a3, _boot_a0
+	lla	a3, _boot_a0
 	REG_S	a0, 0(a3)
-	la	a3, _boot_a1
+	lla	a3, _boot_a1
 	REG_S	a1, 0(a3)
 
 	/* Zero-out BSS */
-	la	a4, _bss_start
-	la	a5, _bss_end
+	lla	a4, _bss_start
+	lla	a5, _bss_end
 _bss_zero:
 	REG_S	zero, (a4)
 	add	a4, a4, __SIZEOF_POINTER__
@@ -53,18 +53,18 @@ _start_warm:
 	csrw	CSR_SIP, zero
 
 	/* Setup exception vectors */
-	la	a3, _start_hang
+	lla	a3, _start_hang
 	csrw	CSR_STVEC, a3
 
 	/* Setup stack */
-	la	a3, _payload_end
+	lla	a3, _payload_end
 	li	a4, 0x2000
 	add	sp, a3, a4
 
 	/* Jump to C main */
-	la	a3, _boot_a0
+	lla	a3, _boot_a0
 	REG_L	a0, 0(a3)
-	la	a3, _boot_a1
+	lla	a3, _boot_a1
 	REG_L	a1, 0(a3)
 	call	test_main
 
Signature · 1 line
2.7.4
#2 lore raw

PATCH v2 1/2 firmware: Use lla to access all global symbols

Quoted 19 lines

-----Original Message-----
From: opensbi <[email protected]> On Behalf Of Vincent Chen
Sent: 05 March 2021 14:34
To: opensbi at lists.infradead.org
Cc: Vincent Chen <[email protected]>
Subject: [PATCH v2 1/2] firmware: Use lla to access all global symbols

When OpenSBI is compiled as fPIE mode, the assembler will translate "la" to GOT reference pattern. It will cause to cost an additional load instruction when obtaining the symbol address. However, if the symbol locates within the positive or negative 2GB region, we can use "lla"
instead of "la" to avoid unneeded GOT references. This patch assumes that the OpenSBI image excluding the payload does not exceed 2GB. Based on this assumption, all "la" instructions are replaced by "lla" to avoid performance degradation when compiling as fPIE mode.

Quoted 1 line

Looks good to me.

Regards,
Anup

Quoted 435 lines
---
 firmware/fw_base.S            | 88 +++++++++++++++++++++----------------------
 firmware/fw_dynamic.S         | 18 ++++-----
 firmware/fw_jump.S            |  2 +-
 firmware/fw_payload.S         |  2 +-
 firmware/payloads/test_head.S | 18 ++++-----
 5 files changed, 64 insertions(+), 64 deletions(-)
firmware/fw_base.S index renamed from firmware/fw_base.S +0 -0

ab33e11..6cc5f88 100644

firmware/fw_base.S +44 -44
@@ -57,39 +57,39 @@ _start:
 	bne	a0, a6, _wait_relocate_copy_done
 _try_lottery:
 	/* Jump to relocation wait loop if we don't get relocation lottery */
-	la	a6, _relocate_lottery
+	lla	a6, _relocate_lottery
 	li	a7, 1
 	amoadd.w a6, a7, (a6)
 	bnez	a6, _wait_relocate_copy_done
 
 	/* Save load address */
-	la	t0, _load_start
-	la	t1, _start
+	lla	t0, _load_start
+	lla	t1, _start
 	REG_S	t1, 0(t0)
 
 	/* Relocate if load address != link address */
 _relocate:
-	la	t0, _link_start
+	lla	t0, _link_start
 	REG_L	t0, 0(t0)
-	la	t1, _link_end
+	lla	t1, _link_end
 	REG_L	t1, 0(t1)
-	la	t2, _load_start
+	lla	t2, _load_start
 	REG_L	t2, 0(t2)
 	sub	t3, t1, t0
 	add	t3, t3, t2
 	beq	t0, t2, _relocate_done
-	la	t4, _relocate_done
+	lla	t4, _relocate_done
 	sub	t4, t4, t2
 	add	t4, t4, t0
 	blt	t2, t0, _relocate_copy_to_upper
 _relocate_copy_to_lower:
 	ble	t1, t2, _relocate_copy_to_lower_loop
-	la	t3, _relocate_lottery
+	lla	t3, _relocate_lottery
 	BRANGE	t2, t1, t3, _start_hang
-	la	t3, _boot_status
+	lla	t3, _boot_status
 	BRANGE	t2, t1, t3, _start_hang
-	la	t3, _relocate
-	la	t5, _relocate_done
+	lla	t3, _relocate
+	lla	t5, _relocate_done
 	BRANGE	t2, t1, t3, _start_hang
 	BRANGE	t2, t1, t5, _start_hang
 	BRANGE  t3, t5, t2, _start_hang
@@ -102,12 +102,12 @@ _relocate_copy_to_lower_loop:
 	jr	t4
 _relocate_copy_to_upper:
 	ble	t3, t0, _relocate_copy_to_upper_loop
-	la	t2, _relocate_lottery
+	lla	t2, _relocate_lottery
 	BRANGE	t0, t3, t2, _start_hang
-	la	t2, _boot_status
+	lla	t2, _boot_status
 	BRANGE	t0, t3, t2, _start_hang
-	la	t2, _relocate
-	la	t5, _relocate_done
+	lla	t2, _relocate
+	lla	t5, _relocate_done
 	BRANGE	t0, t3, t2, _start_hang
 	BRANGE	t0, t3, t5, _start_hang
 	BRANGE	t2, t5, t0, _start_hang
@@ -119,12 +119,12 @@ _relocate_copy_to_upper_loop:
 	blt	t0, t1, _relocate_copy_to_upper_loop
 	jr	t4
 _wait_relocate_copy_done:
-	la	t0, _start
-	la	t1, _link_start
+	lla	t0, _start
+	lla	t1, _link_start
 	REG_L	t1, 0(t1)
 	beq	t0, t1, _wait_for_boot_hart
-	la	t2, _boot_status
-	la	t3, _wait_for_boot_hart
+	lla	t2, _boot_status
+	lla	t3, _wait_for_boot_hart
 	sub	t3, t3, t0
 	add	t3, t3, t1
 1:
@@ -143,10 +143,10 @@ _relocate_done:
 	 * Mark relocate copy done
 	 * Use _boot_status copy relative to the load address
 	 */
-	la	t0, _boot_status
-	la	t1, _link_start
+	lla	t0, _boot_status
+	lla	t1, _link_start
 	REG_L	t1, 0(t1)
-	la	t2, _load_start
+	lla	t2, _load_start
 	REG_L	t2, 0(t2)
 	sub	t0, t0, t1
 	add	t0, t0, t2
@@ -161,19 +161,19 @@ _relocate_done:
 	call	_reset_regs
 
 	/* Zero-out BSS */
-	la	s4, _bss_start
-	la	s5, _bss_end
+	lla	s4, _bss_start
+	lla	s5, _bss_end
 _bss_zero:
 	REG_S	zero, (s4)
 	add	s4, s4, __SIZEOF_POINTER__
 	blt	s4, s5, _bss_zero
 
 	/* Setup temporary trap handler */
-	la	s4, _start_hang
+	lla	s4, _start_hang
 	csrw	CSR_MTVEC, s4
 
 	/* Setup temporary stack */
-	la	s4, _fw_end
+	lla	s4, _fw_end
 	li	s5, (SBI_SCRATCH_SIZE * 2)
 	add	sp, s4, s5
 
@@ -184,7 +184,7 @@ _bss_zero:
 
 #ifdef FW_FDT_PATH
 	/* Override previous arg1 */
-	la	a1, fw_fdt_bin
+	lla	a1, fw_fdt_bin
 #endif
 
 	/*
@@ -202,7 +202,7 @@ _bss_zero:
 	 * s7 -> HART Count
 	 * s8 -> HART Stack Size
 	 */
-	la	a4, platform
+	lla	a4, platform
 #if __riscv_xlen == 64
 	lwu	s7, SBI_PLATFORM_HART_COUNT_OFFSET(a4)
 	lwu	s8, SBI_PLATFORM_HART_STACK_SIZE_OFFSET(a4)
@@ -212,7 +212,7 @@ _bss_zero:
 #endif
 
 	/* Setup scratch space for all the HARTs*/
-	la	tp, _fw_end
+	lla	tp, _fw_end
 	mul	a5, s7, s8
 	add	tp, tp, a5
 	/* Keep a copy of tp */
@@ -230,8 +230,8 @@ _scratch_init:
 
 	/* Initialize scratch space */
 	/* Store fw_start and fw_size in scratch space */
-	la	a4, _fw_start
-	la	a5, _fw_end
+	lla	a4, _fw_start
+	lla	a5, _fw_end
 	mul	t0, s7, s8
 	add	a5, a5, t0
 	sub	a5, a5, a4
@@ -253,16 +253,16 @@ _scratch_init:
 	REG_S	a0, SBI_SCRATCH_NEXT_MODE_OFFSET(tp)
 	MOV_3R	a0, s0, a1, s1, a2, s2
 	/* Store warm_boot address in scratch space */
-	la	a4, _start_warm
+	lla	a4, _start_warm
 	REG_S	a4, SBI_SCRATCH_WARMBOOT_ADDR_OFFSET(tp)
 	/* Store platform address in scratch space */
-	la	a4, platform
+	lla	a4, platform
 	REG_S	a4, SBI_SCRATCH_PLATFORM_ADDR_OFFSET(tp)
 	/* Store hartid-to-scratch function address in scratch space */
-	la	a4, _hartid_to_scratch
+	lla	a4, _hartid_to_scratch
 	REG_S	a4, SBI_SCRATCH_HARTID_TO_SCRATCH_OFFSET(tp)
 	/* Store trap-exit function address in scratch space */
-	la	a4, _trap_exit
+	lla	a4, _trap_exit
 	REG_S	a4, SBI_SCRATCH_TRAP_EXIT_OFFSET(tp)
 	/* Clear tmp0 in scratch space */
 	REG_S	zero, SBI_SCRATCH_TMP0_OFFSET(tp)
@@ -343,7 +343,7 @@ _fdt_reloc_done:
 
 	/* mark boot hart done */
 	li	t0, BOOT_STATUS_BOOT_HART_DONE
-	la	t1, _boot_status
+	lla	t1, _boot_status
 	REG_S	t0, 0(t1)
 	fence	rw, rw
 	j	_start_warm
@@ -351,7 +351,7 @@ _fdt_reloc_done:
 	/* waiting for boot hart to be done (_boot_status == 2) */
 _wait_for_boot_hart:
 	li	t0, BOOT_STATUS_BOOT_HART_DONE
-	la	t1, _boot_status
+	lla	t1, _boot_status
 	REG_L	t1, 0(t1)
 	/* Reduce the bus traffic so that boot hart may proceed faster */
 	nop
@@ -369,7 +369,7 @@ _start_warm:
 	csrw	CSR_MIP, zero
 
 	/* Find HART count and HART stack size */
-	la	a4, platform
+	lla	a4, platform
 #if __riscv_xlen == 64
 	lwu	s7, SBI_PLATFORM_HART_COUNT_OFFSET(a4)
 	lwu	s8, SBI_PLATFORM_HART_STACK_SIZE_OFFSET(a4)
@@ -400,7 +400,7 @@ _start_warm:
 3:	bge	s6, s7, _start_hang
 
 	/* Find the scratch space based on HART index */
-	la	tp, _fw_end
+	lla	tp, _fw_end
 	mul	a5, s7, s8
 	add	tp, tp, a5
 	mul	a5, s8, s6
@@ -415,13 +415,13 @@ _start_warm:
 	add	sp, tp, zero
 
 	/* Setup trap handler */
-	la	a4, _trap_handler
+	lla	a4, _trap_handler
 #if __riscv_xlen == 32
 	csrr	a5, CSR_MISA
 	srli	a5, a5, ('H' - 'A')
 	andi	a5, a5, 0x1
 	beq	a5, zero, _skip_trap_handler_rv32_hyp
-	la	a4, _trap_handler_rv32_hyp
+	lla	a4, _trap_handler_rv32_hyp
 _skip_trap_handler_rv32_hyp:
 #endif
 	csrw	CSR_MTVEC, a4
@@ -432,7 +432,7 @@ _skip_trap_handler_rv32_hyp:
 	srli	a5, a5, ('H' - 'A')
 	andi	a5, a5, 0x1
 	beq	a5, zero, _skip_trap_exit_rv32_hyp
-	la	a4, _trap_exit_rv32_hyp
+	lla	a4, _trap_exit_rv32_hyp
 	csrr	a5, CSR_MSCRATCH
 	REG_S	a4, SBI_SCRATCH_TRAP_EXIT_OFFSET(a5)
 _skip_trap_exit_rv32_hyp:
@@ -468,7 +468,7 @@ _hartid_to_scratch:
 	 * t1 -> HART Stack End
 	 * t2 -> Temporary
 	 */
-	la	t2, platform
+	lla	t2, platform
 #if __riscv_xlen == 64
 	lwu	t0, SBI_PLATFORM_HART_STACK_SIZE_OFFSET(t2)
 	lwu	t2, SBI_PLATFORM_HART_COUNT_OFFSET(t2)
@@ -478,7 +478,7 @@ _hartid_to_scratch:
 #endif
 	sub	t2, t2, a1
 	mul	t2, t2, t0
-	la	t1, _fw_end
+	lla	t1, _fw_end
 	add	t1, t1, t2
 	li	t2, SBI_SCRATCH_SIZE
 	sub	a0, t1, t2
firmware/fw_dynamic.S index renamed from firmware/fw_dynamic.S +0 -0

8b56947..0705e63 100644

firmware/fw_dynamic.S +9 -9
@@ -54,7 +54,7 @@ fw_boot_hart:
 	 */
 fw_save_info:
 	/* Save next arg1 in 'a1' */
-	la	a4, _dynamic_next_arg1
+	lla	a4, _dynamic_next_arg1
 	REG_S	a1, (a4)
 
 	/* Sanity checks */
@@ -66,13 +66,13 @@ fw_save_info:
 	bgt	a3, a4, _bad_dynamic_info
 
 	/* Save version == 0x1 fields */
-	la	a4, _dynamic_next_addr
+	lla	a4, _dynamic_next_addr
 	REG_L	a3, FW_DYNAMIC_INFO_NEXT_ADDR_OFFSET(a2)
 	REG_S	a3, (a4)
-	la	a4, _dynamic_next_mode
+	lla	a4, _dynamic_next_mode
 	REG_L	a3, FW_DYNAMIC_INFO_NEXT_MODE_OFFSET(a2)
 	REG_S	a3, (a4)
-	la	a4, _dynamic_options
+	lla	a4, _dynamic_options
 	REG_L	a3, FW_DYNAMIC_INFO_OPTIONS_OFFSET(a2)
 	REG_S	a3, (a4)
 
@@ -80,7 +80,7 @@ fw_save_info:
 	li	a4, 0x2
 	REG_L	a3, FW_DYNAMIC_INFO_VERSION_OFFSET(a2)
 	blt	a3, a4, 2f
-	la	a4, _dynamic_boot_hart
+	lla	a4, _dynamic_boot_hart
 	REG_L	a3, FW_DYNAMIC_INFO_BOOT_HART_OFFSET(a2)
 	REG_S	a3, (a4)
 2:
@@ -96,7 +96,7 @@ fw_save_info:
 	 * The next arg1 should be returned in 'a0'.
 	 */
 fw_next_arg1:
-	la	a0, _dynamic_next_arg1
+	lla	a0, _dynamic_next_arg1
 	REG_L	a0, (a0)
 	ret
 
@@ -108,7 +108,7 @@ fw_next_arg1:
 	 * The next address should be returned in 'a0'.
 	 */
 fw_next_addr:
-	la	a0, _dynamic_next_addr
+	lla	a0, _dynamic_next_addr
 	REG_L	a0, (a0)
 	ret
 
@@ -120,7 +120,7 @@ fw_next_addr:
 	 * The next address should be returned in 'a0'
 	 */
 fw_next_mode:
-	la	a0, _dynamic_next_mode
+	lla	a0, _dynamic_next_mode
 	REG_L	a0, (a0)
 	ret
 
@@ -133,7 +133,7 @@ fw_next_mode:
 	 * The next address should be returned in 'a0'.
 	 */
 fw_options:
-	la	a0, _dynamic_options
+	lla	a0, _dynamic_options
 	REG_L	a0, (a0)
 	ret
 
firmware/fw_jump.S index renamed from firmware/fw_jump.S +0 -0

8553f8c..5b24f8b 100644

firmware/fw_jump.S +1 -1
@@ -59,7 +59,7 @@ fw_next_arg1:
 	 * The next address should be returned in 'a0'.
 	 */
 fw_next_addr:
-	la	a0, _jump_addr
+	lla	a0, _jump_addr
 	REG_L	a0, (a0)
 	ret
 
firmware/fw_payload.S index renamed from firmware/fw_payload.S +0 -0

1ef121e..c53a3bb 100644

firmware/fw_payload.S +1 -1
@@ -59,7 +59,7 @@ fw_next_arg1:
 	 * The next address should be returned in 'a0'.
 	 */
 fw_next_addr:
-	la	a0, payload_bin
+	lla	a0, payload_bin
 	ret
 
 	.section .entry, "ax", %progbits
(quoted diff) +0 -0

b/firmware/payloads/test_head.S index 840013e..4852f71 100644

firmware/payloads/test_head.S +9 -10
@@ -28,20 +28,20 @@
 	.globl _start
 _start:
 	/* Pick one hart to run the main boot sequence */
-	la	a3, _hart_lottery
+	lla	a3, _hart_lottery
 	li	a2, 1
 	amoadd.w a3, a2, (a3)
 	bnez	a3, _start_hang
 
 	/* Save a0 and a1 */
-	la	a3, _boot_a0
+	lla	a3, _boot_a0
 	REG_S	a0, 0(a3)
-	la	a3, _boot_a1
+	lla	a3, _boot_a1
 	REG_S	a1, 0(a3)
 
 	/* Zero-out BSS */
-	la	a4, _bss_start
-	la	a5, _bss_end
+	lla	a4, _bss_start
+	lla	a5, _bss_end
 _bss_zero:
 	REG_S	zero, (a4)
 	add	a4, a4, __SIZEOF_POINTER__
@@ -53,18 +53,18 @@ _start_warm:
 	csrw	CSR_SIP, zero
 
 	/* Setup exception vectors */
-	la	a3, _start_hang
+	lla	a3, _start_hang
 	csrw	CSR_STVEC, a3
 
 	/* Setup stack */
-	la	a3, _payload_end
+	lla	a3, _payload_end
 	li	a4, 0x2000
 	add	sp, a3, a4
 
 	/* Jump to C main */
-	la	a3, _boot_a0
+	lla	a3, _boot_a0
 	REG_L	a0, 0(a3)
-	la	a3, _boot_a1
+	lla	a3, _boot_a1
 	REG_L	a1, 0(a3)
 	call	test_main
 
--

2.7.4

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

#3 lore raw

PATCH v2 2/2 firmware: Support position independent execution

In reply to Vincent Chen

Enable OpenSBI to support position independent execution. Because the position independent code will cause an additional GOT reference when accessing the global variables, it will reduce performance a bit. Therefore, the position independent execution is disabled by default. Users can through specifying "FW_PIC=y" on the make command to enable this feature.

In theory, after enabling position-independent execution, the OpenSBI can run at arbitrary address with appropriate alignment. Therefore, the original relocation mechanism will be skipped. In other words, OpenSBI will directly run at the load address without any code movement.

---
 Makefile                |  2 +-
 firmware/fw_base.S      | 59 ++++++++++++++++++++++++++++++++++++++++++++++++-
 firmware/fw_base.ldS    | 13 +++++++++++
 firmware/fw_jump.S      | 10 +++++++++
 firmware/fw_payload.S   |  6 +++++
 firmware/objects.mk     |  7 ++++++
 include/sbi/riscv_elf.h |  9 ++++++++
 7 files changed, 104 insertions(+), 2 deletions(-)
 create mode 100644 include/sbi/riscv_elf.h
Makefile +1 -1
index d6f097d..038cc99 100644
@@ -210,8 +210,8 @@ CFLAGS		+=	-mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 CFLAGS		+=	-mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
 CFLAGS		+=	$(GENFLAGS)
 CFLAGS		+=	$(platform-cflags-y)
-CFLAGS		+=	$(firmware-cflags-y)
 CFLAGS		+=	-fno-pie -no-pie
+CFLAGS		+=	$(firmware-cflags-y)
 
 CPPFLAGS	+=	$(GENFLAGS)
 CPPFLAGS	+=	$(platform-cppflags-y)
firmware/fw_base.S +58 -1

101 lines not on this page · raw message

firmware/fw_base.ldS +13 -0
index 0ac75f2..3904959 100644
@@ -61,6 +61,19 @@
 		PROVIDE(_data_end = .);
 	}
 
+	.dynsym : {
+	        PROVIDE(__dyn_sym_start = .);
+	        *(.dynsym)
+	        PROVIDE(__dyn_sym_end = .);
+	}
+
+	.rela.dyn : {
+	        PROVIDE(__rel_dyn_start = .);
+	        *(.rela*)
+		. = ALIGN(8);
+	        PROVIDE(__rel_dyn_end = .);
+	}
+
 	. = ALIGN(0x1000); /* Ensure next section is page aligned */
 
 	.bss :
firmware/fw_jump.S +10 -0
index 5b24f8b..4fb3e0c 100644
@@ -46,6 +46,11 @@ fw_save_info:
 fw_next_arg1:
 #ifdef FW_JUMP_FDT_ADDR
 	li	a0, FW_JUMP_FDT_ADDR
+#ifdef FW_PIC
+	lla	a1, _runtime_offset
+	REG_L	a1, (a1)
+	add	a0, a0, a1
+#endif
 #else
 	add	a0, a1, zero
 #endif
@@ -61,6 +66,11 @@ fw_next_arg1:
 fw_next_addr:
 	lla	a0, _jump_addr
 	REG_L	a0, (a0)
+#ifdef FW_PIC
+	lla	a1, _runtime_offset
+	REG_L	a1, (a1)
+	add	a0, a0, a1
+#endif
 	ret
 
 	.section .entry, "ax", %progbits
firmware/fw_payload.S +6 -0
index c53a3bb..00c7cb7 100644
@@ -46,6 +46,12 @@ fw_save_info:
 fw_next_arg1:
 #ifdef FW_PAYLOAD_FDT_ADDR
 	li	a0, FW_PAYLOAD_FDT_ADDR
+#ifdef FW_PIC
+	lla	a1, _runtime_offset
+	REG_L   a1, (a1)
+	add     a0, a0, a1
+#endif
+
 #else
 	add	a0, a1, zero
 #endif
firmware/objects.mk +7 -0
index b2ace75..c1f632e 100644
@@ -13,6 +13,13 @@ firmware-cflags-y +=
 firmware-asflags-y +=
 firmware-ldflags-y +=
 
+ifeq ($(FW_PIC),y)
+firmware-genflags-y +=	-DFW_PIC
+firmware-asflags-y  +=	-fpic
+firmware-cflags-y   +=	-fPIE -pie
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+endif
+
 ifdef FW_TEXT_START
 firmware-genflags-y += -DFW_TEXT_START=$(FW_TEXT_START)
 endif
include/sbi/riscv_elf.h new file +9 -0
new file mode 100644
index 0000000..cf211ac
@@ -0,0 +1,9 @@
+#include <sbi/riscv_asm.h>
+
+#define R_RISCV_32		1
+#define R_RISCV_64		2
+#define R_RISCV_RELATIVE	3
+
+#define RELOC_TYPE		__REG_SEL(R_RISCV_64, R_RISCV_32)
+#define SYM_INDEX		__REG_SEL(0x20,	0x8)
+#define SYM_SIZE		__REG_SEL(0x18,0x10)
Signature · 1 line
2.7.4
#4 lore raw

PATCH v2 2/2 firmware: Support position independent execution

Quoted 51 lines · Vincent Chen

Enable OpenSBI to support position independent execution. Because the position independent code will cause an additional GOT reference when accessing the global variables, it will reduce performance a bit. Therefore, the position independent execution is disabled by default. Users can through specifying "FW_PIC=y" on the make command to enable this feature.

In theory, after enabling position-independent execution, the OpenSBI can run at arbitrary address with appropriate alignment. Therefore, the original relocation mechanism will be skipped. In other words, OpenSBI will directly run at the load address without any code movement.

---
 Makefile                |  2 +-
 firmware/fw_base.S      | 59 ++++++++++++++++++++++++++++++++++++++++++++++++-
 firmware/fw_base.ldS    | 13 +++++++++++
 firmware/fw_jump.S      | 10 +++++++++
 firmware/fw_payload.S   |  6 +++++
 firmware/objects.mk     |  7 ++++++
 include/sbi/riscv_elf.h |  9 ++++++++
 7 files changed, 104 insertions(+), 2 deletions(-)
 create mode 100644 include/sbi/riscv_elf.h
Makefile +1 -1
index d6f097d..038cc99 100644
@@ -210,8 +210,8 @@ CFLAGS              +=      -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 CFLAGS         +=      -mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
 CFLAGS         +=      $(GENFLAGS)
 CFLAGS         +=      $(platform-cflags-y)
-CFLAGS         +=      $(firmware-cflags-y)
 CFLAGS         +=      -fno-pie -no-pie
+CFLAGS         +=      $(firmware-cflags-y)
 
 CPPFLAGS       +=      $(GENFLAGS)
 CPPFLAGS       +=      $(platform-cppflags-y)
firmware/fw_base.S +1 -0
index 6cc5f88..5f49c88 100644
@@ -9,6 +9,7 @@
 
 #include <sbi/riscv_asm.h>
 #include <sbi/riscv_encoding.h>
+#include <sbi/riscv_elf.h>
 #include <sbi/sbi_platform.h>
 #include <sbi/sbi_scratch.h>
 #include <sbi/sbi_trap.h>

firmware/fw_base.S _try_lottery: · line 74

Quoted 7 lines
firmware/fw_base.S +4 -0
@@ -67,6 +68,57 @@ _try_lottery:
        lla     t1, _start
        REG_S   t1, 0(t0)
 
+#ifdef FW_PIC
+       /* relocate the global table content */
+       lla     t0, _link_start
+       REG_L   t0, 0(t0)

The next instruction assumes that "t1" has address of _start.

I would suggest to have explicit "lla t1, _start" here OR we have to document this assumption here.

Quoted 74 lines
firmware/fw_base.S +50 -0
+       sub     t2, t1, t0
+       lla     t3, _runtime_offset
+       REG_S   t2, (t3)
+       lla     t0, __rel_dyn_start
+       lla     t1, __rel_dyn_end
+       beq     t0, t1, _relocate_done
+       j       5f
+2:
+       REG_L   t5, -(REGBYTES*2)(t0)   /* t5 <-- relocation info:type */
+       li      t3, R_RISCV_RELATIVE    /* reloc type R_RISCV_RELATIVE */
+       bne     t5, t3, 3f
+       REG_L   t3, -(REGBYTES*3)(t0)
+       REG_L   t5, -(REGBYTES)(t0)     /* t5 <-- addend */
+       add     t5, t5, t2
+       add     t3, t3, t2
+       REG_S   t5, 0(t3)               /* store runtime address to the GOT entry */
+       j       5f
+
+3:
+       lla     t4, __dyn_sym_start
+
+4:
+       REG_L   t5, -(REGBYTES*2)(t0)   /* t5 <-- relocation info:type */
+       srli    t6, t5, SYM_INDEX       /* t6 <--- sym table index */
+       andi    t5, t5, 0xFF            /* t5 <--- relocation type */
+       li      t3, RELOC_TYPE
+       bne     t5, t3, 5f
+
+       /* address R_RISCV_64 or R_RISCV_32 cases*/
+       REG_L   t3, -(REGBYTES*3)(t0)
+       li      t5, SYM_SIZE
+       mul     t6, t6, t5
+       add     s5, t4, t6
+       REG_L   t6, -(REGBYTES)(t0)     /* t0 <-- addend */
+       REG_L   t5, REGBYTES(s5)
+       add     t5, t5, t6
+       add     t5, t5, t2              /* t5 <-- location to fix up in RAM */
+       add     t3, t3, t2              /* t3 <-- location to fix up in RAM */
+       REG_S   t5, 0(t3)               /* store runtime address to the variable */
+
+5:
+       addi    t0, t0, (REGBYTES*3)
+       ble     t0, t1, 2b
+       j       _relocate_done
+_wait_relocate_copy_done:
+       j       _wait_for_boot_hart
+#else
        /* Relocate if load address != link address */
 _relocate:
        lla     t0, _link_start
@@ -137,6 +189,7 @@ _wait_relocate_copy_done:
        nop
        bgt     t4, t5, 1b
        jr      t3
+#endif
 _relocate_done:
 
        /*
@@ -144,12 +197,14 @@ _relocate_done:
         * Use _boot_status copy relative to the load address
         */
        lla     t0, _boot_status
+#ifndef FW_PIC
        lla     t1, _link_start
        REG_L   t1, 0(t1)
        lla     t2, _load_start
        REG_L   t2, 0(t2)
        sub     t0, t0, t1
        add     t0, t0, t2
+#endif
        li      t1, BOOT_STATUS_RELOCATE_DONE
        REG_S   t1, 0(t0)
        fence   rw, rw

firmware/fw_base.S _skip_trap_exit_rv32_hyp: · line 505

Quoted 5 lines
firmware/fw_base.S +2 -0
@@ -446,6 +501,8 @@ _skip_trap_exit_rv32_hyp:
        j       _start_hang
 
        .align 3
+_runtime_offset:
+       RISCV_PTR       0

Please add "#ifdef" around _runtime_offset

firmware/fw_base.S _boot_status: · line 513

Quoted 9 lines
firmware/fw_base.S +1 -1
 _relocate_lottery:
        RISCV_PTR       0
 _boot_status:
@@ -453,7 +510,7 @@ _boot_status:
 _load_start:
        RISCV_PTR       _fw_start
 _link_start:
-       RISCV_PTR       _fw_start
+       RISCV_PTR       FW_TEXT_START

Why do we need this changes ?

Quoted 8 lines
firmware/fw_base.S +0 -0
 _link_end:
        RISCV_PTR       _fw_reloc_end
 
firmware/fw_base.ldS +0 -0
index 0ac75f2..3904959 100644

firmware/fw_base.ldS line 67

Quoted 7 lines
firmware/fw_base.ldS +4 -0
@@ -61,6 +61,19 @@
                PROVIDE(_data_end = .);
        }
 
+       .dynsym : {
+               PROVIDE(__dyn_sym_start = .);
+               *(.dynsym)
+               PROVIDE(__dyn_sym_end = .);

Only use tabs for indentation here.

firmware/fw_base.ldS line 74

Quoted 7 lines
firmware/fw_base.ldS +7 -0
+       }
+
+       .rela.dyn : {
+               PROVIDE(__rel_dyn_start = .);
+               *(.rela*)
+               . = ALIGN(8);
+               PROVIDE(__rel_dyn_end = .);

Only use tabs for indentation here.

Quoted 22 lines
firmware/fw_base.ldS +2 -0
+       }
+
        . = ALIGN(0x1000); /* Ensure next section is page aligned */
 
        .bss :
firmware/fw_jump.S +5 -0
index 5b24f8b..4fb3e0c 100644
@@ -46,6 +46,11 @@ fw_save_info:
 fw_next_arg1:
 #ifdef FW_JUMP_FDT_ADDR
        li      a0, FW_JUMP_FDT_ADDR
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
 #else
        add     a0, a1, zero
 #endif

firmware/fw_jump.S fw_next_arg1: · line 74

Quoted 9 lines
firmware/fw_jump.S +5 -0
@@ -61,6 +66,11 @@ fw_next_arg1:
 fw_next_addr:
        lla     a0, _jump_addr
        REG_L   a0, (a0)
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
        ret

The changes in fw_next_addr() breaks the FW_JUMP when FW_PIC=y because FW_JUMP assumes a compile-time fixed address of next booting stage specified by FW_JUMP_ADDR.

Similarly, when FW_JUMP_FDT_ADDR is defined FW_JUMP will assume compile-time fixed address of FDT passed to next booting stage.

Please drop all changes from fw_jump.S

Quoted 7 lines
firmware/fw_jump.S +0 -0
        .section .entry, "ax", %progbits
firmware/fw_payload.S +0 -0
index c53a3bb..00c7cb7 100644

firmware/fw_payload.S fw_save_info: · line 54

Quoted 9 lines
firmware/fw_payload.S +6 -0
@@ -46,6 +46,12 @@ fw_save_info:
 fw_next_arg1:
 #ifdef FW_PAYLOAD_FDT_ADDR
        li      a0, FW_PAYLOAD_FDT_ADDR
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
+

Same as above.

The changes in fw_next_arg1() breaks FW_PAYLOAD when FW_PIC=y because FW_PAYLOAD assumes a compile-time fixed address of FDT passed to next booting-stage when FW_PAYLOAD_FDT_ADDR is defined.

Please drop all changes in fw_payload.S

include/sbi/riscv_elf.h

Quoted 27 lines
firmware/fw_payload.S +0 -0
 #else
        add     a0, a1, zero
 #endif
firmware/objects.mk +7 -0
index b2ace75..c1f632e 100644
@@ -13,6 +13,13 @@ firmware-cflags-y +=
 firmware-asflags-y +=
 firmware-ldflags-y +=
 
+ifeq ($(FW_PIC),y)
+firmware-genflags-y += -DFW_PIC
+firmware-asflags-y  += -fpic
+firmware-cflags-y   += -fPIE -pie
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+endif
+
 ifdef FW_TEXT_START
 firmware-genflags-y += -DFW_TEXT_START=$(FW_TEXT_START)
 endif
include/sbi/riscv_elf.h new file +0 -0
new file mode 100644
index 0000000..cf211ac
@@ -0,0 +1,9 @@

Please "#ifndef" and "#define" pair here to protect multiple header inclusions

include/sbi/riscv_elf.h line 9

Quoted 9 lines
include/sbi/riscv_elf.h new file +9 -0
+#include <sbi/riscv_asm.h>
+
+#define R_RISCV_32             1
+#define R_RISCV_64             2
+#define R_RISCV_RELATIVE       3
+
+#define RELOC_TYPE             __REG_SEL(R_RISCV_64, R_RISCV_32)
+#define SYM_INDEX              __REG_SEL(0x20, 0x8)
+#define SYM_SIZE               __REG_SEL(0x18,0x10)

Add "#endif" here.

Quoted 8 lines

--
2.7.4

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup

#5 lore raw

PATCH v2 2/2 firmware: Support position independent execution

Quoted 63 lines · Anup Patel
Quoted 51 lines · Vincent Chen

Enable OpenSBI to support position independent execution. Because the position independent code will cause an additional GOT reference when accessing the global variables, it will reduce performance a bit. Therefore, the position independent execution is disabled by default. Users can through specifying "FW_PIC=y" on the make command to enable this feature.

In theory, after enabling position-independent execution, the OpenSBI can run at arbitrary address with appropriate alignment. Therefore, the original relocation mechanism will be skipped. In other words, OpenSBI will directly run at the load address without any code movement.

---
 Makefile                |  2 +-
 firmware/fw_base.S      | 59 ++++++++++++++++++++++++++++++++++++++++++++++++-
 firmware/fw_base.ldS    | 13 +++++++++++
 firmware/fw_jump.S      | 10 +++++++++
 firmware/fw_payload.S   |  6 +++++
 firmware/objects.mk     |  7 ++++++
 include/sbi/riscv_elf.h |  9 ++++++++
 7 files changed, 104 insertions(+), 2 deletions(-)
 create mode 100644 include/sbi/riscv_elf.h
Makefile +1 -1
index d6f097d..038cc99 100644
@@ -210,8 +210,8 @@ CFLAGS              +=      -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 CFLAGS         +=      -mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
 CFLAGS         +=      $(GENFLAGS)
 CFLAGS         +=      $(platform-cflags-y)
-CFLAGS         +=      $(firmware-cflags-y)
 CFLAGS         +=      -fno-pie -no-pie
+CFLAGS         +=      $(firmware-cflags-y)
 
 CPPFLAGS       +=      $(GENFLAGS)
 CPPFLAGS       +=      $(platform-cppflags-y)
firmware/fw_base.S +1 -0
index 6cc5f88..5f49c88 100644
@@ -9,6 +9,7 @@
 
 #include <sbi/riscv_asm.h>
 #include <sbi/riscv_encoding.h>
+#include <sbi/riscv_elf.h>
 #include <sbi/sbi_platform.h>
 #include <sbi/sbi_scratch.h>
 #include <sbi/sbi_trap.h>

firmware/fw_base.S _try_lottery: · line 74

Quoted 7 lines
firmware/fw_base.S +4 -0
@@ -67,6 +68,57 @@ _try_lottery:
        lla     t1, _start
        REG_S   t1, 0(t0)
 
+#ifdef FW_PIC
+       /* relocate the global table content */
+       lla     t0, _link_start
+       REG_L   t0, 0(t0)

The next instruction assumes that "t1" has address of _start.

Quoted 3 lines

I would suggest to have explicit "lla t1, _start" here OR we have to document this assumption here.

"lla t1, _start" was executed before the 4 instructions of "sub t2, t1, t0", so I think it may be OK to assume that "t1" has the address of _start.

Quoted 76 lines
Quoted 76 lines
firmware/fw_base.S +50 -0
+       sub     t2, t1, t0
+       lla     t3, _runtime_offset
+       REG_S   t2, (t3)
+       lla     t0, __rel_dyn_start
+       lla     t1, __rel_dyn_end
+       beq     t0, t1, _relocate_done
+       j       5f
+2:
+       REG_L   t5, -(REGBYTES*2)(t0)   /* t5 <-- relocation info:type */
+       li      t3, R_RISCV_RELATIVE    /* reloc type R_RISCV_RELATIVE */
+       bne     t5, t3, 3f
+       REG_L   t3, -(REGBYTES*3)(t0)
+       REG_L   t5, -(REGBYTES)(t0)     /* t5 <-- addend */
+       add     t5, t5, t2
+       add     t3, t3, t2
+       REG_S   t5, 0(t3)               /* store runtime address to the GOT entry */
+       j       5f
+
+3:
+       lla     t4, __dyn_sym_start
+
+4:
+       REG_L   t5, -(REGBYTES*2)(t0)   /* t5 <-- relocation info:type */
+       srli    t6, t5, SYM_INDEX       /* t6 <--- sym table index */
+       andi    t5, t5, 0xFF            /* t5 <--- relocation type */
+       li      t3, RELOC_TYPE
+       bne     t5, t3, 5f
+
+       /* address R_RISCV_64 or R_RISCV_32 cases*/
+       REG_L   t3, -(REGBYTES*3)(t0)
+       li      t5, SYM_SIZE
+       mul     t6, t6, t5
+       add     s5, t4, t6
+       REG_L   t6, -(REGBYTES)(t0)     /* t0 <-- addend */
+       REG_L   t5, REGBYTES(s5)
+       add     t5, t5, t6
+       add     t5, t5, t2              /* t5 <-- location to fix up in RAM */
+       add     t3, t3, t2              /* t3 <-- location to fix up in RAM */
+       REG_S   t5, 0(t3)               /* store runtime address to the variable */
+
+5:
+       addi    t0, t0, (REGBYTES*3)
+       ble     t0, t1, 2b
+       j       _relocate_done
+_wait_relocate_copy_done:
+       j       _wait_for_boot_hart
+#else
        /* Relocate if load address != link address */
 _relocate:
        lla     t0, _link_start
@@ -137,6 +189,7 @@ _wait_relocate_copy_done:
        nop
        bgt     t4, t5, 1b
        jr      t3
+#endif
 _relocate_done:
 
        /*
@@ -144,12 +197,14 @@ _relocate_done:
         * Use _boot_status copy relative to the load address
         */
        lla     t0, _boot_status
+#ifndef FW_PIC
        lla     t1, _link_start
        REG_L   t1, 0(t1)
        lla     t2, _load_start
        REG_L   t2, 0(t2)
        sub     t0, t0, t1
        add     t0, t0, t2
+#endif
        li      t1, BOOT_STATUS_RELOCATE_DONE
        REG_S   t1, 0(t0)
        fence   rw, rw
@@ -446,6 +501,8 @@ _skip_trap_exit_rv32_hyp:
        j       _start_hang
 
Quoted 6 lines

firmware/fw_base.S _skip_trap_exit_rv32_hyp: · line 505

Quoted 3 lines
firmware/fw_base.S +2 -0
        .align 3
+_runtime_offset:
+       RISCV_PTR       0

Please add "#ifdef" around _runtime_offset

OK, thanks for the reminder.

Quoted 12 lines

firmware/fw_base.S _boot_status: · line 513

Quoted 9 lines
firmware/fw_base.S +1 -1
 _relocate_lottery:
        RISCV_PTR       0
 _boot_status:
@@ -453,7 +510,7 @@ _boot_status:
 _load_start:
        RISCV_PTR       _fw_start
 _link_start:
-       RISCV_PTR       _fw_start
+       RISCV_PTR       FW_TEXT_START

Why do we need this changes ?

The _fw_start is a global variable. When we use -fPIE to compile the code, the linker does not know the exact address at runtime. Therefore, the linker will fill 0 here and create a relocation entry at the ".rela.dyn" section to let the dynamic loader resolve it at runtime.

_load_start:
RISCV_PTR _fw_start
_link_start:
RISCV_PTR _fw_start

1. objdump -D fw_payload.elf
0000000080000400 <_link_start>:

        ...

0000000080000408 <_link_end>:

        ...

0000000080000410 <_hartid_to_scratch>:

2. readelf -a fw_payload.elf
Relocation section '.rela.dyn' at offset 0x12f38 contains 175 entries:
Offset Info Type Sym. Value Sym. Name + Addend

0000800003f8  000000000003 R_RISCV_RELATIVE                     80000000
000080000400  000000000003 R_RISCV_RELATIVE                     80000000
000080000408  000000000003 R_RISCV_RELATIVE                     81ae3e08

However, from the code, I deduce that the variable _link_start is used to store the starting address of the image at compile time. Therefore, I change the _fw_start to FW_TEXT_START to ensure the value of _link_start is the exact start address of the image at the compile-time. The dynamic loader can use _link_start to calculate the offset between run time and compile-time as well.

Quoted 11 lines
Quoted 11 lines
firmware/fw_base.S +0 -0
 _link_end:
        RISCV_PTR       _fw_reloc_end
 
firmware/fw_base.ldS +0 -0
index 0ac75f2..3904959 100644
@@ -61,6 +61,19 @@
                PROVIDE(_data_end = .);
        }
 
Quoted 7 lines

firmware/fw_base.ldS line 67

Quoted 4 lines
firmware/fw_base.ldS +4 -0
+       .dynsym : {
+               PROVIDE(__dyn_sym_start = .);
+               *(.dynsym)
+               PROVIDE(__dyn_sym_end = .);

Only use tabs for indentation here.

OK

Quoted 10 lines

firmware/fw_base.ldS line 74

Quoted 7 lines
firmware/fw_base.ldS +7 -0
+       }
+
+       .rela.dyn : {
+               PROVIDE(__rel_dyn_start = .);
+               *(.rela*)
+               . = ALIGN(8);
+               PROVIDE(__rel_dyn_end = .);

Only use tabs for indentation here.

OK

Quoted 36 lines
Quoted 22 lines
firmware/fw_base.ldS +2 -0
+       }
+
        . = ALIGN(0x1000); /* Ensure next section is page aligned */
 
        .bss :
firmware/fw_jump.S +5 -0
index 5b24f8b..4fb3e0c 100644
@@ -46,6 +46,11 @@ fw_save_info:
 fw_next_arg1:
 #ifdef FW_JUMP_FDT_ADDR
        li      a0, FW_JUMP_FDT_ADDR
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
 #else
        add     a0, a1, zero
 #endif

firmware/fw_jump.S fw_next_arg1: · line 74

Quoted 9 lines
firmware/fw_jump.S +5 -0
@@ -61,6 +66,11 @@ fw_next_arg1:
 fw_next_addr:
        lla     a0, _jump_addr
        REG_L   a0, (a0)
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
        ret

The changes in fw_next_addr() breaks the FW_JUMP when FW_PIC=y because FW_JUMP assumes a compile-time fixed address of next booting stage specified by FW_JUMP_ADDR.

Quoted 6 lines

Similarly, when FW_JUMP_FDT_ADDR is defined FW_JUMP will assume compile-time fixed address of FDT passed to next booting stage.

Please drop all changes from fw_jump.S

OK, I will drop it.

Quoted 19 lines
Quoted 7 lines
firmware/fw_jump.S +0 -0
        .section .entry, "ax", %progbits
firmware/fw_payload.S +0 -0
index c53a3bb..00c7cb7 100644

firmware/fw_payload.S fw_save_info: · line 54

Quoted 9 lines
firmware/fw_payload.S +6 -0
@@ -46,6 +46,12 @@ fw_save_info:
 fw_next_arg1:
 #ifdef FW_PAYLOAD_FDT_ADDR
        li      a0, FW_PAYLOAD_FDT_ADDR
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
+

Same as above.

Quoted 7 lines

The changes in fw_next_arg1() breaks FW_PAYLOAD when FW_PIC=y because FW_PAYLOAD assumes a compile-time fixed address of FDT passed to next booting-stage when FW_PAYLOAD_FDT_ADDR is defined.

Please drop all changes in fw_payload.S

OK

Quoted 22 lines
Quoted 22 lines
firmware/fw_payload.S +0 -0
 #else
        add     a0, a1, zero
 #endif
firmware/objects.mk +7 -0
index b2ace75..c1f632e 100644
@@ -13,6 +13,13 @@ firmware-cflags-y +=
 firmware-asflags-y +=
 firmware-ldflags-y +=
 
+ifeq ($(FW_PIC),y)
+firmware-genflags-y += -DFW_PIC
+firmware-asflags-y  += -fpic
+firmware-cflags-y   += -fPIE -pie
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+endif
+
 ifdef FW_TEXT_START
 firmware-genflags-y += -DFW_TEXT_START=$(FW_TEXT_START)
 endif
include/sbi/riscv_elf.h +0 -0
Quoted 9 lines

include/sbi/riscv_elf.h

Quoted 5 lines

new file mode 100644
index 0000000..cf211ac

include/sbi/riscv_elf.h new file +0 -0
@@ -0,0 +1,9 @@

Please "#ifndef" and "#define" pair here to protect multiple header inclusions

OK, thank you for your reminder.

Quoted 23 lines

include/sbi/riscv_elf.h line 9

Quoted 9 lines
include/sbi/riscv_elf.h new file +9 -0
+#include <sbi/riscv_asm.h>
+
+#define R_RISCV_32             1
+#define R_RISCV_64             2
+#define R_RISCV_RELATIVE       3
+
+#define RELOC_TYPE             __REG_SEL(R_RISCV_64, R_RISCV_32)
+#define SYM_INDEX              __REG_SEL(0x20, 0x8)
+#define SYM_SIZE               __REG_SEL(0x18,0x10)

Add "#endif" here.

Quoted 8 lines

--
2.7.4

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup

#6 lore raw

PATCH v2 2/2 firmware: Support position independent execution

Quoted 65 lines · Vincent Chen
Quoted 63 lines · Anup Patel
Quoted 51 lines · Vincent Chen

Enable OpenSBI to support position independent execution. Because the position independent code will cause an additional GOT reference when accessing the global variables, it will reduce performance a bit. Therefore, the position independent execution is disabled by default. Users can through specifying "FW_PIC=y" on the make command to enable this feature.

In theory, after enabling position-independent execution, the OpenSBI can run at arbitrary address with appropriate alignment. Therefore, the original relocation mechanism will be skipped. In other words, OpenSBI will directly run at the load address without any code movement.

---
 Makefile                |  2 +-
 firmware/fw_base.S      | 59 ++++++++++++++++++++++++++++++++++++++++++++++++-
 firmware/fw_base.ldS    | 13 +++++++++++
 firmware/fw_jump.S      | 10 +++++++++
 firmware/fw_payload.S   |  6 +++++
 firmware/objects.mk     |  7 ++++++
 include/sbi/riscv_elf.h |  9 ++++++++
 7 files changed, 104 insertions(+), 2 deletions(-)
 create mode 100644 include/sbi/riscv_elf.h
Makefile +1 -1
index d6f097d..038cc99 100644
@@ -210,8 +210,8 @@ CFLAGS              +=      -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 CFLAGS         +=      -mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
 CFLAGS         +=      $(GENFLAGS)
 CFLAGS         +=      $(platform-cflags-y)
-CFLAGS         +=      $(firmware-cflags-y)
 CFLAGS         +=      -fno-pie -no-pie
+CFLAGS         +=      $(firmware-cflags-y)
 
 CPPFLAGS       +=      $(GENFLAGS)
 CPPFLAGS       +=      $(platform-cppflags-y)
firmware/fw_base.S +1 -0
index 6cc5f88..5f49c88 100644
@@ -9,6 +9,7 @@
 
 #include <sbi/riscv_asm.h>
 #include <sbi/riscv_encoding.h>
+#include <sbi/riscv_elf.h>
 #include <sbi/sbi_platform.h>
 #include <sbi/sbi_scratch.h>
 #include <sbi/sbi_trap.h>

firmware/fw_base.S _try_lottery: · line 74

Quoted 7 lines
firmware/fw_base.S +4 -0
@@ -67,6 +68,57 @@ _try_lottery:
        lla     t1, _start
        REG_S   t1, 0(t0)
 
+#ifdef FW_PIC
+       /* relocate the global table content */
+       lla     t0, _link_start
+       REG_L   t0, 0(t0)

The next instruction assumes that "t1" has address of _start.

Quoted 6 lines
Quoted 3 lines

I would suggest to have explicit "lla t1, _start" here OR we have to document this assumption here.

"lla t1, _start" was executed before the 4 instructions of "sub t2, t1, t0", so I think it may be OK to assume that "t1" has the address of _start.

At least add a single line comment here about the assumption on "t1" register so that in future if there is code added in-between then "t1" register should not be clobbered.

Quoted 125 lines
Quoted 77 lines
Quoted 77 lines
firmware/fw_base.S +50 -0
+       sub     t2, t1, t0
+       lla     t3, _runtime_offset
+       REG_S   t2, (t3)
+       lla     t0, __rel_dyn_start
+       lla     t1, __rel_dyn_end
+       beq     t0, t1, _relocate_done
+       j       5f
+2:
+       REG_L   t5, -(REGBYTES*2)(t0)   /* t5 <-- relocation info:type */
+       li      t3, R_RISCV_RELATIVE    /* reloc type R_RISCV_RELATIVE */
+       bne     t5, t3, 3f
+       REG_L   t3, -(REGBYTES*3)(t0)
+       REG_L   t5, -(REGBYTES)(t0)     /* t5 <-- addend */
+       add     t5, t5, t2
+       add     t3, t3, t2
+       REG_S   t5, 0(t3)               /* store runtime address to the GOT entry */
+       j       5f
+
+3:
+       lla     t4, __dyn_sym_start
+
+4:
+       REG_L   t5, -(REGBYTES*2)(t0)   /* t5 <-- relocation info:type */
+       srli    t6, t5, SYM_INDEX       /* t6 <--- sym table index */
+       andi    t5, t5, 0xFF            /* t5 <--- relocation type */
+       li      t3, RELOC_TYPE
+       bne     t5, t3, 5f
+
+       /* address R_RISCV_64 or R_RISCV_32 cases*/
+       REG_L   t3, -(REGBYTES*3)(t0)
+       li      t5, SYM_SIZE
+       mul     t6, t6, t5
+       add     s5, t4, t6
+       REG_L   t6, -(REGBYTES)(t0)     /* t0 <-- addend */
+       REG_L   t5, REGBYTES(s5)
+       add     t5, t5, t6
+       add     t5, t5, t2              /* t5 <-- location to fix up in RAM */
+       add     t3, t3, t2              /* t3 <-- location to fix up in RAM */
+       REG_S   t5, 0(t3)               /* store runtime address to the variable */
+
+5:
+       addi    t0, t0, (REGBYTES*3)
+       ble     t0, t1, 2b
+       j       _relocate_done
+_wait_relocate_copy_done:
+       j       _wait_for_boot_hart
+#else
        /* Relocate if load address != link address */
 _relocate:
        lla     t0, _link_start
@@ -137,6 +189,7 @@ _wait_relocate_copy_done:
        nop
        bgt     t4, t5, 1b
        jr      t3
+#endif
 _relocate_done:
 
        /*
@@ -144,12 +197,14 @@ _relocate_done:
         * Use _boot_status copy relative to the load address
         */
        lla     t0, _boot_status
+#ifndef FW_PIC
        lla     t1, _link_start
        REG_L   t1, 0(t1)
        lla     t2, _load_start
        REG_L   t2, 0(t2)
        sub     t0, t0, t1
        add     t0, t0, t2
+#endif
        li      t1, BOOT_STATUS_RELOCATE_DONE
        REG_S   t1, 0(t0)
        fence   rw, rw
@@ -446,6 +501,8 @@ _skip_trap_exit_rv32_hyp:
        j       _start_hang
 
Quoted 6 lines

firmware/fw_base.S _skip_trap_exit_rv32_hyp: · line 505

Quoted 3 lines
firmware/fw_base.S +2 -0
        .align 3
+_runtime_offset:
+       RISCV_PTR       0

Please add "#ifdef" around _runtime_offset

OK, thanks for the reminder.

Quoted 12 lines

firmware/fw_base.S _boot_status: · line 513

Quoted 9 lines
firmware/fw_base.S +1 -1
 _relocate_lottery:
        RISCV_PTR       0
 _boot_status:
@@ -453,7 +510,7 @@ _boot_status:
 _load_start:
        RISCV_PTR       _fw_start
 _link_start:
-       RISCV_PTR       _fw_start
+       RISCV_PTR       FW_TEXT_START

Why do we need this changes ?

The _fw_start is a global variable. When we use -fPIE to compile the code, the linker does not know the exact address at runtime. Therefore, the linker will fill 0 here and create a relocation entry at the ".rela.dyn" section to let the dynamic loader resolve it at runtime.

_load_start:
RISCV_PTR _fw_start
_link_start:
RISCV_PTR _fw_start

1. objdump -D fw_payload.elf
0000000080000400 <_link_start>:

        ...

0000000080000408 <_link_end>:

        ...

0000000080000410 <_hartid_to_scratch>:

2. readelf -a fw_payload.elf
Relocation section '.rela.dyn' at offset 0x12f38 contains 175 entries:
Offset Info Type Sym. Value Sym. Name + Addend

0000800003f8  000000000003 R_RISCV_RELATIVE                     80000000
000080000400  000000000003 R_RISCV_RELATIVE                     80000000
000080000408  000000000003 R_RISCV_RELATIVE                     81ae3e08
Quoted 6 lines

However, from the code, I deduce that the variable _link_start is used to store the starting address of the image at compile time. Therefore, I change the _fw_start to FW_TEXT_START to ensure the value of _link_start is the exact start address of the image at the compile-time. The dynamic loader can use _link_start to calculate the offset between run time and compile-time as well.

Okay, thanks for the explanation.

Quoted 132 lines
Quoted 12 lines
Quoted 12 lines
firmware/fw_base.S +0 -0
 _link_end:
        RISCV_PTR       _fw_reloc_end
 
firmware/fw_base.ldS +0 -0
index 0ac75f2..3904959 100644
@@ -61,6 +61,19 @@
                PROVIDE(_data_end = .);
        }
 
Quoted 7 lines

firmware/fw_base.ldS line 67

Quoted 4 lines
firmware/fw_base.ldS +4 -0
+       .dynsym : {
+               PROVIDE(__dyn_sym_start = .);
+               *(.dynsym)
+               PROVIDE(__dyn_sym_end = .);

Only use tabs for indentation here.

OK

Quoted 10 lines

firmware/fw_base.ldS line 74

Quoted 7 lines
firmware/fw_base.ldS +7 -0
+       }
+
+       .rela.dyn : {
+               PROVIDE(__rel_dyn_start = .);
+               *(.rela*)
+               . = ALIGN(8);
+               PROVIDE(__rel_dyn_end = .);

Only use tabs for indentation here.

OK

Quoted 36 lines
Quoted 22 lines
firmware/fw_base.ldS +2 -0
+       }
+
        . = ALIGN(0x1000); /* Ensure next section is page aligned */
 
        .bss :
firmware/fw_jump.S +5 -0
index 5b24f8b..4fb3e0c 100644
@@ -46,6 +46,11 @@ fw_save_info:
 fw_next_arg1:
 #ifdef FW_JUMP_FDT_ADDR
        li      a0, FW_JUMP_FDT_ADDR
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
 #else
        add     a0, a1, zero
 #endif

firmware/fw_jump.S fw_next_arg1: · line 74

Quoted 9 lines
firmware/fw_jump.S +5 -0
@@ -61,6 +66,11 @@ fw_next_arg1:
 fw_next_addr:
        lla     a0, _jump_addr
        REG_L   a0, (a0)
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
        ret

The changes in fw_next_addr() breaks the FW_JUMP when FW_PIC=y because FW_JUMP assumes a compile-time fixed address of next booting stage specified by FW_JUMP_ADDR.

Quoted 6 lines

Similarly, when FW_JUMP_FDT_ADDR is defined FW_JUMP will assume compile-time fixed address of FDT passed to next booting stage.

Please drop all changes from fw_jump.S

OK, I will drop it.

Quoted 19 lines
Quoted 7 lines
firmware/fw_jump.S +0 -0
        .section .entry, "ax", %progbits
firmware/fw_payload.S +0 -0
index c53a3bb..00c7cb7 100644

firmware/fw_payload.S fw_save_info: · line 54

Quoted 9 lines
firmware/fw_payload.S +6 -0
@@ -46,6 +46,12 @@ fw_save_info:
 fw_next_arg1:
 #ifdef FW_PAYLOAD_FDT_ADDR
        li      a0, FW_PAYLOAD_FDT_ADDR
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
+

Same as above.

Quoted 7 lines

The changes in fw_next_arg1() breaks FW_PAYLOAD when FW_PIC=y because FW_PAYLOAD assumes a compile-time fixed address of FDT passed to next booting-stage when FW_PAYLOAD_FDT_ADDR is defined.

Please drop all changes in fw_payload.S

OK

Quoted 22 lines
Quoted 22 lines
firmware/fw_payload.S +0 -0
 #else
        add     a0, a1, zero
 #endif
firmware/objects.mk +7 -0
index b2ace75..c1f632e 100644
@@ -13,6 +13,13 @@ firmware-cflags-y +=
 firmware-asflags-y +=
 firmware-ldflags-y +=
 
+ifeq ($(FW_PIC),y)
+firmware-genflags-y += -DFW_PIC
+firmware-asflags-y  += -fpic
+firmware-cflags-y   += -fPIE -pie
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+endif
+
 ifdef FW_TEXT_START
 firmware-genflags-y += -DFW_TEXT_START=$(FW_TEXT_START)
 endif
include/sbi/riscv_elf.h +0 -0
Quoted 9 lines

include/sbi/riscv_elf.h

Quoted 5 lines

new file mode 100644
index 0000000..cf211ac

include/sbi/riscv_elf.h new file +0 -0
@@ -0,0 +1,9 @@

Please "#ifndef" and "#define" pair here to protect multiple header inclusions

Quoted 24 lines

OK, thank you for your reminder.

Quoted 23 lines

include/sbi/riscv_elf.h line 9

Quoted 9 lines
include/sbi/riscv_elf.h new file +9 -0
+#include <sbi/riscv_asm.h>
+
+#define R_RISCV_32             1
+#define R_RISCV_64             2
+#define R_RISCV_RELATIVE       3
+
+#define RELOC_TYPE             __REG_SEL(R_RISCV_64, R_RISCV_32)
+#define SYM_INDEX              __REG_SEL(0x20, 0x8)
+#define SYM_SIZE               __REG_SEL(0x18,0x10)

Add "#endif" here.

Quoted 8 lines

--
2.7.4

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup

Regards,
Anup

#7 lore raw

PATCH v2 2/2 firmware: Support position independent execution

Quoted 70 lines · Anup Patel
Quoted 68 lines · Vincent Chen
Quoted 66 lines · Anup Patel
Quoted 51 lines · Vincent Chen

Enable OpenSBI to support position independent execution. Because the position independent code will cause an additional GOT reference when accessing the global variables, it will reduce performance a bit. Therefore, the position independent execution is disabled by default. Users can through specifying "FW_PIC=y" on the make command to enable this feature.

In theory, after enabling position-independent execution, the OpenSBI can run at arbitrary address with appropriate alignment. Therefore, the original relocation mechanism will be skipped. In other words, OpenSBI will directly run at the load address without any code movement.

---
 Makefile                |  2 +-
 firmware/fw_base.S      | 59 ++++++++++++++++++++++++++++++++++++++++++++++++-
 firmware/fw_base.ldS    | 13 +++++++++++
 firmware/fw_jump.S      | 10 +++++++++
 firmware/fw_payload.S   |  6 +++++
 firmware/objects.mk     |  7 ++++++
 include/sbi/riscv_elf.h |  9 ++++++++
 7 files changed, 104 insertions(+), 2 deletions(-)
 create mode 100644 include/sbi/riscv_elf.h
Makefile +1 -1
index d6f097d..038cc99 100644
@@ -210,8 +210,8 @@ CFLAGS              +=      -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA)
 CFLAGS         +=      -mcmodel=$(PLATFORM_RISCV_CODE_MODEL)
 CFLAGS         +=      $(GENFLAGS)
 CFLAGS         +=      $(platform-cflags-y)
-CFLAGS         +=      $(firmware-cflags-y)
 CFLAGS         +=      -fno-pie -no-pie
+CFLAGS         +=      $(firmware-cflags-y)
 
 CPPFLAGS       +=      $(GENFLAGS)
 CPPFLAGS       +=      $(platform-cppflags-y)
firmware/fw_base.S +1 -0
index 6cc5f88..5f49c88 100644
@@ -9,6 +9,7 @@
 
 #include <sbi/riscv_asm.h>
 #include <sbi/riscv_encoding.h>
+#include <sbi/riscv_elf.h>
 #include <sbi/sbi_platform.h>
 #include <sbi/sbi_scratch.h>
 #include <sbi/sbi_trap.h>

firmware/fw_base.S _try_lottery: · line 74

Quoted 7 lines
firmware/fw_base.S +4 -0
@@ -67,6 +68,57 @@ _try_lottery:
        lla     t1, _start
        REG_S   t1, 0(t0)
 
+#ifdef FW_PIC
+       /* relocate the global table content */
+       lla     t0, _link_start
+       REG_L   t0, 0(t0)

The next instruction assumes that "t1" has address of _start.

I would suggest to have explicit "lla t1, _start" here OR we have to document this assumption here.

Quoted 8 lines
Quoted 3 lines

"lla t1, _start" was executed before the 4 instructions of "sub t2, t1, t0", so I think it may be OK to assume that "t1" has the address of _start.

At least add a single line comment here about the assumption on "t1" register so that in future if there is code added in-between then "t1" register should not be clobbered.

I got it. Thanks for your explanation. I will add a comment here in my next version patch.

Quoted 125 lines
Quoted 125 lines
Quoted 77 lines
Quoted 77 lines
firmware/fw_base.S +50 -0
+       sub     t2, t1, t0
+       lla     t3, _runtime_offset
+       REG_S   t2, (t3)
+       lla     t0, __rel_dyn_start
+       lla     t1, __rel_dyn_end
+       beq     t0, t1, _relocate_done
+       j       5f
+2:
+       REG_L   t5, -(REGBYTES*2)(t0)   /* t5 <-- relocation info:type */
+       li      t3, R_RISCV_RELATIVE    /* reloc type R_RISCV_RELATIVE */
+       bne     t5, t3, 3f
+       REG_L   t3, -(REGBYTES*3)(t0)
+       REG_L   t5, -(REGBYTES)(t0)     /* t5 <-- addend */
+       add     t5, t5, t2
+       add     t3, t3, t2
+       REG_S   t5, 0(t3)               /* store runtime address to the GOT entry */
+       j       5f
+
+3:
+       lla     t4, __dyn_sym_start
+
+4:
+       REG_L   t5, -(REGBYTES*2)(t0)   /* t5 <-- relocation info:type */
+       srli    t6, t5, SYM_INDEX       /* t6 <--- sym table index */
+       andi    t5, t5, 0xFF            /* t5 <--- relocation type */
+       li      t3, RELOC_TYPE
+       bne     t5, t3, 5f
+
+       /* address R_RISCV_64 or R_RISCV_32 cases*/
+       REG_L   t3, -(REGBYTES*3)(t0)
+       li      t5, SYM_SIZE
+       mul     t6, t6, t5
+       add     s5, t4, t6
+       REG_L   t6, -(REGBYTES)(t0)     /* t0 <-- addend */
+       REG_L   t5, REGBYTES(s5)
+       add     t5, t5, t6
+       add     t5, t5, t2              /* t5 <-- location to fix up in RAM */
+       add     t3, t3, t2              /* t3 <-- location to fix up in RAM */
+       REG_S   t5, 0(t3)               /* store runtime address to the variable */
+
+5:
+       addi    t0, t0, (REGBYTES*3)
+       ble     t0, t1, 2b
+       j       _relocate_done
+_wait_relocate_copy_done:
+       j       _wait_for_boot_hart
+#else
        /* Relocate if load address != link address */
 _relocate:
        lla     t0, _link_start
@@ -137,6 +189,7 @@ _wait_relocate_copy_done:
        nop
        bgt     t4, t5, 1b
        jr      t3
+#endif
 _relocate_done:
 
        /*
@@ -144,12 +197,14 @@ _relocate_done:
         * Use _boot_status copy relative to the load address
         */
        lla     t0, _boot_status
+#ifndef FW_PIC
        lla     t1, _link_start
        REG_L   t1, 0(t1)
        lla     t2, _load_start
        REG_L   t2, 0(t2)
        sub     t0, t0, t1
        add     t0, t0, t2
+#endif
        li      t1, BOOT_STATUS_RELOCATE_DONE
        REG_S   t1, 0(t0)
        fence   rw, rw
@@ -446,6 +501,8 @@ _skip_trap_exit_rv32_hyp:
        j       _start_hang
 
Quoted 6 lines

firmware/fw_base.S _skip_trap_exit_rv32_hyp: · line 505

Quoted 3 lines
firmware/fw_base.S +2 -0
        .align 3
+_runtime_offset:
+       RISCV_PTR       0

Please add "#ifdef" around _runtime_offset

OK, thanks for the reminder.

Quoted 12 lines

firmware/fw_base.S _boot_status: · line 513

Quoted 9 lines
firmware/fw_base.S +1 -1
 _relocate_lottery:
        RISCV_PTR       0
 _boot_status:
@@ -453,7 +510,7 @@ _boot_status:
 _load_start:
        RISCV_PTR       _fw_start
 _link_start:
-       RISCV_PTR       _fw_start
+       RISCV_PTR       FW_TEXT_START

Why do we need this changes ?

The _fw_start is a global variable. When we use -fPIE to compile the code, the linker does not know the exact address at runtime. Therefore, the linker will fill 0 here and create a relocation entry at the ".rela.dyn" section to let the dynamic loader resolve it at runtime.

_load_start:
RISCV_PTR _fw_start
_link_start:
RISCV_PTR _fw_start

1. objdump -D fw_payload.elf
0000000080000400 <_link_start>:

        ...

0000000080000408 <_link_end>:

        ...

0000000080000410 <_hartid_to_scratch>:

2. readelf -a fw_payload.elf
Relocation section '.rela.dyn' at offset 0x12f38 contains 175 entries:
Offset Info Type Sym. Value Sym. Name + Addend

0000800003f8  000000000003 R_RISCV_RELATIVE                     80000000
000080000400  000000000003 R_RISCV_RELATIVE                     80000000
000080000408  000000000003 R_RISCV_RELATIVE                     81ae3e08
Quoted 83 lines
Quoted 6 lines

However, from the code, I deduce that the variable _link_start is used to store the starting address of the image at compile time. Therefore, I change the _fw_start to FW_TEXT_START to ensure the value of _link_start is the exact start address of the image at the compile-time. The dynamic loader can use _link_start to calculate the offset between run time and compile-time as well.

Okay, thanks for the explanation.

Quoted 74 lines
Quoted 12 lines
Quoted 12 lines
firmware/fw_base.S +0 -0
 _link_end:
        RISCV_PTR       _fw_reloc_end
 
firmware/fw_base.ldS +0 -0
index 0ac75f2..3904959 100644
@@ -61,6 +61,19 @@
                PROVIDE(_data_end = .);
        }
 
Quoted 7 lines

firmware/fw_base.ldS line 67

Quoted 4 lines
firmware/fw_base.ldS +4 -0
+       .dynsym : {
+               PROVIDE(__dyn_sym_start = .);
+               *(.dynsym)
+               PROVIDE(__dyn_sym_end = .);

Only use tabs for indentation here.

OK

Quoted 10 lines

firmware/fw_base.ldS line 74

Quoted 7 lines
firmware/fw_base.ldS +7 -0
+       }
+
+       .rela.dyn : {
+               PROVIDE(__rel_dyn_start = .);
+               *(.rela*)
+               . = ALIGN(8);
+               PROVIDE(__rel_dyn_end = .);

Only use tabs for indentation here.

OK

Quoted 36 lines
Quoted 22 lines
firmware/fw_base.ldS +2 -0
+       }
+
        . = ALIGN(0x1000); /* Ensure next section is page aligned */
 
        .bss :
firmware/fw_jump.S +5 -0
index 5b24f8b..4fb3e0c 100644
@@ -46,6 +46,11 @@ fw_save_info:
 fw_next_arg1:
 #ifdef FW_JUMP_FDT_ADDR
        li      a0, FW_JUMP_FDT_ADDR
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
 #else
        add     a0, a1, zero
 #endif

firmware/fw_jump.S fw_next_arg1: · line 74

Quoted 9 lines
firmware/fw_jump.S +5 -0
@@ -61,6 +66,11 @@ fw_next_arg1:
 fw_next_addr:
        lla     a0, _jump_addr
        REG_L   a0, (a0)
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
        ret

The changes in fw_next_addr() breaks the FW_JUMP when FW_PIC=y because FW_JUMP assumes a compile-time fixed address of next booting stage specified by FW_JUMP_ADDR.

Quoted 6 lines

Similarly, when FW_JUMP_FDT_ADDR is defined FW_JUMP will assume compile-time fixed address of FDT passed to next booting stage.

Please drop all changes from fw_jump.S

OK, I will drop it.

Sorry, I forgot to ask a question. Does the assumption of FW_JUMP also imply that FW_JUMP cannot support PIE mode, right?

Quoted 85 lines
Quoted 58 lines
Quoted 19 lines
Quoted 7 lines
firmware/fw_jump.S +0 -0
        .section .entry, "ax", %progbits
firmware/fw_payload.S +0 -0
index c53a3bb..00c7cb7 100644

firmware/fw_payload.S fw_save_info: · line 54

Quoted 9 lines
firmware/fw_payload.S +6 -0
@@ -46,6 +46,12 @@ fw_save_info:
 fw_next_arg1:
 #ifdef FW_PAYLOAD_FDT_ADDR
        li      a0, FW_PAYLOAD_FDT_ADDR
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
+

Same as above.

Quoted 7 lines

The changes in fw_next_arg1() breaks FW_PAYLOAD when FW_PIC=y because FW_PAYLOAD assumes a compile-time fixed address of FDT passed to next booting-stage when FW_PAYLOAD_FDT_ADDR is defined.

Please drop all changes in fw_payload.S

OK

Quoted 22 lines
Quoted 22 lines
firmware/fw_payload.S +0 -0
 #else
        add     a0, a1, zero
 #endif
firmware/objects.mk +7 -0
index b2ace75..c1f632e 100644
@@ -13,6 +13,13 @@ firmware-cflags-y +=
 firmware-asflags-y +=
 firmware-ldflags-y +=
 
+ifeq ($(FW_PIC),y)
+firmware-genflags-y += -DFW_PIC
+firmware-asflags-y  += -fpic
+firmware-cflags-y   += -fPIE -pie
+firmware-ldflags-y  +=  -Wl,--no-dynamic-linker
+endif
+
 ifdef FW_TEXT_START
 firmware-genflags-y += -DFW_TEXT_START=$(FW_TEXT_START)
 endif
include/sbi/riscv_elf.h +0 -0
Quoted 9 lines

include/sbi/riscv_elf.h

Quoted 5 lines

new file mode 100644
index 0000000..cf211ac

include/sbi/riscv_elf.h new file +0 -0
@@ -0,0 +1,9 @@

Please "#ifndef" and "#define" pair here to protect multiple header inclusions

Quoted 24 lines

OK, thank you for your reminder.

Quoted 23 lines

include/sbi/riscv_elf.h line 9

Quoted 9 lines
include/sbi/riscv_elf.h new file +9 -0
+#include <sbi/riscv_asm.h>
+
+#define R_RISCV_32             1
+#define R_RISCV_64             2
+#define R_RISCV_RELATIVE       3
+
+#define RELOC_TYPE             __REG_SEL(R_RISCV_64, R_RISCV_32)
+#define SYM_INDEX              __REG_SEL(0x20, 0x8)
+#define SYM_SIZE               __REG_SEL(0x18,0x10)

Add "#endif" here.

Quoted 8 lines

--
2.7.4

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup

Regards,
Anup

#8 lore raw

PATCH v2 2/2 firmware: Support position independent execution

Quoted 43 lines · Vincent Chen
Quoted 41 lines · Anup Patel
Quoted 39 lines · Vincent Chen
Quoted 37 lines · Anup Patel
Quoted 20 lines · Vincent Chen
firmware/fw_jump.S +0 -0
index 5b24f8b..4fb3e0c 100644
@@ -46,6 +46,11 @@ fw_save_info:

fw_next_arg1:
#ifdef FW_JUMP_FDT_ADDR
li a0, FW_JUMP_FDT_ADDR

firmware/fw_jump.S +5 -0
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif

#else
add a0, a1, zero
#endif

firmware/fw_jump.S +0 -0
@@ -61,6 +66,11 @@ fw_next_arg1:

fw_next_addr:
lla a0, _jump_addr
REG_L a0, (a0)

firmware/fw_jump.S fw_next_arg1: · line 74

Quoted 6 lines
firmware/fw_jump.S +5 -0
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
       ret

The changes in fw_next_addr() breaks the FW_JUMP when FW_PIC=y because FW_JUMP assumes a compile-time fixed address of next booting stage specified by FW_JUMP_ADDR.

Similarly, when FW_JUMP_FDT_ADDR is defined FW_JUMP will assume compile-time fixed address of FDT passed to next booting stage.

Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines

Please drop all changes from fw_jump.S

OK, I will drop it.

Sorry, I forgot to ask a question. Does the assumption of FW_JUMP also imply that FW_JUMP cannot support PIE mode, right?

No, you can have a PIE FW_JUMP OpenSBI. The problem is that you added relocbase to FW_JUMP_ADDR and FW_JUMP_FDT_ADDR when those are platform-defined constants regardless of where OpenSBI has been loaded, and so should remain unmodified.

Jess

#9 lore raw

PATCH v2 2/2 firmware: Support position independent execution

Quoted 49 lines · Jessica Clarke
Quoted 47 lines · Vincent Chen
Quoted 44 lines · Anup Patel
Quoted 42 lines · Vincent Chen
Quoted 33 lines · Anup Patel
Quoted 20 lines · Vincent Chen
firmware/fw_jump.S +0 -0
index 5b24f8b..4fb3e0c 100644
@@ -46,6 +46,11 @@ fw_save_info:

fw_next_arg1:
#ifdef FW_JUMP_FDT_ADDR
li a0, FW_JUMP_FDT_ADDR

firmware/fw_jump.S +5 -0
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif

#else
add a0, a1, zero
#endif

firmware/fw_jump.S +0 -0
@@ -61,6 +66,11 @@ fw_next_arg1:

fw_next_addr:
lla a0, _jump_addr
REG_L a0, (a0)

firmware/fw_jump.S fw_next_arg1: · line 74

Quoted 6 lines
firmware/fw_jump.S +5 -0
+#ifdef FW_PIC
+       lla     a1, _runtime_offset
+       REG_L   a1, (a1)
+       add     a0, a0, a1
+#endif
       ret

The changes in fw_next_addr() breaks the FW_JUMP when FW_PIC=y because FW_JUMP assumes a compile-time fixed address of next booting stage specified by FW_JUMP_ADDR.

Quoted 6 lines

Similarly, when FW_JUMP_FDT_ADDR is defined FW_JUMP will assume compile-time fixed address of FDT passed to next booting stage.

Please drop all changes from fw_jump.S

OK, I will drop it.

Quoted 9 lines
Quoted 2 lines

Sorry, I forgot to ask a question. Does the assumption of FW_JUMP also imply that FW_JUMP cannot support PIE mode, right?

No, you can have a PIE FW_JUMP OpenSBI. The problem is that you added relocbase to FW_JUMP_ADDR and FW_JUMP_FDT_ADDR when those are platform-defined constants regardless of where OpenSBI has been loaded, and so should remain unmodified.

Jess

I got it. Thank you for your explanation.