ARM: simplify handover from UEFI to decompressor
Subsystems (2)
Ard Biesheuvel
maintainerv2 cover ARM: simplify handover from UEFI to decompressor
The EFI stub in the ARM kernel runs in the context of the firmware, which means it runs with the caches and MMU on. Currently, we relocate the zImage so it appears in the first 128 MiB, disable the MMU and caches and invoke the decompressor via its ordinary entry point. However, since we can pass the base of DRAM directly, there is no need to relocate the zImage, which also means there is no need to disable and re-enable the caches and create new page tables etc.
This simplification is implemented by patch #5. Patches #1 - #4 are prerequisite changes to permit the decompressor to execute from the offset chosen by the UEFI firmware.
Note that this applies onto v5.7-rc1 with Geert's patch 'ARM: boot: Obtain start of physical memory from DTB' applied [0]
Changes since v1:
- tweak some asm sequences in #2 to fix the Thumb2 build
- switch immediately to the new stack in #5
[0] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Ard Biesheuvel (5):
ARM: decompressor: move headroom variable out of LC0
ARM: decompressor: split off _edata and stack base into separate
object
ARM: decompressor: defer loading of the contents of the LC0 structure
ARM: decompressor: move GOT into .data for EFI enabled builds
ARM: decompressor: run decompressor in place if loaded via UEFI
arch/arm/boot/compressed/head.S | 89 ++++++++------------ arch/arm/boot/compressed/vmlinux.lds.S | 5 ++ drivers/firmware/efi/libstub/arm32-stub.c | 45 ++-------- 3 files changed, 45 insertions(+), 94 deletions(-)
Signature · 1 line
2.17.1
Ard Biesheuvel
maintainerPATCH v2 1/5 ARM: decompressor: move headroom variable out of LC0
Before breaking up LC0 into different pieces, move out the variable that is already place-relative (given that it subtracts 'restart' in the expression) and so its value does not need to be added to the runtime address of the LC0 symbol itself.
--- arch/arm/boot/compressed/head.S | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
arch/arm/boot/compressed/head.S +4 -2
@@ -339,7 +339,7 @@ not_angel: */ mov r0, pc cmp r0, r4 - ldrcc r0, LC0+28 + ldrcc r0, .Lheadroom addcc r0, r0, pc cmpcc r4, r0 orrcc r4, r4, #1 @ remember we skipped cache_on
@@ -716,9 +716,11 @@ LC0: .word LC0 @ r1 .word _got_start @ r11 .word _got_end @ ip .word .L_user_stack_end @ sp - .word _end - restart + 16384 + 1024*1024 .size LC0, . - LC0 +.Lheadroom: + .word _end - restart + 16384 + 1024*1024 + .Linflated_image_size_offset: .long (input_data_end - 4) - .
Signature · 1 line
2.17.1
Ard Biesheuvel
maintainerPATCH v2 2/5 ARM: decompressor: split off _edata and stack base into separate object
In reply to Ard BiesheuvelIn preparation of moving the handling of the LC0 object to a later stage in the decompressor startup code, move out _edata and the initial value of the stack pointer, which are now used in two places, and are needed earlier than the remaining contents of LC0.
--- arch/arm/boot/compressed/head.S | 35 ++++++++++---------- 1 file changed, 17 insertions(+), 18 deletions(-)
arch/arm/boot/compressed/head.S +17 -18
@@ -259,19 +259,17 @@ not_angel: * Find the start of physical memory. * Try the DTB first, if available. */ - adr r0, LC0 - ldr r1, [r0] @ get absolute LC0 - ldr sp, [r0, #24] @ get stack location - sub r1, r0, r1 @ compute relocation offset - add sp, sp, r1 @ apply relocation + adr r0, LC1 + ldr sp, [r0] @ get stack location + add sp, sp, r0 @ apply relocation #ifdef CONFIG_ARM_APPENDED_DTB /* * Look for an appended DTB. If found, use it and * move stack away from it. */ - ldr r6, [r0, #12] @ get &_edata - add r6, r6, r1 @ relocate it + ldr r6, [r0, #4] @ get &_edata + add r6, r6, r0 @ relocate it ldmia r6, {r0, r5} @ get DTB signature and size #ifndef __ARMEB__ ldr r1, =0xedfe0dd0 @ sig is 0xd00dfeed big endian
@@ -345,22 +343,20 @@ not_angel: orrcc r4, r4, #1 @ remember we skipped cache_on blcs cache_on -restart: adr r0, LC0 - ldmia r0, {r1, r2, r3, r6, r11, r12} - ldr sp, [r0, #24] +restart: adr r0, LC1 + ldr sp, [r0] + ldr r6, [r0, #4] + add sp, sp, r0 + add r6, r6, r0 - /* - * We might be running at a different address. We need - * to fix up various pointers. - */ + adr r0, LC0 + ldmia r0, {r1, r2, r3, r11, r12} sub r0, r0, r1 @ calculate the delta offset - add r6, r6, r0 @ _edata get_inflated_image_size r9, r10, lr #ifndef CONFIG_ZBOOT_ROM /* malloc space is above the relocated stack (64k max) */ - add sp, sp, r0 add r10, sp, #0x10000 #else /*
@@ -712,12 +708,15 @@ not_relocated: mov r0, #0 LC0: .word LC0 @ r1 .word __bss_start @ r2 .word _end @ r3 - .word _edata @ r6 .word _got_start @ r11 .word _got_end @ ip - .word .L_user_stack_end @ sp .size LC0, . - LC0 + .type LC1, #object +LC1: .word .L_user_stack_end - LC1 @ sp + .word _edata - LC1 @ r6 + .size LC1, . - LC1 + .Lheadroom: .word _end - restart + 16384 + 1024*1024
Signature · 1 line
2.17.1
Ard Biesheuvel
maintainerPATCH v2 3/5 ARM: decompressor: defer loading of the contents of the LC0 structure
In reply to Ard BiesheuvelThe remaining contents of LC0 are only used after the point in the decompressor startup code where we enter via 'wont_overwrite'. So move the loading of the LC0 structure after it. This will allow us to jump to wont_overwrite directly from the EFI stub, and execute the decompressor in place at the offset it was loaded by the UEFI firmware.
--- arch/arm/boot/compressed/head.S | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)
arch/arm/boot/compressed/head.S +4 -11
@@ -349,10 +349,6 @@ restart: adr r0, LC1 add sp, sp, r0 add r6, r6, r0 - adr r0, LC0 - ldmia r0, {r1, r2, r3, r11, r12} - sub r0, r0, r1 @ calculate the delta offset - get_inflated_image_size r9, r10, lr #ifndef CONFIG_ZBOOT_ROM
@@ -370,9 +366,6 @@ restart: adr r0, LC1 mov r5, #0 @ init dtb size to 0 #ifdef CONFIG_ARM_APPENDED_DTB /* - * r0 = delta - * r2 = BSS start - * r3 = BSS end * r4 = final kernel address (possibly with LSB set) * r5 = appended dtb size (still unknown) * r6 = _edata
@@ -380,8 +373,6 @@ restart: adr r0, LC1 * r8 = atags/device tree pointer * r9 = size of decompressed image * r10 = end of this image, including bss/stack/malloc space if non XIP - * r11 = GOT start - * r12 = GOT end * sp = stack pointer * * if there are device trees (dtb) appended to zImage, advance r10 so that the
@@ -429,7 +420,6 @@ restart: adr r0, LC1 /* temporarily relocate the stack past the DTB work space */ add sp, sp, r5 - stmfd sp!, {r0-r3, ip, lr} mov r0, r8 mov r1, r6 mov r2, r5
@@ -448,7 +438,6 @@ restart: adr r0, LC1 mov r2, r5 bleq atags_to_fdt - ldmfd sp!, {r0-r3, ip, lr} sub sp, sp, r5 #endif
@@ -585,6 +574,10 @@ dtb_check_done: mov pc, r0 wont_overwrite: + adr r0, LC0 + ldmia r0, {r1, r2, r3, r11, r12} + sub r0, r0, r1 @ calculate the delta offset + /* * If delta is zero, we are running at the address we were linked at. * r0 = delta
Signature · 1 line
2.17.1
Ard Biesheuvel
maintainerPATCH v2 4/5 ARM: decompressor: move GOT into .data for EFI enabled builds
In reply to Ard BiesheuvelWe will be running the decompressor in place after a future patch, instead of copying it around first. This means we no longer have to disable and re-enable the MMU and caches either. However, this means we will be loaded with the restricted permissions set by the UEFI firmware, which means that we have to move the GOT table into the data section in order for the contents to be writable by the code itself.
--- arch/arm/boot/compressed/vmlinux.lds.S | 5 +++++ 1 file changed, 5 insertions(+)
arch/arm/boot/compressed/vmlinux.lds.S +5 -0
@@ -63,9 +63,11 @@ SECTIONS _etext = .; .got.plt : { *(.got.plt) } +#ifndef CONFIG_EFI_STUB _got_start = .; .got : { *(.got) } _got_end = .; +#endif /* ensure the zImage file size is always a multiple of 64 bits */ /* (without a dummy byte, ld just ignores the empty section) */
@@ -74,6 +76,9 @@ SECTIONS #ifdef CONFIG_EFI_STUB .data : ALIGN(4096) { __pecoff_data_start = .; + _got_start = .; + *(.got) + _got_end = .; /* * The EFI stub always executes from RAM, and runs strictly before the * decompressor, so we can make an exception for its r/w data, and keep it
Signature · 1 line
2.17.1
Ard Biesheuvel
maintainerPATCH v2 5/5 ARM: decompressor: run decompressor in place if loaded via UEFI
In reply to Ard BiesheuvelThe decompressor can load from anywhere in memory, and the only reason the EFI stub code relocates it is to ensure it appears within the first 128 MiB of memory, so that the uncompressed kernel ends up at the right offset in memory.
We can short circuit this, and simply jump into the decompressor startup code at the point where it knows where the base of memory lives. This also means there is no need to disable the MMU and caches, create new page tables and re-enable them.
--- arch/arm/boot/compressed/head.S | 37 +++++----------- drivers/firmware/efi/libstub/arm32-stub.c | 45 +++----------------- 2 files changed, 17 insertions(+), 65 deletions(-)
arch/arm/boot/compressed/head.S +11 -26
@@ -1480,37 +1480,22 @@ reloc_code_end: #ifdef CONFIG_EFI_STUB ENTRY(efi_enter_kernel) - mov r7, r0 @ preserve image base - mov r4, r1 @ preserve DT pointer + mov r4, r0 @ preserve image base + mov r8, r1 @ preserve DT pointer - mov r0, r4 @ DT start - add r1, r4, r2 @ DT end + mov r0, r8 @ DT start + add r1, r8, r2 @ DT end bl cache_clean_flush - mov r0, r7 @ relocated zImage - ldr r1, =_edata @ size of zImage - add r1, r1, r0 @ end of zImage - bl cache_clean_flush - - @ The PE/COFF loader might not have cleaned the code we are - @ running beyond the PoU, and so calling cache_off below from - @ inside the PE/COFF loader allocated region is unsafe unless - @ we explicitly clean it to the PoC. - adr r0, call_cache_fn @ region of code we will - adr r1, 0f @ run with MMU off - bl cache_clean_flush - bl cache_off + adr r0, 0f @ switch to our stack + ldr sp, [r0] + add sp, sp, r0 - @ Set parameters for booting zImage according to boot protocol - @ put FDT address in r2, it was returned by efi_entry() - @ r1 is the machine type, and r0 needs to be 0 - mov r0, #0 - mov r1, #0xFFFFFFFF - mov r2, r4 - add r7, r7, #(__efi_start - start) - mov pc, r7 @ no mode switch + mov r5, #0 @ appended DTB size + mov r7, #0xFFFFFFFF @ machine ID + b wont_overwrite ENDPROC(efi_enter_kernel) -0: +0: .long .L_user_stack_end - . #endif .align
drivers/firmware/efi/libstub/arm32-stub.c +6 -39
61 lines not on this page · raw message
Signature · 1 line
2.17.1
Russell King - ARM Linux admin
maintainerQuoted 11 lines · Ard Biesheuvel
The EFI stub in the ARM kernel runs in the context of the firmware, which means it runs with the caches and MMU on. Currently, we relocate the zImage so it appears in the first 128 MiB, disable the MMU and caches and invoke the decompressor via its ordinary entry point. However, since we can pass the base of DRAM directly, there is no need to relocate the zImage, which also means there is no need to disable and re-enable the caches and create new page tables etc.
This simplification is implemented by patch #5. Patches #1 - #4 are prerequisite changes to permit the decompressor to execute from the offset chosen by the UEFI firmware.
Why? The decompressor is already fully relocatable, so this doesn't explain why all these changes breaking up the single place where data is stored into multiple smaller pieces, making the code more complex is really necessary. To me, this seems ot be change for change sake.
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTC broadband for 0.8mile line in suburbia: sync at 10.2Mbps down 587kbps up
Ard Biesheuvel
maintainerQuoted 10 lines · Russell King - ARM Linux admin
Quoted 8 lines · Ard Biesheuvel
The EFI stub in the ARM kernel runs in the context of the firmware, which means it runs with the caches and MMU on. Currently, we relocate the zImage so it appears in the first 128 MiB, disable the MMU and caches and invoke the decompressor via its ordinary entry point. However, since we can pass the base of DRAM directly, there is no need to relocate the zImage, which also means there is no need to disable and re-enable the caches and create new page tables etc.
Quoted 9 lines
Quoted 3 lines
This simplification is implemented by patch #5. Patches #1 - #4 are prerequisite changes to permit the decompressor to execute from the offset chosen by the UEFI firmware.
Why? The decompressor is already fully relocatable, so this doesn't explain why all these changes breaking up the single place where data is stored into multiple smaller pieces, making the code more complex is really necessary. To me, this seems ot be change for change sake.
Please refer to patch #3 for the explanation. The EFI stub will no longer enter the decompressor startup code at the top, but jump straight to the 'wont_overwrite' label. Most of the contents of LC0 are never used before that point, so its load can easily be deferred until afterwards. If you look carefully, you'll notice that that by itself simplifies the code, because we no longer need to stack/unstack those registers either. So I would argue that reducing the liveness scope of r0, r2, r3, r11 and r12, and deferring their initialization to the point where their values are actually needed is an improvement in itself, regardless of whether the EFI stub simplification in patch #5 depends on it or not.
Quoted 8 lines · Ard Biesheuvel
The EFI stub in the ARM kernel runs in the context of the firmware, which means it runs with the caches and MMU on. Currently, we relocate the zImage so it appears in the first 128 MiB, disable the MMU and caches and invoke the decompressor via its ordinary entry point. However, since we can pass the base of DRAM directly, there is no need to relocate the zImage, which also means there is no need to disable and re-enable the caches and create new page tables etc.
Quoted 6 lines
This simplification is implemented by patch #5. Patches #1 - #4 are prerequisite changes to permit the decompressor to execute from the offset chosen by the UEFI firmware.
Note that this applies onto v5.7-rc1 with Geert's patch 'ARM: boot: Obtain start of physical memory from DTB' applied [0]
Looks fine to me. As you mentioned, I really like the register liveness area reduction.
Nicolas