Skip to contentSkip to conversation

x86/efi/mixed: Decouple from legacy decompressor

PATCH Superseded Ard Biesheuvel 2 participants 8 messages Jan 8, 2025 – Feb 25, 2025

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover x86/efi/mixed: Decouple from legacy decompressor

From: Ard Biesheuvel <[email protected]>

Since commit

a1b87d54f4e4 ("x86/efistub: Avoid legacy decompressor when doing EFI boot")

booting via the EFI stub no longer relies on the legacy decompressor, and instead, the kernel proper is decompressed by code executing in the context of the EFI boot services, and subsequently invoked directly.

The only remaining dependency is the EFI mixed mode startup code, which makes a detour via the legacy decompressor's 32-bit entrypoint, in order to obtain a 1:1 mapping of memory, which is a prerequisite for 64-bit execution on x86.

This detour requires some fiddly setup on the part of the mixed mode startup code, which has to stash the firmware stack pointer and boot arguments in memory, and create a fake struct boot_params to trick the code in startup_32 to behave as intended.

This dependency also impedes reuse of the EFI stub code in other contexts, such as generic EFI zboot, which will reuse the EFI stub but not the legacy decompressor.

So remove this dependency, by replacing this detour with a minimal reimplementation of the 1:1 mapping code. With some further cleanup applied on top, the line count barely changes, but the resulting code can operate independently from the legacy decompressor, and is therefore moved out of arch/x86/boot/compressed and into the EFI libstub/ directory.

Ard Biesheuvel (6):
x86/efi/mixed: Check CPU compatibility without relying on verify_cpu() x86/efi/mixed: Remove dependency on legacy startup_32 code x86/efi/mixed: Don't bother preserving 64-bit mode segment selectors x86/efi/mixed: Simplify and document thunking logic x86/efi/mixed: Reduce padding by moving some code around x86/efi/mixed: Move mixed mode startup code into libstub

 arch/x86/boot/compressed/Makefile                                                |   1 -
 arch/x86/boot/compressed/head_64.S                                               |   7 -
 drivers/firmware/efi/libstub/Makefile                                            |   3 +
 arch/x86/boot/compressed/efi_mixed.S => drivers/firmware/efi/libstub/x86-mixed.S | 320 ++++++++++----------
 4 files changed, 169 insertions(+), 162 deletions(-)
 rename arch/x86/boot/compressed/efi_mixed.S => drivers/firmware/efi/libstub/x86-mixed.S (54%)
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
#1 lore raw

PATCH 1/6 x86/efi/mixed: Check CPU compatibility without relying on verify_cpu()

From: Ard Biesheuvel <[email protected]>

In order for the EFI mixed mode startup code to be reusable in a context where the legacy decompressor is not used, replace the call to verify_cpu() [which performs an elaborate set of checks] with a simple check against the 'long mode' bit in the appropriate CPUID leaf.

This is reasonable, given that EFI support is implied when booting in this manner, and so there is no need to consider very old CPUs when performing this check.

---
 arch/x86/boot/compressed/efi_mixed.S | 22 ++++++++------------
 1 file changed, 9 insertions(+), 13 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +9 -13
index 876fc6d46a13..067e10eb7897 100644
@@ -293,24 +293,20 @@ SYM_FUNC_END(efi32_entry)
  *			       efi_system_table_32_t *sys_table)
  */
 SYM_FUNC_START(efi32_pe_entry)
-	pushl	%ebp
-	movl	%esp, %ebp
 	pushl	%ebx				// save callee-save registers
-	pushl	%edi
-
-	call	verify_cpu			// check for long mode support
-	testl	%eax, %eax
-	movl	$0x80000003, %eax		// EFI_UNSUPPORTED
-	jnz	2f
 
-	movl	8(%ebp), %ecx			// image_handle
-	movl	12(%ebp), %edx			// sys_table
+	/* Check whether the CPU supports long mode */
+	movl	$0x80000001, %eax		// assume extended info support
+	cpuid
+	btl	$29, %edx			// check long mode bit
+	jnc	1f
+	leal	8(%esp), %esp			// preserve stack alignment
+	movl	(%esp), %ecx			// image_handle
+	movl	4(%esp), %edx			// sys_table
 	jmp	efi32_entry			// pass %ecx, %edx
 						// no other registers remain live
-
-2:	popl	%edi				// restore callee-save registers
+1:	movl	$0x80000003, %eax		// EFI_UNSUPPORTED
 	popl	%ebx
-	leave
 	RET
 SYM_FUNC_END(efi32_pe_entry)
 
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
#2 lore raw

PATCH 2/6 x86/efi/mixed: Remove dependency on legacy startup_32 code

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

The EFI mixed mode startup code calls into startup_32 in the legacy decompressor with a mocked up boot_params struct, only to get it to set up the 1:1 mapping of the lower 4 GiB of memory and switch to a GDT that supports 64-bit mode.

In order to be able to reuse the EFI mixed mode startup code in EFI zboot images, which do not incorporate the legacy decompressor code, decouple it, by building the 1:1 map and installing the GDT directly.

This also removes the need to preserve and restore the stack pointer and the arguments passed to the entrypoint, and instead, the firmware stack can be retained, and the arguments can be loaded from it at the point where they are needed, i.e., before calling into the EFI stub C code.

---
 arch/x86/boot/compressed/efi_mixed.S | 143 ++++++++++++--------
 arch/x86/boot/compressed/head_64.S   |   7 -
 2 files changed, 86 insertions(+), 64 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +86 -57
index 067e10eb7897..c010aba7a0d0 100644
@@ -15,15 +15,18 @@
  */
 
 #include <linux/linkage.h>
-#include <asm/asm-offsets.h>
 #include <asm/msr.h>
 #include <asm/page_types.h>
+#include <asm/pgtable_types.h>
 #include <asm/processor-flags.h>
 #include <asm/segment.h>
-#include <asm/setup.h>
 
 	.code64
 	.text
+	.balign	8
+SYM_DATA_LOCAL(gdt, .quad 0x0, 0x0, 0xaf9a000000ffff)	/* __KERNEL_CS */
+	.set	gdt_size, . - gdt
+
 /*
  * When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode()
  * is the first thing that runs after switching to long mode. Depending on
@@ -35,30 +38,34 @@
  * pointer is used to disambiguate.
  *
  *                                                             +--------------+
- *  +------------------+     +------------+            +------>| efi_pe_entry |
- *  | efi32_pe_entry   |---->|            |            |       +-----------+--+
- *  +------------------+     |            |     +------+----------------+  |
- *                           | startup_32 |---->| startup_64_mixed_mode |  |
- *  +------------------+     |            |     +------+----------------+  |
- *  | efi32_stub_entry |---->|            |            |                   |
- *  +------------------+     +------------+            |                   |
+ *  +------------------+    +-------------+            +------>| efi_pe_entry |
+ *  | efi32_pe_entry   |--->|             |            |       +-----------+--+
+ *  +------------------+    |             |     +------+----------------+  |
+ *                          | efi32_entry |---->| startup_64_mixed_mode |  |
+ *  +------------------+    |             |     +------+----------------+  |
+ *  | efi32_stub_entry |--->|             |            |                   |
+ *  +------------------+    +-------------+            |                   |
  *                                                     V                   |
- *                           +------------+     +----------------+         |
- *                           | startup_64 |<----| efi_stub_entry |<--------+
- *                           +------------+     +----------------+
+ *                          +-------------+     +----------------+         |
+ *                          |   vmlinux   |<----| efi_stub_entry |<--------+
+ *                          +-------------+     +----------------+
  */
-SYM_FUNC_START(startup_64_mixed_mode)
-	lea	efi32_boot_args(%rip), %rdx
-	mov	0(%rdx), %edi
-	mov	4(%rdx), %esi
+SYM_FUNC_START_LOCAL_NOALIGN(startup_64_mixed_mode)
+	xorl	%eax, %eax
+	movl	%eax, %ds
+	movl	%eax, %es
+	movl	%eax, %ss
+	movl	%eax, %fs
+	movl	%eax, %gs
 
-	/* Switch to the firmware's stack */
-	movl	efi32_boot_sp(%rip), %esp
-	andl	$~7, %esp
+	movl	0(%rsp), %ecx		// MS calling convention
+	movl	4(%rsp), %edx
 
 #ifdef CONFIG_EFI_HANDOVER_PROTOCOL
-	mov	8(%rdx), %edx		// saved bootparams pointer
-	test	%edx, %edx
+	test	%edi, %edi		// struct boot_params provided?
+	movl	%edx, %esi		// SysV calling convention
+	cmovnzl	%edi, %edx
+	movl	%ecx, %edi
 	jnz	efi_stub_entry
 #endif
 	/*
@@ -69,8 +76,6 @@ SYM_FUNC_START(startup_64_mixed_mode)
 	 * the correct stack alignment for entry.
 	 */
 	sub	$40, %rsp
-	mov	%rdi, %rcx		// MS calling convention
-	mov	%rsi, %rdx
 	jmp	efi_pe_entry
 SYM_FUNC_END(startup_64_mixed_mode)
 
@@ -151,7 +156,6 @@ SYM_FUNC_END(__efi64_thunk)
 SYM_FUNC_START(efi32_stub_entry)
 	call	1f
 1:	popl	%ecx
-	leal	(efi32_boot_args - 1b)(%ecx), %ebx
 
 	/* Clear BSS */
 	xorl	%eax, %eax
@@ -163,10 +167,7 @@ SYM_FUNC_START(efi32_stub_entry)
 	rep	stosl
 
 	add	$0x4, %esp		/* Discard return address */
-	popl	%ecx
-	popl	%edx
-	popl	%esi
-	movl	%esi, 8(%ebx)
+	movl	8(%esp), %edi		/* struct boot_params pointer */
 	jmp	efi32_entry
 SYM_FUNC_END(efi32_stub_entry)
 #endif
@@ -241,8 +242,9 @@ SYM_FUNC_END(efi_enter32)
 /*
  * This is the common EFI stub entry point for mixed mode.
  *
- * Arguments:	%ecx	image handle
- * 		%edx	EFI system table pointer
+ * Arguments:	0(%esp)	image handle
+ * 		4(%esp)	EFI system table pointer
+ *		%edi	struct boot_params pointer (or NULL)
  *
  * Since this is the point of no return for ordinary execution, no registers
  * are considered live except for the function parameters. [Note that the EFI
@@ -261,31 +263,58 @@ SYM_FUNC_START_LOCAL(efi32_entry)
 	/* Store firmware IDT descriptor */
 	sidtl	(efi32_boot_idt - 1b)(%ebx)
 
-	/* Store firmware stack pointer */
-	movl	%esp, (efi32_boot_sp - 1b)(%ebx)
-
-	/* Store boot arguments */
-	leal	(efi32_boot_args - 1b)(%ebx), %ebx
-	movl	%ecx, 0(%ebx)
-	movl	%edx, 4(%ebx)
-	movb	$0x0, 12(%ebx)          // efi_is64
-
-	/*
-	 * Allocate some memory for a temporary struct boot_params, which only
-	 * needs the minimal pieces that startup_32() relies on.
-	 */
-	subl	$PARAM_SIZE, %esp
-	movl	%esp, %esi
-	movl	$PAGE_SIZE, BP_kernel_alignment(%esi)
-	movl	$_end - 1b, BP_init_size(%esi)
-	subl	$startup_32 - 1b, BP_init_size(%esi)
+	/* Record mixed mode entry */
+	movb	$0x0, (efi_is64 - 1b)(%ebx)
 
 	/* Disable paging */
 	movl	%cr0, %eax
 	btrl	$X86_CR0_PG_BIT, %eax
 	movl	%eax, %cr0
 
-	jmp	startup_32
+	/* Set up 1:1 mapping */
+	leal	(pte - 1b)(%ebx), %eax
+	movl	$_PAGE_PRESENT | _PAGE_RW | _PAGE_PSE, %ecx
+	leal	(_PAGE_PRESENT | _PAGE_RW)(%eax), %edx
+2:	movl	%ecx, (%eax)
+	addl	$8, %eax
+	addl	$PMD_SIZE, %ecx
+	jnc	2b
+
+	xor	%ecx, %ecx
+	movl	$PAGE_SIZE, %esi
+3:	movl	%edx, (%eax,%ecx,8)
+	addl	%esi, %edx
+	inc	%ecx
+	cmp	$4, %ecx
+	jl	3b
+
+	addl	%esi, %eax
+	movl	%edx, (%eax)
+	movl	%eax, %cr3
+
+	movl	%cr4, %eax
+	orl	$X86_CR4_PAE, %eax
+	movl	%eax, %cr4
+
+	movl	$MSR_EFER, %ecx
+	rdmsr
+	btsl	$_EFER_LME, %eax
+	wrmsr
+
+	leal	(gdt - 1b)(%ebx), %ecx
+	pushl	%ecx
+	pushw	$gdt_size - 1
+	lgdtl	(%esp)
+	lea	6(%esp), %esp
+
+	/* Enable paging and jump to long mode */
+	leal	(startup_64_mixed_mode - 1b)(%ebx), %ecx
+	pushl	$__KERNEL_CS
+	pushl	%ecx
+	movl	%cr0, %eax
+	btsl	$X86_CR0_PG_BIT, %eax
+	movl	%eax, %cr0
+	lret
 SYM_FUNC_END(efi32_entry)
 
 /*
@@ -301,10 +330,8 @@ SYM_FUNC_START(efi32_pe_entry)
 	btl	$29, %edx			// check long mode bit
 	jnc	1f
 	leal	8(%esp), %esp			// preserve stack alignment
-	movl	(%esp), %ecx			// image_handle
-	movl	4(%esp), %edx			// sys_table
-	jmp	efi32_entry			// pass %ecx, %edx
-						// no other registers remain live
+	xor	%edi, %edi			// no struct boot_params in EDI
+	jmp	efi32_entry			// only ESP and EDI remain live
 1:	movl	$0x80000003, %eax		// EFI_UNSUPPORTED
 	popl	%ebx
 	RET
@@ -318,8 +345,10 @@ SYM_FUNC_START_NOALIGN(efi64_stub_entry)
 SYM_FUNC_END(efi64_stub_entry)
 #endif
 
-	.data
-	.balign	8
+	.bss
+	.balign PAGE_SIZE
+SYM_DATA_LOCAL(pte, .fill 6 * PAGE_SIZE, 1, 0)
+
 SYM_DATA_START_LOCAL(efi32_boot_gdt)
 	.word	0
 	.quad	0
@@ -330,8 +359,8 @@ SYM_DATA_START_LOCAL(efi32_boot_idt)
 	.quad	0
 SYM_DATA_END(efi32_boot_idt)
 
+	.data
+	.balign	4
 SYM_DATA_LOCAL(efi32_boot_cs, .word 0)
 SYM_DATA_LOCAL(efi32_boot_ds, .word 0)
-SYM_DATA_LOCAL(efi32_boot_sp, .long 0)
-SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0)
 SYM_DATA(efi_is64, .byte 1)
arch/x86/boot/compressed/head_64.S +0 -7
index 1dcb794c5479..5db6495a3bb9 100644
@@ -263,13 +263,6 @@ SYM_FUNC_START(startup_32)
 	 * used to perform that far jump.
 	 */
 	leal	rva(startup_64)(%ebp), %eax
-#ifdef CONFIG_EFI_MIXED
-	cmpb	$1, rva(efi_is64)(%ebp)
-	je	1f
-	leal	rva(startup_64_mixed_mode)(%ebp), %eax
-1:
-#endif
-
 	pushl	$__KERNEL_CS
 	pushl	%eax
 
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
#3 lore raw

PATCH 3/6 x86/efi/mixed: Don't bother preserving 64-bit mode segment selectors

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Commit

e10848a26a96 ("x86/efi: Preserve segment registers in mixed mode")

added code to preserve and restore the caller's data segment selector values when temporarily switching back to the firmware's GDT and segment layout to perform an EFI runtime call.

At that point, this code was shared between the boot-time and runtime mixed mode implementations, and the issue this commit aimed to address only affected the latter.

Since commit

96738c69a7fc ("x86/efi: Avoid triple faults during EFI mixed mode calls")

the runtime version of the mixed mode handling is entirely separate, and relies on IA-32e compatibility mode to call into the 32-bit firmware while retaining the OS's GDT/IDT and segment layout.

At boot-time, preserving/restoring the values is pointless, and the data segment selectors can now simply be zeroed after (re)entering 64-bit mode.

---
 arch/x86/boot/compressed/efi_mixed.S | 24 ++++++--------------
 1 file changed, 7 insertions(+), 17 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +7 -17
index c010aba7a0d0..066f4365af4f 100644
@@ -83,17 +83,10 @@ SYM_FUNC_START(__efi64_thunk)
 	push	%rbp
 	push	%rbx
 
-	movl	%ds, %eax
-	push	%rax
-	movl	%es, %eax
-	push	%rax
-	movl	%ss, %eax
-	push	%rax
-
 	/* Copy args passed on stack */
-	movq	0x30(%rsp), %rbp
-	movq	0x38(%rsp), %rbx
-	movq	0x40(%rsp), %rax
+	movq	0x18(%rsp), %rbp
+	movq	0x20(%rsp), %rbx
+	movq	0x28(%rsp), %rax
 
 	/*
 	 * Convert x86-64 ABI params to i386 ABI
@@ -135,14 +128,11 @@ SYM_FUNC_START(__efi64_thunk)
 1:	addq	$64, %rsp
 	movq	%rdi, %rax
 
-	pop	%rbx
-	movl	%ebx, %ss
-	pop	%rbx
-	movl	%ebx, %es
-	pop	%rbx
-	movl	%ebx, %ds
-	/* Clear out 32-bit selector from FS and GS */
+	/* Clear out 32-bit segment selectors */
 	xorl	%ebx, %ebx
+	movl	%ebx, %ds
+	movl	%ebx, %es
+	movl	%ebx, %ss
 	movl	%ebx, %fs
 	movl	%ebx, %gs
 
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
#4 lore raw

PATCH 4/6 x86/efi/mixed: Simplify and document thunking logic

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

The current boot-time version of the thunk into 32-bit mode, to invoke EFI services and protocols using the 32-bit calling convention, is a jumble of argument marshalling code and GDT/IDT and segment register handling, with an undocumented de facto calling convention that passes the data segment descriptor, the return address and the GDT/IDT base pointer for the return in registers EDX, EBP and EBX respectively.

Let's clean this up, and replace this with documented logic that separates the handling of the segment registers into the 64-bit caller, and the marshalling of the arguments into the 32-bit callee. Also, replace the open coded far call with an actual one, removing the need to pass the return address via a register.

---
 arch/x86/boot/compressed/efi_mixed.S | 92 ++++++++++----------
 1 file changed, 44 insertions(+), 48 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +44 -48
index 066f4365af4f..a44e522bbbed 100644
@@ -80,52 +80,36 @@ SYM_FUNC_START_LOCAL_NOALIGN(startup_64_mixed_mode)
 SYM_FUNC_END(startup_64_mixed_mode)
 
 SYM_FUNC_START(__efi64_thunk)
-	push	%rbp
 	push	%rbx
 
-	/* Copy args passed on stack */
-	movq	0x18(%rsp), %rbp
-	movq	0x20(%rsp), %rbx
-	movq	0x28(%rsp), %rax
-
-	/*
-	 * Convert x86-64 ABI params to i386 ABI
-	 */
-	subq	$64, %rsp
-	movl	%esi, 0x0(%rsp)
-	movl	%edx, 0x4(%rsp)
-	movl	%ecx, 0x8(%rsp)
-	movl	%r8d, 0xc(%rsp)
-	movl	%r9d, 0x10(%rsp)
-	movl	%ebp, 0x14(%rsp)
-	movl	%ebx, 0x18(%rsp)
-	movl	%eax, 0x1c(%rsp)
-
-	leaq	0x20(%rsp), %rbx
-	sgdt	(%rbx)
-	sidt	16(%rbx)
-
-	leaq	1f(%rip), %rbp
+	/* Store live GDT and IDT descriptors */
+	subq	$16, %rsp
+	sgdt	(%rsp)
+	sidt	6(%rsp)
 
 	/*
 	 * Switch to IDT and GDT with 32-bit segments. These are the firmware
 	 * GDT and IDT that were installed when the kernel started executing.
 	 * The pointers were saved by the efi32_entry() routine below.
-	 *
-	 * Pass the saved DS selector to the 32-bit code, and use far return to
-	 * restore the saved CS selector.
 	 */
 	lidt	efi32_boot_idt(%rip)
 	lgdt	efi32_boot_gdt(%rip)
 
-	movzwl	efi32_boot_ds(%rip), %edx
-	movzwq	efi32_boot_cs(%rip), %rax
-	pushq	%rax
-	leaq	efi_enter32(%rip), %rax
-	pushq	%rax
-	lretq
+	/* Reload firmware's data segment selectors */
+	movw	efi32_boot_ds(%rip), %bx
+	movl	%ebx, %ds
+	movl	%ebx, %es
+	movl	%ebx, %ss
+	movl	%ebx, %fs
+	movl	%ebx, %gs
+
+	/* Move args #5 and #6 into 32-bit accessible registers */
+	movl	%r8d, %eax
+	movl	%r9d, %ebx
 
-1:	addq	$64, %rsp
+	lcalll	*efi32_thunk(%rip)
+
+	addq	$16, %rsp
 	movq	%rdi, %rax
 
 	/* Clear out 32-bit segment selectors */
@@ -137,7 +121,6 @@ SYM_FUNC_START(__efi64_thunk)
 	movl	%ebx, %gs
 
 	pop	%rbx
-	pop	%rbp
 	RET
 SYM_FUNC_END(__efi64_thunk)
 
@@ -163,17 +146,27 @@ SYM_FUNC_END(efi32_stub_entry)
 #endif
 
 /*
- * EFI service pointer must be in %edi.
+ * Called using a far call from 64-bit code, using the x86_64 SysV ABI (except
+ * for R8/R9 which are inaccessible to 32-bit code - EAX/EBX are used instead).
+ *
+ * The first argument (EDI) is a pointer to the boot service or protocol, to
+ * which the remaining arguments are passed, each truncated to 32 bits.
  *
- * The stack should represent the 32-bit calling convention.
+ * Entered with ESP+40 pointing to the arguments passed via the stack, and with
+ * the 64-bit mode GDT and IDT descriptors at ESP+8 and ESP+14, respectively.
  */
 SYM_FUNC_START_LOCAL(efi_enter32)
-	/* Load firmware selector into data and stack segment registers */
-	movl	%edx, %ds
-	movl	%edx, %es
-	movl	%edx, %fs
-	movl	%edx, %gs
-	movl	%edx, %ss
+	/*
+	 * Convert x86-64 SysV ABI params to i386 ABI
+	 */
+	pushl	56(%esp)	/* Up to 3 args passed via the caller's stack */
+	pushl	52(%esp)
+	pushl	48(%esp)
+	pushl	%ebx		/* R9 */
+	pushl	%eax		/* R8 */
+	pushl	%ecx
+	pushl	%edx
+	pushl	%esi
 
 	/* Reload pgtables */
 	movl	%cr3, %eax
@@ -201,8 +194,9 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 	 */
 	cli
 
-	lidtl	16(%ebx)
-	lgdtl	(%ebx)
+	addl	$32, %esp
+	lidtl	14(%esp)
+	lgdtl	8(%esp)
 
 	movl	%cr4, %eax
 	btsl	$(X86_CR4_PAE_BIT), %eax
@@ -219,9 +213,6 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 	xorl	%eax, %eax
 	lldt	%ax
 
-	pushl	$__KERNEL_CS
-	pushl	%ebp
-
 	/* Enable paging */
 	movl	%cr0, %eax
 	btsl	$X86_CR0_PG_BIT, %eax
@@ -250,6 +241,10 @@ SYM_FUNC_START_LOCAL(efi32_entry)
 	movw	%cs, (efi32_boot_cs - 1b)(%ebx)
 	movw	%ds, (efi32_boot_ds - 1b)(%ebx)
 
+	/* Fix up absolute reference */
+	leal	(efi32_thunk - 1b)(%ebx), %eax
+	addl	%eax, (%eax)
+
 	/* Store firmware IDT descriptor */
 	sidtl	(efi32_boot_idt - 1b)(%ebx)
 
@@ -351,6 +346,7 @@ SYM_DATA_END(efi32_boot_idt)
 
 	.data
 	.balign	4
+SYM_DATA_LOCAL(efi32_thunk, .long efi_enter32 - .)
 SYM_DATA_LOCAL(efi32_boot_cs, .word 0)
 SYM_DATA_LOCAL(efi32_boot_ds, .word 0)
 SYM_DATA(efi_is64, .byte 1)
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
#5 lore raw

PATCH 5/6 x86/efi/mixed: Reduce padding by moving some code around

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

The 32-bit and 64-bit entrypoints for the EFI handover protocol need to be exactly 0x200 bytes apart, and so the 64-bit version is placed at the end, using padding to move it into place.

This padding can be reduced now, since the code in between has reduced in size as well. So move efi32_stub_entry() to the beginning of the source file.

Note that this code is outside of the compressed kernel image, and so the padding is not compressed.

---
 arch/x86/boot/compressed/efi_mixed.S | 51 ++++++++++----------
 1 file changed, 26 insertions(+), 25 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +26 -25
index a44e522bbbed..b5a46a4ce083 100644
@@ -21,12 +21,32 @@
 #include <asm/processor-flags.h>
 #include <asm/segment.h>
 
-	.code64
 	.text
 	.balign	8
 SYM_DATA_LOCAL(gdt, .quad 0x0, 0x0, 0xaf9a000000ffff)	/* __KERNEL_CS */
 	.set	gdt_size, . - gdt
 
+#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
+	.code32
+SYM_FUNC_START_NOALIGN(efi32_stub_entry)
+	call	1f
+1:	popl	%ecx
+
+	/* Clear BSS */
+	xorl	%eax, %eax
+	leal	(_bss - 1b)(%ecx), %edi
+	leal	(_ebss - 1b)(%ecx), %ecx
+	subl	%edi, %ecx
+	shrl	$2, %ecx
+	cld
+	rep	stosl
+
+	add	$0x4, %esp		/* Discard return address */
+	movl	8(%esp), %edi		/* struct boot_params pointer */
+	jmp	efi32_entry
+SYM_FUNC_END(efi32_stub_entry)
+#endif
+
 /*
  * When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode()
  * is the first thing that runs after switching to long mode. Depending on
@@ -50,6 +70,7 @@ SYM_DATA_LOCAL(gdt, .quad 0x0, 0x0, 0xaf9a000000ffff)	/* __KERNEL_CS */
  *                          |   vmlinux   |<----| efi_stub_entry |<--------+
  *                          +-------------+     +----------------+
  */
+	.code64
 SYM_FUNC_START_LOCAL_NOALIGN(startup_64_mixed_mode)
 	xorl	%eax, %eax
 	movl	%eax, %ds
@@ -79,7 +100,7 @@ SYM_FUNC_START_LOCAL_NOALIGN(startup_64_mixed_mode)
 	jmp	efi_pe_entry
 SYM_FUNC_END(startup_64_mixed_mode)
 
-SYM_FUNC_START(__efi64_thunk)
+SYM_FUNC_START_NOALIGN(__efi64_thunk)
 	push	%rbx
 
 	/* Store live GDT and IDT descriptors */
@@ -125,26 +146,6 @@ SYM_FUNC_START(__efi64_thunk)
 SYM_FUNC_END(__efi64_thunk)
 
 	.code32
-#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
-SYM_FUNC_START(efi32_stub_entry)
-	call	1f
-1:	popl	%ecx
-
-	/* Clear BSS */
-	xorl	%eax, %eax
-	leal	(_bss - 1b)(%ecx), %edi
-	leal	(_ebss - 1b)(%ecx), %ecx
-	subl	%edi, %ecx
-	shrl	$2, %ecx
-	cld
-	rep	stosl
-
-	add	$0x4, %esp		/* Discard return address */
-	movl	8(%esp), %edi		/* struct boot_params pointer */
-	jmp	efi32_entry
-SYM_FUNC_END(efi32_stub_entry)
-#endif
-
 /*
  * Called using a far call from 64-bit code, using the x86_64 SysV ABI (except
  * for R8/R9 which are inaccessible to 32-bit code - EAX/EBX are used instead).
@@ -155,7 +156,7 @@ SYM_FUNC_END(efi32_stub_entry)
  * Entered with ESP+40 pointing to the arguments passed via the stack, and with
  * the 64-bit mode GDT and IDT descriptors at ESP+8 and ESP+14, respectively.
  */
-SYM_FUNC_START_LOCAL(efi_enter32)
+SYM_FUNC_START_LOCAL_NOALIGN(efi_enter32)
 	/*
 	 * Convert x86-64 SysV ABI params to i386 ABI
 	 */
@@ -232,7 +233,7 @@ SYM_FUNC_END(efi_enter32)
  * stub may still exit and return to the firmware using the Exit() EFI boot
  * service.]
  */
-SYM_FUNC_START_LOCAL(efi32_entry)
+SYM_FUNC_START_LOCAL_NOALIGN(efi32_entry)
 	call	1f
 1:	pop	%ebx
 
@@ -306,7 +307,7 @@ SYM_FUNC_END(efi32_entry)
  * efi_status_t efi32_pe_entry(efi_handle_t image_handle,
  *			       efi_system_table_32_t *sys_table)
  */
-SYM_FUNC_START(efi32_pe_entry)
+SYM_FUNC_START_NOALIGN(efi32_pe_entry)
 	pushl	%ebx				// save callee-save registers
 
 	/* Check whether the CPU supports long mode */
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
#6 lore raw

PATCH 6/6 x86/efi/mixed: Move mixed mode startup code into libstub

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

The EFI mixed mode code has been decoupled from the legacy decompressor, in order to be able to reuse it with generic EFI zboot images for x86.

Move the source file into the libstub source directory to facilitate this.

---
 arch/x86/boot/compressed/Makefile                                                | 1 -
 drivers/firmware/efi/libstub/Makefile                                            | 3 +++
 arch/x86/boot/compressed/efi_mixed.S => drivers/firmware/efi/libstub/x86-mixed.S | 0
 3 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/boot/compressed/Makefile +0 -1
index f2051644de94..fc5563704466 100644
@@ -104,7 +104,6 @@ vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o $(obj)/tdcall.o $(obj)/td
 vmlinux-objs-$(CONFIG_UNACCEPTED_MEMORY) += $(obj)/mem.o
 
 vmlinux-objs-$(CONFIG_EFI) += $(obj)/efi.o
-vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_mixed.o
 vmlinux-libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
 
 $(obj)/vmlinux: $(vmlinux-objs-y) $(vmlinux-libs-y) FORCE
drivers/firmware/efi/libstub/Makefile +3 -0
index e04285a7a6b9..f8cbd4557131 100644
@@ -62,6 +62,8 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
 # `-fdata-sections` flag from KBUILD_CFLAGS_KERNEL
 KBUILD_CFLAGS_KERNEL := $(filter-out -fdata-sections, $(KBUILD_CFLAGS_KERNEL))
 
+KBUILD_AFLAGS			:= $(KBUILD_CFLAGS) -D__ASSEMBLY__
+
 lib-y				:= efi-stub-helper.o gop.o secureboot.o tpm.o \
 				   file.o mem.o random.o randomalloc.o pci.o \
 				   skip_spaces.o lib-cmdline.o lib-ctype.o \
@@ -83,6 +85,7 @@ lib-$(CONFIG_EFI_GENERIC_STUB)	+= efi-stub.o string.o intrinsics.o systable.o \
 lib-$(CONFIG_ARM)		+= arm32-stub.o
 lib-$(CONFIG_ARM64)		+= kaslr.o arm64.o arm64-stub.o smbios.o
 lib-$(CONFIG_X86)		+= x86-stub.o smbios.o
+lib-$(CONFIG_EFI_MIXED)		+= x86-mixed.o
 lib-$(CONFIG_X86_64)		+= x86-5lvl.o
 lib-$(CONFIG_RISCV)		+= kaslr.o riscv.o riscv-stub.o
 lib-$(CONFIG_LOONGARCH)		+= loongarch.o loongarch-stub.o
drivers/firmware/efi/libstub/x86-mixed.S renamed from arch/x86/boot/compressed/efi_mixed.S +0 -0
similarity index 100%
rename from arch/x86/boot/compressed/efi_mixed.S
rename to drivers/firmware/efi/libstub/x86-mixed.S
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Acked-by on the v1 series
#7 lore raw
In reply to Ard Biesheuvel
Quoted 39 lines · Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Since commit

a1b87d54f4e4 ("x86/efistub: Avoid legacy decompressor when doing EFI boot")

booting via the EFI stub no longer relies on the legacy decompressor, and instead, the kernel proper is decompressed by code executing in the context of the EFI boot services, and subsequently invoked directly.

The only remaining dependency is the EFI mixed mode startup code, which makes a detour via the legacy decompressor's 32-bit entrypoint, in order to obtain a 1:1 mapping of memory, which is a prerequisite for 64-bit execution on x86.

This detour requires some fiddly setup on the part of the mixed mode startup code, which has to stash the firmware stack pointer and boot arguments in memory, and create a fake struct boot_params to trick the code in startup_32 to behave as intended.

This dependency also impedes reuse of the EFI stub code in other contexts, such as generic EFI zboot, which will reuse the EFI stub but not the legacy decompressor.

So remove this dependency, by replacing this detour with a minimal reimplementation of the 1:1 mapping code. With some further cleanup applied on top, the line count barely changes, but the resulting code can operate independently from the legacy decompressor, and is therefore moved out of arch/x86/boot/compressed and into the EFI libstub/ directory.

Ard Biesheuvel (6):
x86/efi/mixed: Check CPU compatibility without relying on verify_cpu() x86/efi/mixed: Remove dependency on legacy startup_32 code x86/efi/mixed: Don't bother preserving 64-bit mode segment selectors x86/efi/mixed: Simplify and document thunking logic x86/efi/mixed: Reduce padding by moving some code around x86/efi/mixed: Move mixed mode startup code into libstub

Quoted 6 lines
 arch/x86/boot/compressed/Makefile                                                |   1 -
 arch/x86/boot/compressed/head_64.S                                               |   7 -
 drivers/firmware/efi/libstub/Makefile                                            |   3 +
 arch/x86/boot/compressed/efi_mixed.S => drivers/firmware/efi/libstub/x86-mixed.S | 320 ++++++++++----------
 4 files changed, 169 insertions(+), 162 deletions(-)
 rename arch/x86/boot/compressed/efi_mixed.S => drivers/firmware/efi/libstub/x86-mixed.S (54%)

A belated:

Thanks,

Ingo