Skip to contentSkip to conversation

x86/efi/mixed: Decouple from legacy decompressor

PATCH In review Ard Biesheuvel 5 participants 14 messages Feb 10, 2025 – Feb 27, 2025

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover x86/efi/mixed: Decouple from legacy decompressor

From: Ard Biesheuvel <[email protected]>

Since commit

a1b87d54f4e4 ("x86/efistub: Avoid legacy decompressor when doing EFI boot")

booting via the EFI stub no longer relies on the legacy decompressor, and instead, the kernel proper is decompressed by code executing in the context of the EFI boot services, and subsequently invoked directly.

The only remaining dependency is the EFI mixed mode startup code, which makes a detour via the legacy decompressor's 32-bit entrypoint, in order to obtain a 1:1 mapping of memory, which is a prerequisite for 64-bit execution on x86.

This detour requires some fiddly setup on the part of the mixed mode startup code, which has to stash the firmware stack pointer and boot arguments in memory, and create a fake struct boot_params to trick the code in startup_32 to behave as intended.

This dependency also impedes reuse of the EFI stub code in other contexts, such as generic EFI zboot, which will reuse the EFI stub but not the legacy decompressor.

So remove this dependency, by replacing this detour with a minimal reimplementation of the 1:1 mapping code. With some further cleanup applied on top, the line count drops substantially, but without loss of functionality. The resulting code can operate independently from the legacy decompressor, and is therefore moved out of arch/x86/boot/ and into the EFI libstub/ directory.

Changes since v1 [0]:
- Create new long mode GDT that extends the firmware's 32-bit only GDT so that preserving/restoring data segment selectors or swapping out GDTs and IDTs is no longer needed at all.
- Rebase onto v6.14-rc1

[0] https://lore.kernel.org/all/[email protected]/

Ard Biesheuvel (7):
x86/efistub: Merge PE and handover entrypoints x86/efi/mixed: Check CPU compatibility without relying on verify_cpu() x86/efi/mixed: Factor out and clean up long mode entry x86/efi/mixed: Set up 1:1 mapping of lower 4GiB in the stub x86/efi/mixed: Remove dependency on legacy startup_32 code x86/efi/mixed: Simplify and document thunking logic x86/efi/mixed: Move mixed mode startup code into libstub

 arch/x86/boot/compressed/Makefile        |   1 -
 arch/x86/boot/compressed/efi_mixed.S     | 341 --------------------
 arch/x86/boot/compressed/head_64.S       |   7 -
 drivers/firmware/efi/libstub/Makefile    |   3 +
 drivers/firmware/efi/libstub/x86-mixed.S | 253 +++++++++++++++
 drivers/firmware/efi/libstub/x86-stub.c  |  52 +--
 6 files changed, 285 insertions(+), 372 deletions(-)
 delete mode 100644 arch/x86/boot/compressed/efi_mixed.S
 create mode 100644 drivers/firmware/efi/libstub/x86-mixed.S
Signature · 1 line
2.48.1.362.g079036d154-goog
#1 lore raw

PATCH v2 1/7 x86/efistub: Merge PE and handover entrypoints

From: Ard Biesheuvel <[email protected]>

The difference between the PE and handover entrypoints in the EFI stub is that the former allocates a struct boot_params whereas the latter expects one from the caller. Currently, these are two completely separate entrypoints, duplicating some logic and both relying of efi_exit() to return straight back to the firmware on an error.

Simplify this by making the PE entrypoint call the handover entrypoint with NULL as the argument for the struct boot_params parameter. This makes the code easier to follow, and removes the need to support two different calling conventions in the mixed mode asm code.

While at it, move the assignment of boot_params_ptr into the function that actually calls into the legacy decompressor, which is where its value is required.

---
 arch/x86/boot/compressed/efi_mixed.S    | 16 +-----
 drivers/firmware/efi/libstub/x86-stub.c | 52 +++++++++++---------
 2 files changed, 30 insertions(+), 38 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +1 -15
index 876fc6d46a13..d681e30c6732 100644
@@ -56,22 +56,8 @@ SYM_FUNC_START(startup_64_mixed_mode)
 	movl	efi32_boot_sp(%rip), %esp
 	andl	$~7, %esp
 
-#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
 	mov	8(%rdx), %edx		// saved bootparams pointer
-	test	%edx, %edx
-	jnz	efi_stub_entry
-#endif
-	/*
-	 * efi_pe_entry uses MS calling convention, which requires 32 bytes of
-	 * shadow space on the stack even if all arguments are passed in
-	 * registers. We also need an additional 8 bytes for the space that
-	 * would be occupied by the return address, and this also results in
-	 * the correct stack alignment for entry.
-	 */
-	sub	$40, %rsp
-	mov	%rdi, %rcx		// MS calling convention
-	mov	%rsi, %rdx
-	jmp	efi_pe_entry
+	call	efi_stub_entry
 SYM_FUNC_END(startup_64_mixed_mode)
 
 SYM_FUNC_START(__efi64_thunk)
drivers/firmware/efi/libstub/x86-stub.c +29 -23

134 lines not on this page · raw message

Signature · 1 line
2.48.1.362.g079036d154-goog
#2 lore raw

PATCH v2 2/7 x86/efi/mixed: Check CPU compatibility without relying on verify_cpu()

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

In order for the EFI mixed mode startup code to be reusable in a context where the legacy decompressor is not used, replace the call to verify_cpu() [which performs an elaborate set of checks] with a simple check against the 'long mode' bit in the appropriate CPUID leaf.

This is reasonable, given that EFI support is implied when booting in this manner, and so there is no need to consider very old CPUs when performing this check.

---
 arch/x86/boot/compressed/efi_mixed.S | 22 ++++++++------------
 1 file changed, 9 insertions(+), 13 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +9 -13
index d681e30c6732..b7886e2591fc 100644
@@ -279,24 +279,20 @@ SYM_FUNC_END(efi32_entry)
  *			       efi_system_table_32_t *sys_table)
  */
 SYM_FUNC_START(efi32_pe_entry)
-	pushl	%ebp
-	movl	%esp, %ebp
 	pushl	%ebx				// save callee-save registers
-	pushl	%edi
-
-	call	verify_cpu			// check for long mode support
-	testl	%eax, %eax
-	movl	$0x80000003, %eax		// EFI_UNSUPPORTED
-	jnz	2f
 
-	movl	8(%ebp), %ecx			// image_handle
-	movl	12(%ebp), %edx			// sys_table
+	/* Check whether the CPU supports long mode */
+	movl	$0x80000001, %eax		// assume extended info support
+	cpuid
+	btl	$29, %edx			// check long mode bit
+	jnc	1f
+	leal	8(%esp), %esp			// preserve stack alignment
+	movl	(%esp), %ecx			// image_handle
+	movl	4(%esp), %edx			// sys_table
 	jmp	efi32_entry			// pass %ecx, %edx
 						// no other registers remain live
-
-2:	popl	%edi				// restore callee-save registers
+1:	movl	$0x80000003, %eax		// EFI_UNSUPPORTED
 	popl	%ebx
-	leave
 	RET
 SYM_FUNC_END(efi32_pe_entry)
 
Signature · 1 line
2.48.1.362.g079036d154-goog
#3 lore raw

PATCH v2 3/7 x86/efi/mixed: Factor out and clean up long mode entry

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Entering long mode involves setting the EFER_LME and CR4.PAE bits before enabling paging by setting CR0.PG bit.

It also involves disabling interrupts, given that the firmware's 32-bit IDT becomes invalid as soon as the CPU transitions into long mode.

Reloading the CR3 register is not necessary at boot time, given that the EFI firmware as well as the kernel's EFI stub use a 1:1 mapping of the 32-bit addressable memory in the system.

Break out this code into a separate helper for clarity, and so that it can be reused in a subsequent patch.

---
 arch/x86/boot/compressed/efi_mixed.S | 29 ++++++++++----------
 1 file changed, 15 insertions(+), 14 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +15 -14
index b7886e2591fc..0b6b37b08f82 100644
@@ -170,10 +170,6 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 	movl	%edx, %gs
 	movl	%edx, %ss
 
-	/* Reload pgtables */
-	movl	%cr3, %eax
-	movl	%eax, %cr3
-
 	/* Disable paging */
 	movl	%cr0, %eax
 	btrl	$X86_CR0_PG_BIT, %eax
@@ -199,30 +195,35 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 	lidtl	16(%ebx)
 	lgdtl	(%ebx)
 
+	xorl	%eax, %eax
+	lldt	%ax
+
+	call	efi32_enable_long_mode
+
+	pushl	$__KERNEL_CS
+	pushl	%ebp
+	lret
+SYM_FUNC_END(efi_enter32)
+
+SYM_FUNC_START_LOCAL(efi32_enable_long_mode)
 	movl	%cr4, %eax
 	btsl	$(X86_CR4_PAE_BIT), %eax
 	movl	%eax, %cr4
 
-	movl	%cr3, %eax
-	movl	%eax, %cr3
-
 	movl	$MSR_EFER, %ecx
 	rdmsr
 	btsl	$_EFER_LME, %eax
 	wrmsr
 
-	xorl	%eax, %eax
-	lldt	%ax
-
-	pushl	$__KERNEL_CS
-	pushl	%ebp
+	/* Disable interrupts - the firmware's IDT does not work in long mode */
+	cli
 
 	/* Enable paging */
 	movl	%cr0, %eax
 	btsl	$X86_CR0_PG_BIT, %eax
 	movl	%eax, %cr0
-	lret
-SYM_FUNC_END(efi_enter32)
+	ret
+SYM_FUNC_END(efi32_enable_long_mode)
 
 /*
  * This is the common EFI stub entry point for mixed mode.
Signature · 1 line
2.48.1.362.g079036d154-goog
#4 lore raw

PATCH v2 4/7 x86/efi/mixed: Set up 1:1 mapping of lower 4GiB in the stub

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

In preparation for dropping the dependency on startup_32 entirely in the next patch, add the code that sets up the 1:1 mapping of the lower 4 GiB of system RAM to the mixed mode stub.

The reload of CR3 after the long mode switch will be removed in a subsequent patch, when it is no longer needed.

---
 arch/x86/boot/compressed/efi_mixed.S | 29 ++++++++++++++++++++
 1 file changed, 29 insertions(+)
arch/x86/boot/compressed/efi_mixed.S +29 -0
index 0b6b37b08f82..dca916c3e6f0 100644
@@ -18,6 +18,7 @@
 #include <asm/asm-offsets.h>
 #include <asm/msr.h>
 #include <asm/page_types.h>
+#include <asm/pgtable_types.h>
 #include <asm/processor-flags.h>
 #include <asm/segment.h>
 #include <asm/setup.h>
@@ -52,6 +53,9 @@ SYM_FUNC_START(startup_64_mixed_mode)
 	mov	0(%rdx), %edi
 	mov	4(%rdx), %esi
 
+	leaq	(pte + 5 * PAGE_SIZE)(%rip), %rax
+	movq	%rax, %cr3		// reload after startup_32
+
 	/* Switch to the firmware's stack */
 	movl	efi32_boot_sp(%rip), %esp
 	andl	$~7, %esp
@@ -267,11 +271,32 @@ SYM_FUNC_START_LOCAL(efi32_entry)
 	movl	$_end - 1b, BP_init_size(%esi)
 	subl	$startup_32 - 1b, BP_init_size(%esi)
 
+	call	1f
+1:	pop	%edi
+
 	/* Disable paging */
 	movl	%cr0, %eax
 	btrl	$X86_CR0_PG_BIT, %eax
 	movl	%eax, %cr0
 
+	/* Set up 1:1 mapping */
+	leal	(pte - 1b)(%edi), %eax
+	movl	$_PAGE_PRESENT | _PAGE_RW | _PAGE_PSE, %ecx
+	leal	(_PAGE_PRESENT | _PAGE_RW)(%eax), %edx
+2:	movl	%ecx, (%eax)
+	addl	$8, %eax
+	addl	$PMD_SIZE, %ecx
+	jnc	2b
+
+	movl	$PAGE_SIZE, %ecx
+	.irpc	l, 0123
+	movl	%edx, \l * 8(%eax)
+	addl	%ecx, %edx
+	.endr
+	addl	%ecx, %eax
+	movl	%edx, (%eax)
+	movl	%eax, %cr3
+
 	jmp	startup_32
 SYM_FUNC_END(efi32_entry)
 
@@ -322,3 +347,7 @@ SYM_DATA_LOCAL(efi32_boot_ds, .word 0)
 SYM_DATA_LOCAL(efi32_boot_sp, .long 0)
 SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0)
 SYM_DATA(efi_is64, .byte 1)
+
+	.bss
+	.balign PAGE_SIZE
+SYM_DATA_LOCAL(pte, .fill 6 * PAGE_SIZE, 1, 0)
Signature · 1 line
2.48.1.362.g079036d154-goog
#5 lore raw

PATCH v2 5/7 x86/efi/mixed: Remove dependency on legacy startup_32 code

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

The EFI mixed mode startup code calls into startup_32() in the legacy decompressor, passing a mocked up boot_params struct, only to get it to set up the 1:1 mapping of the lower 4 GiB of memory and switch to a GDT that supports 64-bit mode.

In order to be able to reuse the EFI mixed mode startup code in EFI boot images that do not incorporate the legacy decompressor code, decouple it, by populating the GDT directly.

Doing so allows constructing a GDT that is compatible with the one the firmware uses, with one additional entry for a 64-bit mode code segment appended. This removes the need entirely to switch between GDTs and IDTs or data segment selector values on every call into the firmware, and all of this code can be removed.

---
 arch/x86/boot/compressed/efi_mixed.S | 227 ++++++--------------
 arch/x86/boot/compressed/head_64.S   |   7 -
 2 files changed, 65 insertions(+), 169 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +65 -162
index dca916c3e6f0..984956931ed7 100644
@@ -15,70 +15,23 @@
  */
 
 #include <linux/linkage.h>
-#include <asm/asm-offsets.h>
+#include <asm/desc_defs.h>
 #include <asm/msr.h>
 #include <asm/page_types.h>
 #include <asm/pgtable_types.h>
 #include <asm/processor-flags.h>
 #include <asm/segment.h>
-#include <asm/setup.h>
 
 	.code64
 	.text
-/*
- * When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode()
- * is the first thing that runs after switching to long mode. Depending on
- * whether the EFI handover protocol or the compat entry point was used to
- * enter the kernel, it will either branch to the common 64-bit EFI stub
- * entrypoint efi_stub_entry() directly, or via the 64-bit EFI PE/COFF
- * entrypoint efi_pe_entry(). In the former case, the bootloader must provide a
- * struct bootparams pointer as the third argument, so the presence of such a
- * pointer is used to disambiguate.
- *
- *                                                             +--------------+
- *  +------------------+     +------------+            +------>| efi_pe_entry |
- *  | efi32_pe_entry   |---->|            |            |       +-----------+--+
- *  +------------------+     |            |     +------+----------------+  |
- *                           | startup_32 |---->| startup_64_mixed_mode |  |
- *  +------------------+     |            |     +------+----------------+  |
- *  | efi32_stub_entry |---->|            |            |                   |
- *  +------------------+     +------------+            |                   |
- *                                                     V                   |
- *                           +------------+     +----------------+         |
- *                           | startup_64 |<----| efi_stub_entry |<--------+
- *                           +------------+     +----------------+
- */
-SYM_FUNC_START(startup_64_mixed_mode)
-	lea	efi32_boot_args(%rip), %rdx
-	mov	0(%rdx), %edi
-	mov	4(%rdx), %esi
-
-	leaq	(pte + 5 * PAGE_SIZE)(%rip), %rax
-	movq	%rax, %cr3		// reload after startup_32
-
-	/* Switch to the firmware's stack */
-	movl	efi32_boot_sp(%rip), %esp
-	andl	$~7, %esp
-
-	mov	8(%rdx), %edx		// saved bootparams pointer
-	call	efi_stub_entry
-SYM_FUNC_END(startup_64_mixed_mode)
-
 SYM_FUNC_START(__efi64_thunk)
 	push	%rbp
 	push	%rbx
 
-	movl	%ds, %eax
-	push	%rax
-	movl	%es, %eax
-	push	%rax
-	movl	%ss, %eax
-	push	%rax
-
 	/* Copy args passed on stack */
-	movq	0x30(%rsp), %rbp
-	movq	0x38(%rsp), %rbx
-	movq	0x40(%rsp), %rax
+	movq	0x18(%rsp), %rbp
+	movq	0x20(%rsp), %rbx
+	movq	0x28(%rsp), %rax
 
 	/*
 	 * Convert x86-64 ABI params to i386 ABI
@@ -93,44 +46,14 @@ SYM_FUNC_START(__efi64_thunk)
 	movl	%ebx, 0x18(%rsp)
 	movl	%eax, 0x1c(%rsp)
 
-	leaq	0x20(%rsp), %rbx
-	sgdt	(%rbx)
-	sidt	16(%rbx)
-
 	leaq	1f(%rip), %rbp
+	movl	%cs, %ebx
 
-	/*
-	 * Switch to IDT and GDT with 32-bit segments. These are the firmware
-	 * GDT and IDT that were installed when the kernel started executing.
-	 * The pointers were saved by the efi32_entry() routine below.
-	 *
-	 * Pass the saved DS selector to the 32-bit code, and use far return to
-	 * restore the saved CS selector.
-	 */
-	lidt	efi32_boot_idt(%rip)
-	lgdt	efi32_boot_gdt(%rip)
-
-	movzwl	efi32_boot_ds(%rip), %edx
-	movzwq	efi32_boot_cs(%rip), %rax
-	pushq	%rax
-	leaq	efi_enter32(%rip), %rax
-	pushq	%rax
-	lretq
+	ljmpl	*efi32_call(%rip)
 
 1:	addq	$64, %rsp
 	movq	%rdi, %rax
 
-	pop	%rbx
-	movl	%ebx, %ss
-	pop	%rbx
-	movl	%ebx, %es
-	pop	%rbx
-	movl	%ebx, %ds
-	/* Clear out 32-bit selector from FS and GS */
-	xorl	%ebx, %ebx
-	movl	%ebx, %fs
-	movl	%ebx, %gs
-
 	pop	%rbx
 	pop	%rbp
 	RET
@@ -141,7 +64,6 @@ SYM_FUNC_END(__efi64_thunk)
 SYM_FUNC_START(efi32_stub_entry)
 	call	1f
 1:	popl	%ecx
-	leal	(efi32_boot_args - 1b)(%ecx), %ebx
 
 	/* Clear BSS */
 	xorl	%eax, %eax
@@ -153,11 +75,8 @@ SYM_FUNC_START(efi32_stub_entry)
 	rep	stosl
 
 	add	$0x4, %esp		/* Discard return address */
-	popl	%ecx
-	popl	%edx
-	popl	%esi
-	movl	%esi, 8(%ebx)
-	jmp	efi32_entry
+	movl	8(%esp), %ebx		/* struct boot_params pointer */
+	jmp	efi32_startup
 SYM_FUNC_END(efi32_stub_entry)
 #endif
 
@@ -167,13 +86,6 @@ SYM_FUNC_END(efi32_stub_entry)
  * The stack should represent the 32-bit calling convention.
  */
 SYM_FUNC_START_LOCAL(efi_enter32)
-	/* Load firmware selector into data and stack segment registers */
-	movl	%edx, %ds
-	movl	%edx, %es
-	movl	%edx, %fs
-	movl	%edx, %gs
-	movl	%edx, %ss
-
 	/* Disable paging */
 	movl	%cr0, %eax
 	btrl	$X86_CR0_PG_BIT, %eax
@@ -190,21 +102,9 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 	/* We must preserve return value */
 	movl	%eax, %edi
 
-	/*
-	 * Some firmware will return with interrupts enabled. Be sure to
-	 * disable them before we switch GDTs and IDTs.
-	 */
-	cli
-
-	lidtl	16(%ebx)
-	lgdtl	(%ebx)
-
-	xorl	%eax, %eax
-	lldt	%ax
-
 	call	efi32_enable_long_mode
 
-	pushl	$__KERNEL_CS
+	pushl	%ebx
 	pushl	%ebp
 	lret
 SYM_FUNC_END(efi_enter32)
@@ -230,50 +130,56 @@ SYM_FUNC_START_LOCAL(efi32_enable_long_mode)
 SYM_FUNC_END(efi32_enable_long_mode)
 
 /*
- * This is the common EFI stub entry point for mixed mode.
+ * This is the common EFI stub entry point for mixed mode. It sets up the GDT
+ * and page tables needed for 64-bit execution, after which it calls the
+ * common 64-bit EFI entrypoint efi_stub_entry().
  *
- * Arguments:	%ecx	image handle
- * 		%edx	EFI system table pointer
+ * Arguments:	0(%esp)	image handle
+ * 		4(%esp)	EFI system table pointer
+ *		%ebx	struct boot_params pointer (or NULL)
  *
  * Since this is the point of no return for ordinary execution, no registers
  * are considered live except for the function parameters. [Note that the EFI
  * stub may still exit and return to the firmware using the Exit() EFI boot
  * service.]
  */
-SYM_FUNC_START_LOCAL(efi32_entry)
-	call	1f
-1:	pop	%ebx
-
-	/* Save firmware GDTR and code/data selectors */
-	sgdtl	(efi32_boot_gdt - 1b)(%ebx)
-	movw	%cs, (efi32_boot_cs - 1b)(%ebx)
-	movw	%ds, (efi32_boot_ds - 1b)(%ebx)
-
-	/* Store firmware IDT descriptor */
-	sidtl	(efi32_boot_idt - 1b)(%ebx)
-
-	/* Store firmware stack pointer */
-	movl	%esp, (efi32_boot_sp - 1b)(%ebx)
-
-	/* Store boot arguments */
-	leal	(efi32_boot_args - 1b)(%ebx), %ebx
-	movl	%ecx, 0(%ebx)
-	movl	%edx, 4(%ebx)
-	movb	$0x0, 12(%ebx)          // efi_is64
-
-	/*
-	 * Allocate some memory for a temporary struct boot_params, which only
-	 * needs the minimal pieces that startup_32() relies on.
-	 */
-	subl	$PARAM_SIZE, %esp
-	movl	%esp, %esi
-	movl	$PAGE_SIZE, BP_kernel_alignment(%esi)
-	movl	$_end - 1b, BP_init_size(%esi)
-	subl	$startup_32 - 1b, BP_init_size(%esi)
+SYM_FUNC_START_LOCAL(efi32_startup)
+	movl	%esp, %ebp
+
+	subl	$8, %esp
+	sgdtl	(%esp)			/* Save GDT descriptor to the stack */
+	movl	2(%esp), %esi		/* Existing GDT pointer */
+	movzwl	(%esp), %ecx		/* Existing GDT limit */
+	inc	%ecx			/* Existing GDT size */
+	andl	$~7, %ecx		/* Ensure size is multiple of 8 */
+
+	subl	%ecx, %esp		/* Allocate new GDT */
+	andl	$~15, %esp		/* Realign the stack */
+	movl	%esp, %edi		/* New GDT address */
+	leal	7(%ecx), %eax		/* New GDT limit */
+	pushw	%cx			/* Push 64-bit CS (for LJMP below) */
+	pushl	%edi			/* Push new GDT address */
+	pushw	%ax			/* Push new GDT limit */
+
+	/* Copy GDT to the stack and add a 64-bit code segment at the end */
+	movl	$GDT_ENTRY(DESC_CODE64, 0, 0xfffff) & 0xffffffff, (%edi,%ecx)
+	movl	$GDT_ENTRY(DESC_CODE64, 0, 0xfffff) >> 32, 4(%edi,%ecx)
+	shrl	$2, %ecx
+	cld
+	rep	movsl			/* Copy the firmware GDT */
+	lgdtl	(%esp)			/* Switch to the new GDT */
 
 	call	1f
 1:	pop	%edi
 
+	/* Record mixed mode entry */
+	movb	$0x0, (efi_is64 - 1b)(%edi)
+
+	/* Set up indirect far call to re-enter 32-bit mode */
+	leal	(efi32_call - 1b)(%edi), %eax
+	addl	%eax, (%eax)
+	movw	%cs, 4(%eax)
+
 	/* Disable paging */
 	movl	%cr0, %eax
 	btrl	$X86_CR0_PG_BIT, %eax
@@ -297,8 +203,17 @@ SYM_FUNC_START_LOCAL(efi32_entry)
 	movl	%edx, (%eax)
 	movl	%eax, %cr3
 
-	jmp	startup_32
-SYM_FUNC_END(efi32_entry)
+	call	efi32_enable_long_mode
+
+	/* Set up far jump to 64-bit mode (CS is already on the stack) */
+	leal	(efi_stub_entry - 1b)(%edi), %eax
+	movl	%eax, 2(%esp)
+
+	movl	0(%ebp), %edi
+	movl	4(%ebp), %esi
+	movl	%ebx, %edx
+	ljmpl	*2(%esp)
+SYM_FUNC_END(efi32_startup)
 
 /*
  * efi_status_t efi32_pe_entry(efi_handle_t image_handle,
@@ -313,10 +228,8 @@ SYM_FUNC_START(efi32_pe_entry)
 	btl	$29, %edx			// check long mode bit
 	jnc	1f
 	leal	8(%esp), %esp			// preserve stack alignment
-	movl	(%esp), %ecx			// image_handle
-	movl	4(%esp), %edx			// sys_table
-	jmp	efi32_entry			// pass %ecx, %edx
-						// no other registers remain live
+	xor	%ebx, %ebx			// no struct boot_params pointer
+	jmp	efi32_startup			// only ESP and EBX remain live
 1:	movl	$0x80000003, %eax		// EFI_UNSUPPORTED
 	popl	%ebx
 	RET
@@ -332,20 +245,10 @@ SYM_FUNC_END(efi64_stub_entry)
 
 	.data
 	.balign	8
-SYM_DATA_START_LOCAL(efi32_boot_gdt)
-	.word	0
-	.quad	0
-SYM_DATA_END(efi32_boot_gdt)
-
-SYM_DATA_START_LOCAL(efi32_boot_idt)
-	.word	0
-	.quad	0
-SYM_DATA_END(efi32_boot_idt)
-
-SYM_DATA_LOCAL(efi32_boot_cs, .word 0)
-SYM_DATA_LOCAL(efi32_boot_ds, .word 0)
-SYM_DATA_LOCAL(efi32_boot_sp, .long 0)
-SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0)
+SYM_DATA_START_LOCAL(efi32_call)
+	.long	efi_enter32 - .
+	.word	0x0
+SYM_DATA_END(efi32_call)
 SYM_DATA(efi_is64, .byte 1)
 
 	.bss
arch/x86/boot/compressed/head_64.S +0 -7
index 1dcb794c5479..5db6495a3bb9 100644
@@ -263,13 +263,6 @@ SYM_FUNC_START(startup_32)
 	 * used to perform that far jump.
 	 */
 	leal	rva(startup_64)(%ebp), %eax
-#ifdef CONFIG_EFI_MIXED
-	cmpb	$1, rva(efi_is64)(%ebp)
-	je	1f
-	leal	rva(startup_64_mixed_mode)(%ebp), %eax
-1:
-#endif
-
 	pushl	$__KERNEL_CS
 	pushl	%eax
 
Signature · 1 line
2.48.1.362.g079036d154-goog
#6 lore raw

PATCH v2 6/7 x86/efi/mixed: Simplify and document thunking logic

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Now that the GDT/IDT and data segment selector preserve/restore logic has been removed from the boot-time EFI mixed mode thunking routines, the remaining logic to handle the function arguments can be simplified:
the setup of the arguments on the stack can be moved into the 32-bit callee, which is able to use a more idiomatic sequence of PUSH instructions.

This, in turn, allows the far call and far return to be issued using plain LCALL and LRET instructions, removing the need to set up the return explicitly.

---
 arch/x86/boot/compressed/efi_mixed.S | 77 ++++++++++----------
 1 file changed, 37 insertions(+), 40 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +37 -40
index 984956931ed7..e04ed99bc449 100644
@@ -22,43 +22,7 @@
 #include <asm/processor-flags.h>
 #include <asm/segment.h>
 
-	.code64
 	.text
-SYM_FUNC_START(__efi64_thunk)
-	push	%rbp
-	push	%rbx
-
-	/* Copy args passed on stack */
-	movq	0x18(%rsp), %rbp
-	movq	0x20(%rsp), %rbx
-	movq	0x28(%rsp), %rax
-
-	/*
-	 * Convert x86-64 ABI params to i386 ABI
-	 */
-	subq	$64, %rsp
-	movl	%esi, 0x0(%rsp)
-	movl	%edx, 0x4(%rsp)
-	movl	%ecx, 0x8(%rsp)
-	movl	%r8d, 0xc(%rsp)
-	movl	%r9d, 0x10(%rsp)
-	movl	%ebp, 0x14(%rsp)
-	movl	%ebx, 0x18(%rsp)
-	movl	%eax, 0x1c(%rsp)
-
-	leaq	1f(%rip), %rbp
-	movl	%cs, %ebx
-
-	ljmpl	*efi32_call(%rip)
-
-1:	addq	$64, %rsp
-	movq	%rdi, %rax
-
-	pop	%rbx
-	pop	%rbp
-	RET
-SYM_FUNC_END(__efi64_thunk)
-
 	.code32
 #ifdef CONFIG_EFI_HANDOVER_PROTOCOL
 SYM_FUNC_START(efi32_stub_entry)
@@ -81,11 +45,26 @@ SYM_FUNC_END(efi32_stub_entry)
 #endif
 
 /*
- * EFI service pointer must be in %edi.
+ * Called using a far call from __efi64_thunk() below, using the x86_64 SysV
+ * ABI (except for R8/R9 which are inaccessible to 32-bit code - EAX/EBX are
+ * used instead).  EBP+16 points to the arguments passed via the stack.
  *
- * The stack should represent the 32-bit calling convention.
+ * The first argument (EDI) is a pointer to the boot service or protocol, to
+ * which the remaining arguments are passed, each truncated to 32 bits.
  */
 SYM_FUNC_START_LOCAL(efi_enter32)
+	/*
+	 * Convert x86-64 SysV ABI params to i386 ABI
+	 */
+	pushl	32(%ebp)	/* Up to 3 args passed via the stack */
+	pushl	24(%ebp)
+	pushl	16(%ebp)
+	pushl	%ebx		/* R9 */
+	pushl	%eax		/* R8 */
+	pushl	%ecx
+	pushl	%edx
+	pushl	%esi
+
 	/* Disable paging */
 	movl	%cr0, %eax
 	btrl	$X86_CR0_PG_BIT, %eax
@@ -104,11 +83,29 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 
 	call	efi32_enable_long_mode
 
-	pushl	%ebx
-	pushl	%ebp
+	addl	$32, %esp
+	movl	%edi, %eax
 	lret
 SYM_FUNC_END(efi_enter32)
 
+	.code64
+SYM_FUNC_START(__efi64_thunk)
+	push	%rbp
+	movl	%esp, %ebp
+	push	%rbx
+
+	/* Move args #5 and #6 into 32-bit accessible registers */
+	movl	%r8d, %eax
+	movl	%r9d, %ebx
+
+	lcalll	*efi32_call(%rip)
+
+	pop	%rbx
+	pop	%rbp
+	RET
+SYM_FUNC_END(__efi64_thunk)
+
+	.code32
 SYM_FUNC_START_LOCAL(efi32_enable_long_mode)
 	movl	%cr4, %eax
 	btsl	$(X86_CR4_PAE_BIT), %eax
Signature · 1 line
2.48.1.362.g079036d154-goog
#7 lore raw

PATCH v2 7/7 x86/efi/mixed: Move mixed mode startup code into libstub

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

The EFI mixed mode code has been decoupled from the legacy decompressor, in order to be able to reuse it with generic EFI zboot images for x86.

Move the source file into the libstub source directory to facilitate this.

---
 arch/x86/boot/compressed/Makefile                                                | 1 -
 drivers/firmware/efi/libstub/Makefile                                            | 3 +++
 arch/x86/boot/compressed/efi_mixed.S => drivers/firmware/efi/libstub/x86-mixed.S | 0
 3 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/boot/compressed/Makefile +0 -1
index f2051644de94..fc5563704466 100644
@@ -104,7 +104,6 @@ vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o $(obj)/tdcall.o $(obj)/td
 vmlinux-objs-$(CONFIG_UNACCEPTED_MEMORY) += $(obj)/mem.o
 
 vmlinux-objs-$(CONFIG_EFI) += $(obj)/efi.o
-vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_mixed.o
 vmlinux-libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
 
 $(obj)/vmlinux: $(vmlinux-objs-y) $(vmlinux-libs-y) FORCE
drivers/firmware/efi/libstub/Makefile +3 -0
index 1141cd06011f..903afd2d3d58 100644
@@ -62,6 +62,8 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
 # `-fdata-sections` flag from KBUILD_CFLAGS_KERNEL
 KBUILD_CFLAGS_KERNEL := $(filter-out -fdata-sections, $(KBUILD_CFLAGS_KERNEL))
 
+KBUILD_AFLAGS			:= $(KBUILD_CFLAGS) -D__ASSEMBLY__
+
 lib-y				:= efi-stub-helper.o gop.o secureboot.o tpm.o \
 				   file.o mem.o random.o randomalloc.o pci.o \
 				   skip_spaces.o lib-cmdline.o lib-ctype.o \
@@ -83,6 +85,7 @@ lib-$(CONFIG_EFI_GENERIC_STUB)	+= efi-stub.o string.o intrinsics.o systable.o \
 lib-$(CONFIG_ARM)		+= arm32-stub.o
 lib-$(CONFIG_ARM64)		+= kaslr.o arm64.o arm64-stub.o smbios.o
 lib-$(CONFIG_X86)		+= x86-stub.o smbios.o
+lib-$(CONFIG_EFI_MIXED)		+= x86-mixed.o
 lib-$(CONFIG_X86_64)		+= x86-5lvl.o
 lib-$(CONFIG_RISCV)		+= kaslr.o riscv.o riscv-stub.o
 lib-$(CONFIG_LOONGARCH)		+= loongarch.o loongarch-stub.o
drivers/firmware/efi/libstub/x86-mixed.S renamed from arch/x86/boot/compressed/efi_mixed.S +0 -0
similarity index 100%
rename from arch/x86/boot/compressed/efi_mixed.S
rename to drivers/firmware/efi/libstub/x86-mixed.S
Signature · 1 line
2.48.1.362.g079036d154-goog
maintainer
#8 lore raw
In reply to Ard Biesheuvel
Quoted 40 lines · Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Since commit

a1b87d54f4e4 ("x86/efistub: Avoid legacy decompressor when doing EFI boot")

booting via the EFI stub no longer relies on the legacy decompressor, and instead, the kernel proper is decompressed by code executing in the context of the EFI boot services, and subsequently invoked directly.

The only remaining dependency is the EFI mixed mode startup code, which makes a detour via the legacy decompressor's 32-bit entrypoint, in order to obtain a 1:1 mapping of memory, which is a prerequisite for 64-bit execution on x86.

This detour requires some fiddly setup on the part of the mixed mode startup code, which has to stash the firmware stack pointer and boot arguments in memory, and create a fake struct boot_params to trick the code in startup_32 to behave as intended.

This dependency also impedes reuse of the EFI stub code in other contexts, such as generic EFI zboot, which will reuse the EFI stub but not the legacy decompressor.

So remove this dependency, by replacing this detour with a minimal reimplementation of the 1:1 mapping code. With some further cleanup applied on top, the line count drops substantially, but without loss of functionality. The resulting code can operate independently from the legacy decompressor, and is therefore moved out of arch/x86/boot/ and into the EFI libstub/ directory.

Changes since v1 [0]:
- Create new long mode GDT that extends the firmware's 32-bit only GDT so that preserving/restoring data segment selectors or swapping out GDTs and IDTs is no longer needed at all.
- Rebase onto v6.14-rc1

[0] https://lore.kernel.org/all/[email protected]/

Quoted 9 lines

Ard Biesheuvel (7):
x86/efistub: Merge PE and handover entrypoints x86/efi/mixed: Check CPU compatibility without relying on verify_cpu() x86/efi/mixed: Factor out and clean up long mode entry x86/efi/mixed: Set up 1:1 mapping of lower 4GiB in the stub x86/efi/mixed: Remove dependency on legacy startup_32 code x86/efi/mixed: Simplify and document thunking logic x86/efi/mixed: Move mixed mode startup code into libstub

Unless anyone minds, I'd like to queue this up in the EFI tree.

Boris, Ingo?

maintainer
#9 lore raw
Quoted 3 lines · Ard Biesheuvel

Unless anyone minds, I'd like to queue this up in the EFI tree.

Boris, Ingo?

FWIW, it looks like a nice cleanup to me and it boots in my 64-bit OVMF guest but that doesn't mean a whole lot.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
maintainer
#10 lore raw
Quoted 9 lines · Borislav Petkov
Quoted 3 lines · Ard Biesheuvel

Unless anyone minds, I'd like to queue this up in the EFI tree.

Boris, Ingo?

FWIW, it looks like a nice cleanup to me and it boots in my 64-bit OVMF guest but that doesn't mean a whole lot.

Thanks. For the record, I tested this both on 32-bit OVMF and on a mixed mode tablet (with 32-bit AMI firmware) that I keep for testing purposes. Notably, 32-bit OVMF boots with paging (and PAE) enabled whereas the AMI firmware doesn't.

Not sure how many users of mixed mode remain and how invested they are in upgrading to newer kernels, but decoupling it from the legacy decompressor does help the cleanup work I am doing in that area.

#11 lore raw

Hi Ard,

Quoted 7 lines · Ard Biesheuvel
Quoted 6 lines · Borislav Petkov
Quoted 3 lines · Ard Biesheuvel

Unless anyone minds, I'd like to queue this up in the EFI tree.

Boris, Ingo?

Quoted 8 lines
Quoted 3 lines

FWIW, it looks like a nice cleanup to me and it boots in my 64-bit OVMF guest but that doesn't mean a whole lot.

Thanks. For the record, I tested this both on 32-bit OVMF and on a mixed mode tablet (with 32-bit AMI firmware) that I keep for testing purposes. Notably, 32-bit OVMF boots with paging (and PAE) enabled whereas the AMI firmware doesn't.

Ah good to know that you're still using the mixed-mode tablet for testing.

I've just added this series to my personal kernel testing-tree which I regularly boot on various models of these kinda tablets. I'll let you know if I hit any problems.

Quoted 1 line

Not sure how many users of mixed mode remain

I think the 32 bit mixed-mode Intel Bay Trail tablets are still somewhat popular I still get occasional inquiries about when we will finally have the cameras working on them :)

Regards,

Hans

Acked-by
#13 lore raw
In reply to Ard Biesheuvel
Quoted 51 lines · Ard Biesheuvel
Quoted 40 lines · Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Since commit

a1b87d54f4e4 ("x86/efistub: Avoid legacy decompressor when doing EFI boot")

booting via the EFI stub no longer relies on the legacy decompressor, and instead, the kernel proper is decompressed by code executing in the context of the EFI boot services, and subsequently invoked directly.

The only remaining dependency is the EFI mixed mode startup code, which makes a detour via the legacy decompressor's 32-bit entrypoint, in order to obtain a 1:1 mapping of memory, which is a prerequisite for 64-bit execution on x86.

This detour requires some fiddly setup on the part of the mixed mode startup code, which has to stash the firmware stack pointer and boot arguments in memory, and create a fake struct boot_params to trick the code in startup_32 to behave as intended.

This dependency also impedes reuse of the EFI stub code in other contexts, such as generic EFI zboot, which will reuse the EFI stub but not the legacy decompressor.

So remove this dependency, by replacing this detour with a minimal reimplementation of the 1:1 mapping code. With some further cleanup applied on top, the line count drops substantially, but without loss of functionality. The resulting code can operate independently from the legacy decompressor, and is therefore moved out of arch/x86/boot/ and into the EFI libstub/ directory.

Changes since v1 [0]:
- Create new long mode GDT that extends the firmware's 32-bit only GDT so that preserving/restoring data segment selectors or swapping out GDTs and IDTs is no longer needed at all.
- Rebase onto v6.14-rc1

[0] https://lore.kernel.org/all/[email protected]/

Quoted 9 lines

Ard Biesheuvel (7):
x86/efistub: Merge PE and handover entrypoints x86/efi/mixed: Check CPU compatibility without relying on verify_cpu() x86/efi/mixed: Factor out and clean up long mode entry x86/efi/mixed: Set up 1:1 mapping of lower 4GiB in the stub x86/efi/mixed: Remove dependency on legacy startup_32 code x86/efi/mixed: Simplify and document thunking logic x86/efi/mixed: Move mixed mode startup code into libstub

Quoted 3 lines

Unless anyone minds, I'd like to queue this up in the EFI tree.

Boris, Ingo?

Sure!

Thanks,

Ingo

maintainer
#12 lore raw
In reply to Hans de Goede
Quoted 15 lines · Hans de Goede

Hi Ard,

Quoted 11 lines · Ard Biesheuvel
Quoted 9 lines · Borislav Petkov
Quoted 3 lines · Ard Biesheuvel

Unless anyone minds, I'd like to queue this up in the EFI tree.

Boris, Ingo?

FWIW, it looks like a nice cleanup to me and it boots in my 64-bit OVMF guest but that doesn't mean a whole lot.

Quoted 8 lines
Quoted 4 lines

Thanks. For the record, I tested this both on 32-bit OVMF and on a mixed mode tablet (with 32-bit AMI firmware) that I keep for testing purposes. Notably, 32-bit OVMF boots with paging (and PAE) enabled whereas the AMI firmware doesn't.

Ah good to know that you're still using the mixed-mode tablet for testing.

Yes, thanks again for sending me that thing - it has been quite useful. I bricked my mixed mode iMac24, but that thing is just too old now anyway.

Quoted 4 lines

I've just added this series to my personal kernel testing-tree which I regularly boot on various models of these kinda tablets. I'll let you know if I hit any problems.

Thanks. These changes are in -next now, so that give some more coverage until the next merge window.