Skip to contentSkip to conversation

x86: head_64.S spring cleaning

PATCH Applied Ard Biesheuvel 5 participants 50 messages Nov 22, 2022 – Nov 24, 2022

Subsystems (2)
lore
Share & tools
lore

Ard Biesheuvel

author maintainer
#0 lore raw

v3 cover x86: head_64.S spring cleaning

After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.

Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/master

Changes since v1:
- at Boris's request, split the patches into smaller ones that are easier to review

Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Michael Roth <[email protected]>

Ard Biesheuvel (17):
x86/compressed: efi-mixed: rename efi_thunk_64.S to efi-mixed.S x86/compressed: efi-mixed: move 32-bit entrypoint code into .text
section
x86/compressed: efi-mixed: move bootargs parsing out of 32-bit startup
code
x86/compressed: efi-mixed: move efi32_pe_entry into .text section x86/compressed: efi-mixed: move efi32_entry out of head_64.S x86/compressed: efi-mixed: move efi32_pe_entry() out of head_64.S x86/compressed: efi: merge multiple definitions of image_offset into
one
x86/compressed: efi-mixed: simplify IDT/GDT preserve/restore x86/compressed: avoid touching ECX in startup32_set_idt_entry() x86/compressed: pull global variable ref up into startup32_load_idt() x86/compressed: move startup32_load_idt() into .text section x86/compressed: move startup32_load_idt() out of head_64.S x86/compressed: move startup32_check_sev_cbit() into .text x86/compressed: move startup32_check_sev_cbit() out of head_64.S x86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y
efi: x86: Make the deprecated EFI handover protocol optional

 arch/x86/Kconfig                        |  17 +
 arch/x86/boot/compressed/Makefile       |   8 +-
 arch/x86/boot/compressed/efi_mixed.S    | 351 ++++++++++++++++++++
 arch/x86/boot/compressed/efi_thunk_64.S | 195 -----------
 arch/x86/boot/compressed/head_32.S      |   4 -
 arch/x86/boot/compressed/head_64.S      | 303 +----------------
 arch/x86/boot/compressed/mem_encrypt.S  | 152 ++++++++-
 arch/x86/boot/header.S                  |   2 +-
 arch/x86/boot/tools/build.c             |   2 +
 drivers/firmware/efi/libstub/x86-stub.c |   2 +-
 10 files changed, 533 insertions(+), 503 deletions(-)
 create mode 100644 arch/x86/boot/compressed/efi_mixed.S
 delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S
Signature · 1 line
2.35.1
author maintainer
#1 lore raw

PATCH v3 1/17 x86/compressed: efi-mixed: rename efi_thunk_64.S to efi-mixed.S

In preparation for moving the mixed mode specific code out of head_64.S, rename the existing file to clarify that it contains more than just the mixed mode thunk.

While at it, clean up the Makefile rules that add it to the build.

---
 arch/x86/boot/compressed/Makefile                        | 6 +++---
 arch/x86/boot/compressed/{efi_thunk_64.S => efi_mixed.S} | 0
 2 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/boot/compressed/Makefile +3 -3
index 3a261abb6d158d62..cd1a6295b957ea1d 100644
@@ -108,11 +108,11 @@ endif
 vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o
 vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o $(obj)/tdcall.o
 
-vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o
 vmlinux-objs-$(CONFIG_EFI) += $(obj)/efi.o
-efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.a
+vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_mixed.o
+vmlinux-objs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
 
-$(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE
+$(obj)/vmlinux: $(vmlinux-objs-y) FORCE
 	$(call if_changed,ld)
 
 OBJCOPYFLAGS_vmlinux.bin :=  -R .comment -S
arch/x86/boot/compressed/efi_mixed.S renamed from arch/x86/boot/compressed/efi_thunk_64.S +0 -0
similarity index 100%
rename from arch/x86/boot/compressed/efi_thunk_64.S
rename to arch/x86/boot/compressed/efi_mixed.S
Signature · 1 line
2.35.1
merge bot Applied
#2 lore raw

[tip: x86/boot] x86/boot/compressed: Rename efi_thunk_64.S to efi-mixed.S

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     cb8bda8ad4438b4bcfcf89697fc84803fb210017
Gitweb:        https://git.kernel.org/tip/cb8bda8ad4438b4bcfcf89697fc84803fb210017
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:01 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Tue, 22 Nov 2022 18:54:44 +01:00

x86/boot/compressed: Rename efi_thunk_64.S to efi-mixed.S

In preparation for moving the mixed mode specific code out of head_64.S, rename the existing file to clarify that it contains more than just the mixed mode thunk.

While at it, clean up the Makefile rules that add it to the build.

---
 arch/x86/boot/compressed/Makefile       |   6 +-
 arch/x86/boot/compressed/efi_mixed.S    | 195 +++++++++++++++++++++++-
 arch/x86/boot/compressed/efi_thunk_64.S | 195 +-----------------------
 3 files changed, 198 insertions(+), 198 deletions(-)
 create mode 100644 arch/x86/boot/compressed/efi_mixed.S
 delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S
arch/x86/boot/compressed/Makefile +3 -3
index 3a261ab..cd1a629 100644
@@ -108,11 +108,11 @@ endif
 vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o
 vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o $(obj)/tdcall.o
 
-vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o
 vmlinux-objs-$(CONFIG_EFI) += $(obj)/efi.o
-efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.a
+vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_mixed.o
+vmlinux-objs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
 
-$(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE
+$(obj)/vmlinux: $(vmlinux-objs-y) FORCE
 	$(call if_changed,ld)
 
 OBJCOPYFLAGS_vmlinux.bin :=  -R .comment -S
arch/x86/boot/compressed/efi_mixed.S new file +195 -0

195 lines not on this page · raw message

arch/x86/boot/compressed/efi_thunk_64.S deleted +0 -195

195 lines not on this page · raw message

author maintainer
#3 lore raw

PATCH v3 2/17 x86/compressed: efi-mixed: move 32-bit entrypoint code into .text section

In reply to Ard Biesheuvel

Move the code that stores the arguments passed to the EFI entrypoint into the .text section, so that it can be moved into a separate compilation unit in a subsequent patch.

---
 arch/x86/boot/compressed/head_64.S | 48 ++++++++++++++------
 1 file changed, 34 insertions(+), 14 deletions(-)
arch/x86/boot/compressed/head_64.S +34 -14
index 190b803eb78712e0..74aaffd22b7425f4 100644
@@ -311,24 +311,41 @@ SYM_FUNC_START(efi32_stub_entry)
 	popl	%ecx
 	popl	%edx
 	popl	%esi
+	jmp	efi32_entry
+SYM_FUNC_END(efi32_stub_entry)
 
+	.text
+/*
+ * This is the common EFI stub entry point for mixed mode.
+ *
+ * Arguments:	%ecx	image handle
+ * 		%edx	EFI system table pointer
+ *		%esi	struct bootparams pointer (or NULL when not using
+ *			the EFI handover protocol)
+ *
+ * Since this is the point of no return for ordinary execution, no registers
+ * are considered live except for the function parameters. [Note that the EFI
+ * stub may still exit and return to the firmware using the Exit() EFI boot
+ * service.]
+ */
+SYM_FUNC_START_LOCAL(efi32_entry)
 	call	1f
-1:	pop	%ebp
-	subl	$ rva(1b), %ebp
-
-	movl	%esi, rva(efi32_boot_args+8)(%ebp)
-SYM_INNER_LABEL(efi32_pe_stub_entry, SYM_L_LOCAL)
-	movl	%ecx, rva(efi32_boot_args)(%ebp)
-	movl	%edx, rva(efi32_boot_args+4)(%ebp)
-	movb	$0, rva(efi_is64)(%ebp)
+1:	pop	%ebx
 
 	/* Save firmware GDTR and code/data selectors */
-	sgdtl	rva(efi32_boot_gdt)(%ebp)
-	movw	%cs, rva(efi32_boot_cs)(%ebp)
-	movw	%ds, rva(efi32_boot_ds)(%ebp)
+	sgdtl	(efi32_boot_gdt - 1b)(%ebx)
+	movw	%cs, (efi32_boot_cs - 1b)(%ebx)
+	movw	%ds, (efi32_boot_ds - 1b)(%ebx)
 
 	/* Store firmware IDT descriptor */
-	sidtl	rva(efi32_boot_idt)(%ebp)
+	sidtl	(efi32_boot_idt - 1b)(%ebx)
+
+	/* Store boot arguments */
+	leal	(efi32_boot_args - 1b)(%ebx), %ebx
+	movl	%ecx, 0(%ebx)
+	movl	%edx, 4(%ebx)
+	movl	%esi, 8(%ebx)
+	movb	$0x0, 12(%ebx)          // efi_is64
 
 	/* Disable paging */
 	movl	%cr0, %eax
@@ -336,7 +353,8 @@ SYM_INNER_LABEL(efi32_pe_stub_entry, SYM_L_LOCAL)
 	movl	%eax, %cr0
 
 	jmp	startup_32
-SYM_FUNC_END(efi32_stub_entry)
+SYM_FUNC_END(efi32_entry)
+	__HEAD
 #endif
 
 	.code64
@@ -839,7 +857,9 @@ SYM_FUNC_START(efi32_pe_entry)
 	 */
 	subl	%esi, %ebx
 	movl	%ebx, rva(image_offset)(%ebp)	// save image_offset
-	jmp	efi32_pe_stub_entry
+	xorl	%esi, %esi
+	jmp	efi32_entry			// pass %ecx, %edx, %esi
+						// no other registers remain live
 
 2:	popl	%edi				// restore callee-save registers
 	popl	%ebx
Signature · 1 line
2.35.1
merge bot Applied
#4 lore raw

[tip: x86/boot] x86/boot/compressed: Move 32-bit entrypoint code into .text section

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     e2ab9eab324cdf240de89741e4a1aa79919f0196
Gitweb:        https://git.kernel.org/tip/e2ab9eab324cdf240de89741e4a1aa79919f0196
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:02 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Tue, 22 Nov 2022 19:03:56 +01:00

x86/boot/compressed: Move 32-bit entrypoint code into .text section

Move the code that stores the arguments passed to the EFI entrypoint into the .text section, so that it can be moved into a separate compilation unit in a subsequent patch.

---
 arch/x86/boot/compressed/head_64.S | 48 ++++++++++++++++++++---------
 1 file changed, 34 insertions(+), 14 deletions(-)
arch/x86/boot/compressed/head_64.S +34 -14
index d33f060..5d07ef7 100644
@@ -303,24 +303,41 @@ SYM_FUNC_START(efi32_stub_entry)
 	popl	%ecx
 	popl	%edx
 	popl	%esi
+	jmp	efi32_entry
+SYM_FUNC_END(efi32_stub_entry)
 
+	.text
+/*
+ * This is the common EFI stub entry point for mixed mode.
+ *
+ * Arguments:	%ecx	image handle
+ * 		%edx	EFI system table pointer
+ *		%esi	struct bootparams pointer (or NULL when not using
+ *			the EFI handover protocol)
+ *
+ * Since this is the point of no return for ordinary execution, no registers
+ * are considered live except for the function parameters. [Note that the EFI
+ * stub may still exit and return to the firmware using the Exit() EFI boot
+ * service.]
+ */
+SYM_FUNC_START_LOCAL(efi32_entry)
 	call	1f
-1:	pop	%ebp
-	subl	$ rva(1b), %ebp
-
-	movl	%esi, rva(efi32_boot_args+8)(%ebp)
-SYM_INNER_LABEL(efi32_pe_stub_entry, SYM_L_LOCAL)
-	movl	%ecx, rva(efi32_boot_args)(%ebp)
-	movl	%edx, rva(efi32_boot_args+4)(%ebp)
-	movb	$0, rva(efi_is64)(%ebp)
+1:	pop	%ebx
 
 	/* Save firmware GDTR and code/data selectors */
-	sgdtl	rva(efi32_boot_gdt)(%ebp)
-	movw	%cs, rva(efi32_boot_cs)(%ebp)
-	movw	%ds, rva(efi32_boot_ds)(%ebp)
+	sgdtl	(efi32_boot_gdt - 1b)(%ebx)
+	movw	%cs, (efi32_boot_cs - 1b)(%ebx)
+	movw	%ds, (efi32_boot_ds - 1b)(%ebx)
 
 	/* Store firmware IDT descriptor */
-	sidtl	rva(efi32_boot_idt)(%ebp)
+	sidtl	(efi32_boot_idt - 1b)(%ebx)
+
+	/* Store boot arguments */
+	leal	(efi32_boot_args - 1b)(%ebx), %ebx
+	movl	%ecx, 0(%ebx)
+	movl	%edx, 4(%ebx)
+	movl	%esi, 8(%ebx)
+	movb	$0x0, 12(%ebx)          // efi_is64
 
 	/* Disable paging */
 	movl	%cr0, %eax
@@ -328,7 +345,8 @@ SYM_INNER_LABEL(efi32_pe_stub_entry, SYM_L_LOCAL)
 	movl	%eax, %cr0
 
 	jmp	startup_32
-SYM_FUNC_END(efi32_stub_entry)
+SYM_FUNC_END(efi32_entry)
+	__HEAD
 #endif
 
 	.code64
@@ -831,7 +849,9 @@ SYM_FUNC_START(efi32_pe_entry)
 	 */
 	subl	%esi, %ebx
 	movl	%ebx, rva(image_offset)(%ebp)	// save image_offset
-	jmp	efi32_pe_stub_entry
+	xorl	%esi, %esi
+	jmp	efi32_entry			// pass %ecx, %edx, %esi
+						// no other registers remain live
 
 2:	popl	%edi				// restore callee-save registers
 	popl	%ebx
author maintainer
#5 lore raw

PATCH v3 3/17 x86/compressed: efi-mixed: move bootargs parsing out of 32-bit startup code

In reply to Ard Biesheuvel

Move the logic that chooses between the different EFI entrypoints out of the 32-bit boot path, and into a 64-bit helper that can perform the same task much more cleanly. While at it, document the mixed mode boot flow in a code comment.

---
 arch/x86/boot/compressed/efi_mixed.S | 43 ++++++++++++++++++++
 arch/x86/boot/compressed/head_64.S   | 24 ++---------
 2 files changed, 47 insertions(+), 20 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +43 -0
index 67e7edcdfea8fa44..58ab2e1ffd92afbc 100644
@@ -22,6 +22,49 @@
 
 	.code64
 	.text
+/*
+ * When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode()
+ * is the first thing that runs after switching to long mode. Depending on
+ * whether the EFI handover protocol or the compat entry point was used to
+ * enter the kernel, it will either branch to the 64-bit EFI handover
+ * entrypoint at offset 0x390 in the image, or to the 64-bit EFI PE/COFF
+ * entrypoint efi_pe_entry(). In the former case, the bootloader must provide a
+ * struct bootparams pointer as the third argument, so the presence of such a
+ * pointer is used to disambiguate.
+ *
+ *                                                             +--------------+
+ *  +------------------+     +------------+            +------>| efi_pe_entry |
+ *  | efi32_pe_entry   |---->|            |            |       +-----------+--+
+ *  +------------------+     |            |     +------+----------------+  |
+ *                           | startup_32 |---->| startup_64_mixed_mode |  |
+ *  +------------------+     |            |     +------+----------------+  V
+ *  | efi32_stub_entry |---->|            |            |     +------------------+
+ *  +------------------+     +------------+            +---->| efi64_stub_entry |
+ *                                                           +-------------+----+
+ *                           +------------+     +----------+               |
+ *                           | startup_64 |<----| efi_main |<--------------+
+ *                           +------------+     +----------+
+ */
+SYM_FUNC_START(startup_64_mixed_mode)
+	lea	efi32_boot_args(%rip), %rdx
+	mov	0(%rdx), %edi
+	mov	4(%rdx), %esi
+	mov	8(%rdx), %edx		// saved bootparams pointer
+	test	%edx, %edx
+	jnz	efi64_stub_entry
+	/*
+	 * efi_pe_entry uses MS calling convention, which requires 32 bytes of
+	 * shadow space on the stack even if all arguments are passed in
+	 * registers. We also need an additional 8 bytes for the space that
+	 * would be occupied by the return address, and this also results in
+	 * the correct stack alignment for entry.
+	 */
+	sub	$40, %rsp
+	mov	%rdi, %rcx		// MS calling convention
+	mov	%rsi, %rdx
+	jmp	efi_pe_entry
+SYM_FUNC_END(startup_64_mixed_mode)
+
 SYM_FUNC_START(__efi64_thunk)
 	push	%rbp
 	push	%rbx
arch/x86/boot/compressed/head_64.S +4 -20
index 74aaffd22b7425f4..b9dbc8127038c315 100644
@@ -269,25 +269,9 @@ SYM_FUNC_START(startup_32)
 	 */
 	leal	rva(startup_64)(%ebp), %eax
 #ifdef CONFIG_EFI_MIXED
-	movl	rva(efi32_boot_args)(%ebp), %edi
-	testl	%edi, %edi
-	jz	1f
-	leal	rva(efi64_stub_entry)(%ebp), %eax
-	movl	rva(efi32_boot_args+4)(%ebp), %esi
-	movl	rva(efi32_boot_args+8)(%ebp), %edx	// saved bootparams pointer
-	testl	%edx, %edx
-	jnz	1f
-	/*
-	 * efi_pe_entry uses MS calling convention, which requires 32 bytes of
-	 * shadow space on the stack even if all arguments are passed in
-	 * registers. We also need an additional 8 bytes for the space that
-	 * would be occupied by the return address, and this also results in
-	 * the correct stack alignment for entry.
-	 */
-	subl	$40, %esp
-	leal	rva(efi_pe_entry)(%ebp), %eax
-	movl	%edi, %ecx			// MS calling convention
-	movl	%esi, %edx
+	cmpb	$1, rva(efi_is64)(%ebp)
+	je	1f
+	leal	rva(startup_64_mixed_mode)(%ebp), %eax
 1:
 #endif
 	/* Check if the C-bit position is correct when SEV is active */
@@ -787,7 +771,7 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
 SYM_DATA(image_offset, .long 0)
 #endif
 #ifdef CONFIG_EFI_MIXED
-SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0)
+SYM_DATA(efi32_boot_args, .long 0, 0, 0)
 SYM_DATA(efi_is64, .byte 1)
 
 #define ST32_boottime		60 // offsetof(efi_system_table_32_t, boottime)
Signature · 1 line
2.35.1
merge bot Applied
#6 lore raw

[tip: x86/boot] x86/boot/compressed: Move bootargs parsing out of 32-bit startup code

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     5c3a85f35b583259cf5ca0344cd79c8899ba1bb7
Gitweb:        https://git.kernel.org/tip/5c3a85f35b583259cf5ca0344cd79c8899ba1bb7
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:03 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Tue, 22 Nov 2022 19:21:26 +01:00

x86/boot/compressed: Move bootargs parsing out of 32-bit startup code

Move the logic that chooses between the different EFI entrypoints out of the 32-bit boot path, and into a 64-bit helper that can perform the same task much more cleanly. While at it, document the mixed mode boot flow in a code comment.

---
 arch/x86/boot/compressed/efi_mixed.S | 43 +++++++++++++++++++++++++++-
 arch/x86/boot/compressed/head_64.S   | 24 ++-------------
 2 files changed, 47 insertions(+), 20 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +43 -0
index 67e7edc..58ab2e1 100644
@@ -22,6 +22,49 @@
 
 	.code64
 	.text
+/*
+ * When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode()
+ * is the first thing that runs after switching to long mode. Depending on
+ * whether the EFI handover protocol or the compat entry point was used to
+ * enter the kernel, it will either branch to the 64-bit EFI handover
+ * entrypoint at offset 0x390 in the image, or to the 64-bit EFI PE/COFF
+ * entrypoint efi_pe_entry(). In the former case, the bootloader must provide a
+ * struct bootparams pointer as the third argument, so the presence of such a
+ * pointer is used to disambiguate.
+ *
+ *                                                             +--------------+
+ *  +------------------+     +------------+            +------>| efi_pe_entry |
+ *  | efi32_pe_entry   |---->|            |            |       +-----------+--+
+ *  +------------------+     |            |     +------+----------------+  |
+ *                           | startup_32 |---->| startup_64_mixed_mode |  |
+ *  +------------------+     |            |     +------+----------------+  V
+ *  | efi32_stub_entry |---->|            |            |     +------------------+
+ *  +------------------+     +------------+            +---->| efi64_stub_entry |
+ *                                                           +-------------+----+
+ *                           +------------+     +----------+               |
+ *                           | startup_64 |<----| efi_main |<--------------+
+ *                           +------------+     +----------+
+ */
+SYM_FUNC_START(startup_64_mixed_mode)
+	lea	efi32_boot_args(%rip), %rdx
+	mov	0(%rdx), %edi
+	mov	4(%rdx), %esi
+	mov	8(%rdx), %edx		// saved bootparams pointer
+	test	%edx, %edx
+	jnz	efi64_stub_entry
+	/*
+	 * efi_pe_entry uses MS calling convention, which requires 32 bytes of
+	 * shadow space on the stack even if all arguments are passed in
+	 * registers. We also need an additional 8 bytes for the space that
+	 * would be occupied by the return address, and this also results in
+	 * the correct stack alignment for entry.
+	 */
+	sub	$40, %rsp
+	mov	%rdi, %rcx		// MS calling convention
+	mov	%rsi, %rdx
+	jmp	efi_pe_entry
+SYM_FUNC_END(startup_64_mixed_mode)
+
 SYM_FUNC_START(__efi64_thunk)
 	push	%rbp
 	push	%rbx
arch/x86/boot/compressed/head_64.S +4 -20
index 5d07ef7..f3d7de5 100644
@@ -261,25 +261,9 @@ SYM_FUNC_START(startup_32)
 	 */
 	leal	rva(startup_64)(%ebp), %eax
 #ifdef CONFIG_EFI_MIXED
-	movl	rva(efi32_boot_args)(%ebp), %edi
-	testl	%edi, %edi
-	jz	1f
-	leal	rva(efi64_stub_entry)(%ebp), %eax
-	movl	rva(efi32_boot_args+4)(%ebp), %esi
-	movl	rva(efi32_boot_args+8)(%ebp), %edx	// saved bootparams pointer
-	testl	%edx, %edx
-	jnz	1f
-	/*
-	 * efi_pe_entry uses MS calling convention, which requires 32 bytes of
-	 * shadow space on the stack even if all arguments are passed in
-	 * registers. We also need an additional 8 bytes for the space that
-	 * would be occupied by the return address, and this also results in
-	 * the correct stack alignment for entry.
-	 */
-	subl	$40, %esp
-	leal	rva(efi_pe_entry)(%ebp), %eax
-	movl	%edi, %ecx			// MS calling convention
-	movl	%esi, %edx
+	cmpb	$1, rva(efi_is64)(%ebp)
+	je	1f
+	leal	rva(startup_64_mixed_mode)(%ebp), %eax
 1:
 #endif
 	/* Check if the C-bit position is correct when SEV is active */
@@ -779,7 +763,7 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
 SYM_DATA(image_offset, .long 0)
 #endif
 #ifdef CONFIG_EFI_MIXED
-SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0)
+SYM_DATA(efi32_boot_args, .long 0, 0, 0)
 SYM_DATA(efi_is64, .byte 1)
 
 #define ST32_boottime		60 // offsetof(efi_system_table_32_t, boottime)
author maintainer
#7 lore raw

PATCH v3 4/17 x86/compressed: efi-mixed: move efi32_pe_entry into .text section

In reply to Ard Biesheuvel

Move efi32_pe_entry() into the .text section, so that it can be moved out of head_64.S and into a separate compilation unit in a subsequent patch.

---
 arch/x86/boot/compressed/head_64.S | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
arch/x86/boot/compressed/head_64.S +5 -6
index b9dbc8127038c315..8192ab1c308567ce 100644
@@ -778,7 +778,7 @@ SYM_DATA(efi_is64, .byte 1)
 #define BS32_handle_protocol	88 // offsetof(efi_boot_services_32_t, handle_protocol)
 #define LI32_image_base		32 // offsetof(efi_loaded_image_32_t, image_base)
 
-	__HEAD
+	.text
 	.code32
 SYM_FUNC_START(efi32_pe_entry)
 /*
@@ -800,12 +800,11 @@ SYM_FUNC_START(efi32_pe_entry)
 
 	call	1f
 1:	pop	%ebx
-	subl	$ rva(1b), %ebx
 
 	/* Get the loaded image protocol pointer from the image handle */
 	leal	-4(%ebp), %eax
 	pushl	%eax				// &loaded_image
-	leal	rva(loaded_image_proto)(%ebx), %eax
+	leal	(loaded_image_proto - 1b)(%ebx), %eax
 	pushl	%eax				// pass the GUID address
 	pushl	8(%ebp)				// pass the image handle
 
@@ -834,13 +833,13 @@ SYM_FUNC_START(efi32_pe_entry)
 	movl	12(%ebp), %edx			// sys_table
 	movl	-4(%ebp), %esi			// loaded_image
 	movl	LI32_image_base(%esi), %esi	// loaded_image->image_base
-	movl	%ebx, %ebp			// startup_32 for efi32_pe_stub_entry
+	leal	(startup_32 - 1b)(%ebx), %ebp	// runtime address of startup_32
 	/*
 	 * We need to set the image_offset variable here since startup_32() will
 	 * use it before we get to the 64-bit efi_pe_entry() in C code.
 	 */
-	subl	%esi, %ebx
-	movl	%ebx, rva(image_offset)(%ebp)	// save image_offset
+	subl	%esi, %ebp			// calculate image_offset
+	movl	%ebp, (image_offset - 1b)(%ebx)	// save image_offset
 	xorl	%esi, %esi
 	jmp	efi32_entry			// pass %ecx, %edx, %esi
 						// no other registers remain live
Signature · 1 line
2.35.1
merge bot Applied
#8 lore raw

[tip: x86/boot] x86/boot/compressed: Move efi32_pe_entry into .text section

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     91592b5c0c2f076ff9d8cc0c14aa563448ac9fc4
Gitweb:        https://git.kernel.org/tip/91592b5c0c2f076ff9d8cc0c14aa563448ac9fc4
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:04 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Tue, 22 Nov 2022 19:23:01 +01:00

x86/boot/compressed: Move efi32_pe_entry into .text section

Move efi32_pe_entry() into the .text section, so that it can be moved out of head_64.S and into a separate compilation unit in a subsequent patch.

---
 arch/x86/boot/compressed/head_64.S | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
arch/x86/boot/compressed/head_64.S +5 -6
index f3d7de5..9dd9010 100644
@@ -770,7 +770,7 @@ SYM_DATA(efi_is64, .byte 1)
 #define BS32_handle_protocol	88 // offsetof(efi_boot_services_32_t, handle_protocol)
 #define LI32_image_base		32 // offsetof(efi_loaded_image_32_t, image_base)
 
-	__HEAD
+	.text
 	.code32
 SYM_FUNC_START(efi32_pe_entry)
 /*
@@ -792,12 +792,11 @@ SYM_FUNC_START(efi32_pe_entry)
 
 	call	1f
 1:	pop	%ebx
-	subl	$ rva(1b), %ebx
 
 	/* Get the loaded image protocol pointer from the image handle */
 	leal	-4(%ebp), %eax
 	pushl	%eax				// &loaded_image
-	leal	rva(loaded_image_proto)(%ebx), %eax
+	leal	(loaded_image_proto - 1b)(%ebx), %eax
 	pushl	%eax				// pass the GUID address
 	pushl	8(%ebp)				// pass the image handle
 
@@ -826,13 +825,13 @@ SYM_FUNC_START(efi32_pe_entry)
 	movl	12(%ebp), %edx			// sys_table
 	movl	-4(%ebp), %esi			// loaded_image
 	movl	LI32_image_base(%esi), %esi	// loaded_image->image_base
-	movl	%ebx, %ebp			// startup_32 for efi32_pe_stub_entry
+	leal	(startup_32 - 1b)(%ebx), %ebp	// runtime address of startup_32
 	/*
 	 * We need to set the image_offset variable here since startup_32() will
 	 * use it before we get to the 64-bit efi_pe_entry() in C code.
 	 */
-	subl	%esi, %ebx
-	movl	%ebx, rva(image_offset)(%ebp)	// save image_offset
+	subl	%esi, %ebp			// calculate image_offset
+	movl	%ebp, (image_offset - 1b)(%ebx)	// save image_offset
 	xorl	%esi, %esi
 	jmp	efi32_entry			// pass %ecx, %edx, %esi
 						// no other registers remain live
author maintainer
#9 lore raw

PATCH v3 5/17 x86/compressed: efi-mixed: move efi32_entry out of head_64.S

In reply to Ard Biesheuvel

Move the efi32_entry() routine out of head_64.S and into efi-mixed.S, which reduces clutter in the complicated startup routines. It also permits linkage of some symbols used by code to be made local.

---
 arch/x86/boot/compressed/efi_mixed.S | 57 ++++++++++++++++----
 arch/x86/boot/compressed/head_64.S   | 45 ----------------
 2 files changed, 47 insertions(+), 55 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +47 -10
index 58ab2e1ffd92afbc..3487484ac1fd5c6c 100644
@@ -105,7 +105,7 @@ SYM_FUNC_START(__efi64_thunk)
 	/*
 	 * Switch to IDT and GDT with 32-bit segments. This is the firmware GDT
 	 * and IDT that was installed when the kernel started executing. The
-	 * pointers were saved at the EFI stub entry point in head_64.S.
+	 * pointers were saved by the efi32_entry() routine below.
 	 *
 	 * Pass the saved DS selector to the 32-bit code, and use far return to
 	 * restore the saved CS selector.
@@ -217,22 +217,59 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 	lret
 SYM_FUNC_END(efi_enter32)
 
+/*
+ * This is the common EFI stub entry point for mixed mode.
+ *
+ * Arguments:	%ecx	image handle
+ * 		%edx	EFI system table pointer
+ *		%esi	struct bootparams pointer (or NULL when not using
+ *			the EFI handover protocol)
+ *
+ * Since this is the point of no return for ordinary execution, no registers
+ * are considered live except for the function parameters. [Note that the EFI
+ * stub may still exit and return to the firmware using the Exit() EFI boot
+ * service.]
+ */
+SYM_FUNC_START(efi32_entry)
+	call	1f
+1:	pop	%ebx
+
+	/* Save firmware GDTR and code/data selectors */
+	sgdtl	(efi32_boot_gdt - 1b)(%ebx)
+	movw	%cs, (efi32_boot_cs - 1b)(%ebx)
+	movw	%ds, (efi32_boot_ds - 1b)(%ebx)
+
+	/* Store firmware IDT descriptor */
+	sidtl	(efi32_boot_idt - 1b)(%ebx)
+
+	/* Store boot arguments */
+	leal	(efi32_boot_args - 1b)(%ebx), %ebx
+	movl	%ecx, 0(%ebx)
+	movl	%edx, 4(%ebx)
+	movl	%esi, 8(%ebx)
+	movb	$0x0, 12(%ebx)          // efi_is64
+
+	/* Disable paging */
+	movl	%cr0, %eax
+	btrl	$X86_CR0_PG_BIT, %eax
+	movl	%eax, %cr0
+
+	jmp	startup_32
+SYM_FUNC_END(efi32_entry)
+
 	.data
 	.balign	8
-SYM_DATA_START(efi32_boot_gdt)
+SYM_DATA_START_LOCAL(efi32_boot_gdt)
 	.word	0
 	.quad	0
 SYM_DATA_END(efi32_boot_gdt)
 
-SYM_DATA_START(efi32_boot_idt)
+SYM_DATA_START_LOCAL(efi32_boot_idt)
 	.word	0
 	.quad	0
 SYM_DATA_END(efi32_boot_idt)
 
-SYM_DATA_START(efi32_boot_cs)
-	.word	0
-SYM_DATA_END(efi32_boot_cs)
-
-SYM_DATA_START(efi32_boot_ds)
-	.word	0
-SYM_DATA_END(efi32_boot_ds)
+SYM_DATA_LOCAL(efi32_boot_cs, .word 0)
+SYM_DATA_LOCAL(efi32_boot_ds, .word 0)
+SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0)
+SYM_DATA(efi_is64, .byte 1)
arch/x86/boot/compressed/head_64.S +0 -45

57 lines not on this page · raw message

Signature · 1 line
2.35.1
merge bot Applied
#10 lore raw

[tip: x86/boot] x86/boot/compressed: Move efi32_entry out of head_64.S

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     73a6dec80e2acedaef3ca603d4b5799049f6e9f8
Gitweb:        https://git.kernel.org/tip/73a6dec80e2acedaef3ca603d4b5799049f6e9f8
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:05 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Tue, 22 Nov 2022 19:24:09 +01:00

x86/boot/compressed: Move efi32_entry out of head_64.S

Move the efi32_entry() routine out of head_64.S and into efi-mixed.S, which reduces clutter in the complicated startup routines. It also permits linkage of some symbols used by code to be made local.

---
 arch/x86/boot/compressed/efi_mixed.S | 57 ++++++++++++++++++++++-----
 arch/x86/boot/compressed/head_64.S   | 45 +---------------------
 2 files changed, 47 insertions(+), 55 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +47 -10
index 58ab2e1..3487484 100644
@@ -105,7 +105,7 @@ SYM_FUNC_START(__efi64_thunk)
 	/*
 	 * Switch to IDT and GDT with 32-bit segments. This is the firmware GDT
 	 * and IDT that was installed when the kernel started executing. The
-	 * pointers were saved at the EFI stub entry point in head_64.S.
+	 * pointers were saved by the efi32_entry() routine below.
 	 *
 	 * Pass the saved DS selector to the 32-bit code, and use far return to
 	 * restore the saved CS selector.
@@ -217,22 +217,59 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 	lret
 SYM_FUNC_END(efi_enter32)
 
+/*
+ * This is the common EFI stub entry point for mixed mode.
+ *
+ * Arguments:	%ecx	image handle
+ * 		%edx	EFI system table pointer
+ *		%esi	struct bootparams pointer (or NULL when not using
+ *			the EFI handover protocol)
+ *
+ * Since this is the point of no return for ordinary execution, no registers
+ * are considered live except for the function parameters. [Note that the EFI
+ * stub may still exit and return to the firmware using the Exit() EFI boot
+ * service.]
+ */
+SYM_FUNC_START(efi32_entry)
+	call	1f
+1:	pop	%ebx
+
+	/* Save firmware GDTR and code/data selectors */
+	sgdtl	(efi32_boot_gdt - 1b)(%ebx)
+	movw	%cs, (efi32_boot_cs - 1b)(%ebx)
+	movw	%ds, (efi32_boot_ds - 1b)(%ebx)
+
+	/* Store firmware IDT descriptor */
+	sidtl	(efi32_boot_idt - 1b)(%ebx)
+
+	/* Store boot arguments */
+	leal	(efi32_boot_args - 1b)(%ebx), %ebx
+	movl	%ecx, 0(%ebx)
+	movl	%edx, 4(%ebx)
+	movl	%esi, 8(%ebx)
+	movb	$0x0, 12(%ebx)          // efi_is64
+
+	/* Disable paging */
+	movl	%cr0, %eax
+	btrl	$X86_CR0_PG_BIT, %eax
+	movl	%eax, %cr0
+
+	jmp	startup_32
+SYM_FUNC_END(efi32_entry)
+
 	.data
 	.balign	8
-SYM_DATA_START(efi32_boot_gdt)
+SYM_DATA_START_LOCAL(efi32_boot_gdt)
 	.word	0
 	.quad	0
 SYM_DATA_END(efi32_boot_gdt)
 
-SYM_DATA_START(efi32_boot_idt)
+SYM_DATA_START_LOCAL(efi32_boot_idt)
 	.word	0
 	.quad	0
 SYM_DATA_END(efi32_boot_idt)
 
-SYM_DATA_START(efi32_boot_cs)
-	.word	0
-SYM_DATA_END(efi32_boot_cs)
-
-SYM_DATA_START(efi32_boot_ds)
-	.word	0
-SYM_DATA_END(efi32_boot_ds)
+SYM_DATA_LOCAL(efi32_boot_cs, .word 0)
+SYM_DATA_LOCAL(efi32_boot_ds, .word 0)
+SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0)
+SYM_DATA(efi_is64, .byte 1)
arch/x86/boot/compressed/head_64.S +0 -45

57 lines not on this page · raw message

author maintainer
#11 lore raw

PATCH v3 6/17 x86/compressed: efi-mixed: move efi32_pe_entry() out of head_64.S

In reply to Ard Biesheuvel

Move the implementation of efi32_pe_entry() into efi-mixed.S, which is a more suitable location that only gets built if EFI mixed mode is actually enabled.

---
 arch/x86/boot/compressed/efi_mixed.S | 82 ++++++++++++++++++
 arch/x86/boot/compressed/head_64.S   | 87 +-------------------
 2 files changed, 83 insertions(+), 86 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +82 -0
index 3487484ac1fd5c6c..8844d8ed4b1c7561 100644
@@ -257,6 +257,88 @@ SYM_FUNC_START(efi32_entry)
 	jmp	startup_32
 SYM_FUNC_END(efi32_entry)
 
+#define ST32_boottime		60 // offsetof(efi_system_table_32_t, boottime)
+#define BS32_handle_protocol	88 // offsetof(efi_boot_services_32_t, handle_protocol)
+#define LI32_image_base		32 // offsetof(efi_loaded_image_32_t, image_base)
+
+/*
+ * efi_status_t efi32_pe_entry(efi_handle_t image_handle,
+ *			       efi_system_table_32_t *sys_table)
+ */
+SYM_FUNC_START(efi32_pe_entry)
+	pushl	%ebp
+	movl	%esp, %ebp
+	pushl	%eax				// dummy push to allocate loaded_image
+
+	pushl	%ebx				// save callee-save registers
+	pushl	%edi
+
+	call	verify_cpu			// check for long mode support
+	testl	%eax, %eax
+	movl	$0x80000003, %eax		// EFI_UNSUPPORTED
+	jnz	2f
+
+	call	1f
+1:	pop	%ebx
+
+	/* Get the loaded image protocol pointer from the image handle */
+	leal	-4(%ebp), %eax
+	pushl	%eax				// &loaded_image
+	leal	(loaded_image_proto - 1b)(%ebx), %eax
+	pushl	%eax				// pass the GUID address
+	pushl	8(%ebp)				// pass the image handle
+
+	/*
+	 * Note the alignment of the stack frame.
+	 *   sys_table
+	 *   handle             <-- 16-byte aligned on entry by ABI
+	 *   return address
+	 *   frame pointer
+	 *   loaded_image       <-- local variable
+	 *   saved %ebx		<-- 16-byte aligned here
+	 *   saved %edi
+	 *   &loaded_image
+	 *   &loaded_image_proto
+	 *   handle             <-- 16-byte aligned for call to handle_protocol
+	 */
+
+	movl	12(%ebp), %eax			// sys_table
+	movl	ST32_boottime(%eax), %eax	// sys_table->boottime
+	call	*BS32_handle_protocol(%eax)	// sys_table->boottime->handle_protocol
+	addl	$12, %esp			// restore argument space
+	testl	%eax, %eax
+	jnz	2f
+
+	movl	8(%ebp), %ecx			// image_handle
+	movl	12(%ebp), %edx			// sys_table
+	movl	-4(%ebp), %esi			// loaded_image
+	movl	LI32_image_base(%esi), %esi	// loaded_image->image_base
+	leal	(startup_32 - 1b)(%ebx), %ebp	// runtime address of startup_32
+	/*
+	 * We need to set the image_offset variable here since startup_32() will
+	 * use it before we get to the 64-bit efi_pe_entry() in C code.
+	 */
+	subl	%esi, %ebp			// calculate image_offset
+	movl	%ebp, (image_offset - 1b)(%ebx)	// save image_offset
+	xorl	%esi, %esi
+	jmp	efi32_entry			// pass %ecx, %edx, %esi
+						// no other registers remain live
+
+2:	popl	%edi				// restore callee-save registers
+	popl	%ebx
+	leave
+	RET
+SYM_FUNC_END(efi32_pe_entry)
+
+	.section ".rodata"
+	/* EFI loaded image protocol GUID */
+	.balign 4
+SYM_DATA_START_LOCAL(loaded_image_proto)
+	.long	0x5b1b31a1
+	.word	0x9562, 0x11d2
+	.byte	0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b
+SYM_DATA_END(loaded_image_proto)
+
 	.data
 	.balign	8
 SYM_DATA_START_LOCAL(efi32_boot_gdt)
arch/x86/boot/compressed/head_64.S +1 -86

99 lines not on this page · raw message

Signature · 1 line
2.35.1
merge bot Applied
#12 lore raw

[tip: x86/boot] x86/boot/compressed: Move efi32_pe_entry() out of head_64.S

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     7f22ca396778fea9332d83ec2359dbe8396e9a06
Gitweb:        https://git.kernel.org/tip/7f22ca396778fea9332d83ec2359dbe8396e9a06
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:06 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Tue, 22 Nov 2022 19:24:57 +01:00

x86/boot/compressed: Move efi32_pe_entry() out of head_64.S

Move the implementation of efi32_pe_entry() into efi-mixed.S, which is a more suitable location that only gets built if EFI mixed mode is actually enabled.

---
 arch/x86/boot/compressed/efi_mixed.S | 82 +++++++++++++++++++++++++-
 arch/x86/boot/compressed/head_64.S   | 87 +---------------------------
 2 files changed, 83 insertions(+), 86 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +82 -0
index 3487484..8844d8e 100644
@@ -257,6 +257,88 @@ SYM_FUNC_START(efi32_entry)
 	jmp	startup_32
 SYM_FUNC_END(efi32_entry)
 
+#define ST32_boottime		60 // offsetof(efi_system_table_32_t, boottime)
+#define BS32_handle_protocol	88 // offsetof(efi_boot_services_32_t, handle_protocol)
+#define LI32_image_base		32 // offsetof(efi_loaded_image_32_t, image_base)
+
+/*
+ * efi_status_t efi32_pe_entry(efi_handle_t image_handle,
+ *			       efi_system_table_32_t *sys_table)
+ */
+SYM_FUNC_START(efi32_pe_entry)
+	pushl	%ebp
+	movl	%esp, %ebp
+	pushl	%eax				// dummy push to allocate loaded_image
+
+	pushl	%ebx				// save callee-save registers
+	pushl	%edi
+
+	call	verify_cpu			// check for long mode support
+	testl	%eax, %eax
+	movl	$0x80000003, %eax		// EFI_UNSUPPORTED
+	jnz	2f
+
+	call	1f
+1:	pop	%ebx
+
+	/* Get the loaded image protocol pointer from the image handle */
+	leal	-4(%ebp), %eax
+	pushl	%eax				// &loaded_image
+	leal	(loaded_image_proto - 1b)(%ebx), %eax
+	pushl	%eax				// pass the GUID address
+	pushl	8(%ebp)				// pass the image handle
+
+	/*
+	 * Note the alignment of the stack frame.
+	 *   sys_table
+	 *   handle             <-- 16-byte aligned on entry by ABI
+	 *   return address
+	 *   frame pointer
+	 *   loaded_image       <-- local variable
+	 *   saved %ebx		<-- 16-byte aligned here
+	 *   saved %edi
+	 *   &loaded_image
+	 *   &loaded_image_proto
+	 *   handle             <-- 16-byte aligned for call to handle_protocol
+	 */
+
+	movl	12(%ebp), %eax			// sys_table
+	movl	ST32_boottime(%eax), %eax	// sys_table->boottime
+	call	*BS32_handle_protocol(%eax)	// sys_table->boottime->handle_protocol
+	addl	$12, %esp			// restore argument space
+	testl	%eax, %eax
+	jnz	2f
+
+	movl	8(%ebp), %ecx			// image_handle
+	movl	12(%ebp), %edx			// sys_table
+	movl	-4(%ebp), %esi			// loaded_image
+	movl	LI32_image_base(%esi), %esi	// loaded_image->image_base
+	leal	(startup_32 - 1b)(%ebx), %ebp	// runtime address of startup_32
+	/*
+	 * We need to set the image_offset variable here since startup_32() will
+	 * use it before we get to the 64-bit efi_pe_entry() in C code.
+	 */
+	subl	%esi, %ebp			// calculate image_offset
+	movl	%ebp, (image_offset - 1b)(%ebx)	// save image_offset
+	xorl	%esi, %esi
+	jmp	efi32_entry			// pass %ecx, %edx, %esi
+						// no other registers remain live
+
+2:	popl	%edi				// restore callee-save registers
+	popl	%ebx
+	leave
+	RET
+SYM_FUNC_END(efi32_pe_entry)
+
+	.section ".rodata"
+	/* EFI loaded image protocol GUID */
+	.balign 4
+SYM_DATA_START_LOCAL(loaded_image_proto)
+	.long	0x5b1b31a1
+	.word	0x9562, 0x11d2
+	.byte	0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b
+SYM_DATA_END(loaded_image_proto)
+
 	.data
 	.balign	8
 SYM_DATA_START_LOCAL(efi32_boot_gdt)
arch/x86/boot/compressed/head_64.S +1 -86

99 lines not on this page · raw message

author maintainer
#13 lore raw

PATCH v3 7/17 x86/compressed: efi: merge multiple definitions of image_offset into one

In reply to Ard Biesheuvel

There is no need for head_32.S and head_64.S both declaring a copy of the globale 'image_offset' variable, so drop those and make the extern C declaration the definition.

---
 arch/x86/boot/compressed/head_32.S      | 4 ----
 arch/x86/boot/compressed/head_64.S      | 4 ----
 drivers/firmware/efi/libstub/x86-stub.c | 2 +-
 3 files changed, 1 insertion(+), 9 deletions(-)
arch/x86/boot/compressed/head_32.S +0 -4
index 3b354eb9516df416..6589ddd4cfaf2cb6 100644
@@ -208,10 +208,6 @@ SYM_DATA_START_LOCAL(gdt)
 	.quad	0x00cf92000000ffff	/* __KERNEL_DS */
 SYM_DATA_END_LABEL(gdt, SYM_L_LOCAL, gdt_end)
 
-#ifdef CONFIG_EFI_STUB
-SYM_DATA(image_offset, .long 0)
-#endif
-
 /*
  * Stack and heap for uncompression
  */
arch/x86/boot/compressed/head_64.S +0 -4
index 987f0a3c284fad43..66ad3ab802ca9d0c 100644
@@ -726,10 +726,6 @@ SYM_DATA_START(boot32_idt)
 SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
 #endif
 
-#ifdef CONFIG_EFI_STUB
-SYM_DATA(image_offset, .long 0)
-#endif
-
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 	__HEAD
 	.code32
drivers/firmware/efi/libstub/x86-stub.c +1 -1
index 33a7811e12c65642..cb5f0befee57d2ec 100644
@@ -23,7 +23,7 @@
 
 const efi_system_table_t *efi_system_table;
 const efi_dxe_services_table_t *efi_dxe_table;
-extern u32 image_offset;
+u32 image_offset;
 static efi_loaded_image_t *image = NULL;
 
 static efi_status_t
Signature · 1 line
2.35.1
merge bot Applied
#14 lore raw

[tip: x86/boot] x86/boot/compressed, efi: Merge multiple definitions of image_offset into one

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     4b52016247aeaa55ca3e3bc2e03cd91114c145c2
Gitweb:        https://git.kernel.org/tip/4b52016247aeaa55ca3e3bc2e03cd91114c145c2
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:07 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:55:55 +01:00

x86/boot/compressed, efi: Merge multiple definitions of image_offset into one

There is no need for head_32.S and head_64.S both declaring a copy of the global 'image_offset' variable, so drop those and make the extern C declaration the definition.

When image_offset is moved to the .c file, it needs to be placed particularly in the .data section because it lands by default in the .bss section which is cleared too late, in .Lrelocated, before the first access to it and thus garbage gets read, leading to SEV guests exploding in early boot.

This happens only when the SEV guest kernel is loaded through grub. If supplied with qemu's -kernel command line option, that memory is always cleared upfront by qemu and all is fine there.

[ bp: Expand commit message with SEV aspect. ]

---
 arch/x86/boot/compressed/head_32.S      | 4 ----
 arch/x86/boot/compressed/head_64.S      | 4 ----
 drivers/firmware/efi/libstub/x86-stub.c | 2 +-
 3 files changed, 1 insertion(+), 9 deletions(-)
arch/x86/boot/compressed/head_32.S +0 -4
index 3b354eb..6589ddd 100644
@@ -208,10 +208,6 @@ SYM_DATA_START_LOCAL(gdt)
 	.quad	0x00cf92000000ffff	/* __KERNEL_DS */
 SYM_DATA_END_LABEL(gdt, SYM_L_LOCAL, gdt_end)
 
-#ifdef CONFIG_EFI_STUB
-SYM_DATA(image_offset, .long 0)
-#endif
-
 /*
  * Stack and heap for uncompression
  */
arch/x86/boot/compressed/head_64.S +0 -4
index 36f37f9..34d0395 100644
@@ -718,10 +718,6 @@ SYM_DATA_START(boot32_idt)
 SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
 #endif
 
-#ifdef CONFIG_EFI_STUB
-SYM_DATA(image_offset, .long 0)
-#endif
-
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 	__HEAD
 	.code32
drivers/firmware/efi/libstub/x86-stub.c +1 -1
index 33a7811..a0bfd31 100644
@@ -23,7 +23,7 @@
 
 const efi_system_table_t *efi_system_table;
 const efi_dxe_services_table_t *efi_dxe_table;
-extern u32 image_offset;
+u32 image_offset __section(".data");
 static efi_loaded_image_t *image = NULL;
 
 static efi_status_t
author maintainer
#15 lore raw

PATCH v3 8/17 x86/compressed: efi-mixed: simplify IDT/GDT preserve/restore

In reply to Ard Biesheuvel

Tweak the asm and remove some redundant instructions. While at it, fix the associated comment for style and correctness.

---
 arch/x86/boot/compressed/efi_mixed.S | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +7 -13
index 8844d8ed4b1c7561..8b02e507d3bb055a 100644
@@ -96,24 +96,20 @@ SYM_FUNC_START(__efi64_thunk)
 
 	leaq	0x20(%rsp), %rbx
 	sgdt	(%rbx)
-
-	addq	$16, %rbx
-	sidt	(%rbx)
+	sidt	16(%rbx)
 
 	leaq	1f(%rip), %rbp
 
 	/*
-	 * Switch to IDT and GDT with 32-bit segments. This is the firmware GDT
-	 * and IDT that was installed when the kernel started executing. The
-	 * pointers were saved by the efi32_entry() routine below.
+	 * Switch to IDT and GDT with 32-bit segments. These are the firmware
+	 * GDT and IDT that were installed when the kernel started executing.
+	 * The pointers were saved by the efi32_entry() routine below.
 	 *
 	 * Pass the saved DS selector to the 32-bit code, and use far return to
 	 * restore the saved CS selector.
 	 */
-	leaq	efi32_boot_idt(%rip), %rax
-	lidt	(%rax)
-	leaq	efi32_boot_gdt(%rip), %rax
-	lgdt	(%rax)
+	lidt	efi32_boot_idt(%rip)
+	lgdt	efi32_boot_gdt(%rip)
 
 	movzwl	efi32_boot_ds(%rip), %edx
 	movzwq	efi32_boot_cs(%rip), %rax
@@ -187,9 +183,7 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 	 */
 	cli
 
-	lidtl	(%ebx)
-	subl	$16, %ebx
-
+	lidtl	16(%ebx)
 	lgdtl	(%ebx)
 
 	movl	%cr4, %eax
Signature · 1 line
2.35.1
merge bot Applied
#16 lore raw

[tip: x86/boot] x86/boot/compressed: Simplify IDT/GDT preserve/restore in the EFI thunk

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     630f337f0c4fd80390e8600adcab31550aea33df
Gitweb:        https://git.kernel.org/tip/630f337f0c4fd80390e8600adcab31550aea33df
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:08 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00

x86/boot/compressed: Simplify IDT/GDT preserve/restore in the EFI thunk

Tweak the asm and remove some redundant instructions. While at it, fix the associated comment for style and correctness.

---
 arch/x86/boot/compressed/efi_mixed.S | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +7 -13
index 8844d8e..8b02e50 100644
@@ -96,24 +96,20 @@ SYM_FUNC_START(__efi64_thunk)
 
 	leaq	0x20(%rsp), %rbx
 	sgdt	(%rbx)
-
-	addq	$16, %rbx
-	sidt	(%rbx)
+	sidt	16(%rbx)
 
 	leaq	1f(%rip), %rbp
 
 	/*
-	 * Switch to IDT and GDT with 32-bit segments. This is the firmware GDT
-	 * and IDT that was installed when the kernel started executing. The
-	 * pointers were saved by the efi32_entry() routine below.
+	 * Switch to IDT and GDT with 32-bit segments. These are the firmware
+	 * GDT and IDT that were installed when the kernel started executing.
+	 * The pointers were saved by the efi32_entry() routine below.
 	 *
 	 * Pass the saved DS selector to the 32-bit code, and use far return to
 	 * restore the saved CS selector.
 	 */
-	leaq	efi32_boot_idt(%rip), %rax
-	lidt	(%rax)
-	leaq	efi32_boot_gdt(%rip), %rax
-	lgdt	(%rax)
+	lidt	efi32_boot_idt(%rip)
+	lgdt	efi32_boot_gdt(%rip)
 
 	movzwl	efi32_boot_ds(%rip), %edx
 	movzwq	efi32_boot_cs(%rip), %rax
@@ -187,9 +183,7 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 	 */
 	cli
 
-	lidtl	(%ebx)
-	subl	$16, %ebx
-
+	lidtl	16(%ebx)
 	lgdtl	(%ebx)
 
 	movl	%cr4, %eax
author maintainer
#17 lore raw

PATCH v3 9/17 x86/compressed: avoid touching ECX in startup32_set_idt_entry()

In reply to Ard Biesheuvel

Avoid touching register %ecx in startup32_set_idt_entry(), by folding the MOV, SHL and ORL instructions into a single ORL which no longer requires a temp register.

This permits ECX to be used as a function argument in a subsequent patch.

---
 arch/x86/boot/compressed/head_64.S | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)
arch/x86/boot/compressed/head_64.S +2 -6
index 66ad3ab802ca9d0c..f31277f455e63f5f 100644
@@ -741,7 +741,6 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
  */
 SYM_FUNC_START(startup32_set_idt_entry)
 	push    %ebx
-	push    %ecx
 
 	/* IDT entry address to %ebx */
 	leal    rva(boot32_idt)(%ebp), %ebx
@@ -750,10 +749,8 @@ SYM_FUNC_START(startup32_set_idt_entry)
 
 	/* Build IDT entry, lower 4 bytes */
 	movl    %eax, %edx
-	andl    $0x0000ffff, %edx	# Target code segment offset [15:0]
-	movl    $__KERNEL32_CS, %ecx	# Target code segment selector
-	shl     $16, %ecx
-	orl     %ecx, %edx
+	andl    $0x0000ffff, %edx		# Target code segment offset [15:0]
+	orl	$(__KERNEL32_CS << 16), %edx	# Target code segment selector
 
 	/* Store lower 4 bytes to IDT */
 	movl    %edx, (%ebx)
@@ -766,7 +763,6 @@ SYM_FUNC_START(startup32_set_idt_entry)
 	/* Store upper 4 bytes to IDT */
 	movl    %edx, 4(%ebx)
 
-	pop     %ecx
 	pop     %ebx
 	RET
 SYM_FUNC_END(startup32_set_idt_entry)
Signature · 1 line
2.35.1
merge bot Applied
#18 lore raw

[tip: x86/boot] x86/boot/compressed: Avoid touching ECX in startup32_set_idt_entry()

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     6aac80a8da46d70f2ae7ff97c9f45a15c7c9b3ef
Gitweb:        https://git.kernel.org/tip/6aac80a8da46d70f2ae7ff97c9f45a15c7c9b3ef
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:09 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00

x86/boot/compressed: Avoid touching ECX in startup32_set_idt_entry()

Avoid touching register %ecx in startup32_set_idt_entry(), by folding the MOV, SHL and ORL instructions into a single ORL which no longer requires a temp register.

This permits ECX to be used as a function argument in a subsequent patch.

---
 arch/x86/boot/compressed/head_64.S | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)
arch/x86/boot/compressed/head_64.S +2 -6
index 34d0395..a2d1c03 100644
@@ -733,7 +733,6 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
  */
 SYM_FUNC_START(startup32_set_idt_entry)
 	push    %ebx
-	push    %ecx
 
 	/* IDT entry address to %ebx */
 	leal    rva(boot32_idt)(%ebp), %ebx
@@ -742,10 +741,8 @@ SYM_FUNC_START(startup32_set_idt_entry)
 
 	/* Build IDT entry, lower 4 bytes */
 	movl    %eax, %edx
-	andl    $0x0000ffff, %edx	# Target code segment offset [15:0]
-	movl    $__KERNEL32_CS, %ecx	# Target code segment selector
-	shl     $16, %ecx
-	orl     %ecx, %edx
+	andl    $0x0000ffff, %edx		# Target code segment offset [15:0]
+	orl	$(__KERNEL32_CS << 16), %edx	# Target code segment selector
 
 	/* Store lower 4 bytes to IDT */
 	movl    %edx, (%ebx)
@@ -758,7 +755,6 @@ SYM_FUNC_START(startup32_set_idt_entry)
 	/* Store upper 4 bytes to IDT */
 	movl    %edx, 4(%ebx)
 
-	pop     %ecx
 	pop     %ebx
 	RET
 SYM_FUNC_END(startup32_set_idt_entry)
author maintainer
#19 lore raw

PATCH v3 10/17 x86/compressed: pull global variable ref up into startup32_load_idt()

In reply to Ard Biesheuvel

In preparation for moving startup32_load_idt() out of head_64.S and turning it into an ordinary function using the ordinary 32-bit calling convention, pull the global variable reference to boot32_idt up into startup32_load_idt() so that startup32_set_idt_entry() does not need to discover its own runtime physical address, which will no longer be correlated with startup_32 once this code is moved into .text.

While at it, give startup32_set_idt_entry() static linkage.

---
 arch/x86/boot/compressed/head_64.S | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)
arch/x86/boot/compressed/head_64.S +8 -12
index f31277f455e63f5f..82d7fab99422129f 100644
@@ -736,16 +736,11 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
  *
  * %eax:	Handler address
  * %edx:	Vector number
- *
- * Physical offset is expected in %ebp
+ * %ecx:	IDT address
  */
-SYM_FUNC_START(startup32_set_idt_entry)
-	push    %ebx
-
-	/* IDT entry address to %ebx */
-	leal    rva(boot32_idt)(%ebp), %ebx
-	shl	$3, %edx
-	addl    %edx, %ebx
+SYM_FUNC_START_LOCAL(startup32_set_idt_entry)
+	/* IDT entry address to %ecx */
+	leal	(%ecx, %edx, 8), %ecx
 
 	/* Build IDT entry, lower 4 bytes */
 	movl    %eax, %edx
@@ -753,7 +748,7 @@ SYM_FUNC_START(startup32_set_idt_entry)
 	orl	$(__KERNEL32_CS << 16), %edx	# Target code segment selector
 
 	/* Store lower 4 bytes to IDT */
-	movl    %edx, (%ebx)
+	movl    %edx, (%ecx)
 
 	/* Build IDT entry, upper 4 bytes */
 	movl    %eax, %edx
@@ -761,15 +756,16 @@ SYM_FUNC_START(startup32_set_idt_entry)
 	orl     $0x00008e00, %edx	# Present, Type 32-bit Interrupt Gate
 
 	/* Store upper 4 bytes to IDT */
-	movl    %edx, 4(%ebx)
+	movl    %edx, 4(%ecx)
 
-	pop     %ebx
 	RET
 SYM_FUNC_END(startup32_set_idt_entry)
 #endif
 
 SYM_FUNC_START(startup32_load_idt)
 #ifdef CONFIG_AMD_MEM_ENCRYPT
+	leal    rva(boot32_idt)(%ebp), %ecx
+
 	/* #VC handler */
 	leal    rva(startup32_vc_handler)(%ebp), %eax
 	movl    $X86_TRAP_VC, %edx
Signature · 1 line
2.35.1
merge bot Applied
#20 lore raw

[tip: x86/boot] x86/boot/compressed: Pull global variable reference into startup32_load_idt()

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     d73a257f7f86871c3aac24dc20538e3983096647
Gitweb:        https://git.kernel.org/tip/d73a257f7f86871c3aac24dc20538e3983096647
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:10 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00

x86/boot/compressed: Pull global variable reference into startup32_load_idt()

In preparation for moving startup32_load_idt() out of head_64.S and turning it into an ordinary function using the ordinary 32-bit calling convention, pull the global variable reference to boot32_idt up into startup32_load_idt() so that startup32_set_idt_entry() does not need to discover its own runtime physical address, which will no longer be correlated with startup_32 once this code is moved into .text.

While at it, give startup32_set_idt_entry() static linkage.

---
 arch/x86/boot/compressed/head_64.S | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)
arch/x86/boot/compressed/head_64.S +8 -12
index a2d1c03..f9926b3 100644
@@ -728,16 +728,11 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
  *
  * %eax:	Handler address
  * %edx:	Vector number
- *
- * Physical offset is expected in %ebp
+ * %ecx:	IDT address
  */
-SYM_FUNC_START(startup32_set_idt_entry)
-	push    %ebx
-
-	/* IDT entry address to %ebx */
-	leal    rva(boot32_idt)(%ebp), %ebx
-	shl	$3, %edx
-	addl    %edx, %ebx
+SYM_FUNC_START_LOCAL(startup32_set_idt_entry)
+	/* IDT entry address to %ecx */
+	leal	(%ecx, %edx, 8), %ecx
 
 	/* Build IDT entry, lower 4 bytes */
 	movl    %eax, %edx
@@ -745,7 +740,7 @@ SYM_FUNC_START(startup32_set_idt_entry)
 	orl	$(__KERNEL32_CS << 16), %edx	# Target code segment selector
 
 	/* Store lower 4 bytes to IDT */
-	movl    %edx, (%ebx)
+	movl    %edx, (%ecx)
 
 	/* Build IDT entry, upper 4 bytes */
 	movl    %eax, %edx
@@ -753,15 +748,16 @@ SYM_FUNC_START(startup32_set_idt_entry)
 	orl     $0x00008e00, %edx	# Present, Type 32-bit Interrupt Gate
 
 	/* Store upper 4 bytes to IDT */
-	movl    %edx, 4(%ebx)
+	movl    %edx, 4(%ecx)
 
-	pop     %ebx
 	RET
 SYM_FUNC_END(startup32_set_idt_entry)
 #endif
 
 SYM_FUNC_START(startup32_load_idt)
 #ifdef CONFIG_AMD_MEM_ENCRYPT
+	leal    rva(boot32_idt)(%ebp), %ecx
+
 	/* #VC handler */
 	leal    rva(startup32_vc_handler)(%ebp), %eax
 	movl    $X86_TRAP_VC, %edx
author maintainer
#21 lore raw

PATCH v3 11/17 x86/compressed: move startup32_load_idt() into .text section

In reply to Ard Biesheuvel

Convert startup32_load_idt() into an ordinary function and move it into the .text section. This involves turning the rva() immediates into ones derived from a local label, and preserving/restoring the %ebp and %ebx as per the calling convention.

Also move the #ifdef to the only existing call site. This makes it clear that the function call does nothing if support for memory encryption is not compiled in.

---
 arch/x86/boot/compressed/head_64.S | 31 +++++++++++++-------
 1 file changed, 20 insertions(+), 11 deletions(-)
arch/x86/boot/compressed/head_64.S +20 -11
index 82d7fab99422129f..2d42852d5b828209 100644
@@ -126,7 +126,9 @@ SYM_FUNC_START(startup_32)
 1:
 
 	/* Setup Exception handling for SEV-ES */
+#ifdef CONFIG_AMD_MEM_ENCRYPT
 	call	startup32_load_idt
+#endif
 
 	/* Make sure cpu supports long mode. */
 	call	verify_cpu
@@ -724,10 +726,8 @@ SYM_DATA_START(boot32_idt)
 	.quad 0
 	.endr
 SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
-#endif
 
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-	__HEAD
+	.text
 	.code32
 /*
  * Write an IDT entry into boot32_idt
@@ -760,24 +760,32 @@ SYM_FUNC_START_LOCAL(startup32_set_idt_entry)
 
 	RET
 SYM_FUNC_END(startup32_set_idt_entry)
-#endif
 
 SYM_FUNC_START(startup32_load_idt)
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-	leal    rva(boot32_idt)(%ebp), %ecx
+	push	%ebp
+	push	%ebx
+
+	call	1f
+1:	pop	%ebp
+
+	leal    (boot32_idt - 1b)(%ebp), %ebx
 
 	/* #VC handler */
-	leal    rva(startup32_vc_handler)(%ebp), %eax
+	leal    (startup32_vc_handler - 1b)(%ebp), %eax
 	movl    $X86_TRAP_VC, %edx
+	movl	%ebx, %ecx
 	call    startup32_set_idt_entry
 
 	/* Load IDT */
-	leal	rva(boot32_idt)(%ebp), %eax
-	movl	%eax, rva(boot32_idt_desc+2)(%ebp)
-	lidt    rva(boot32_idt_desc)(%ebp)
-#endif
+	leal	(boot32_idt_desc - 1b)(%ebp), %ecx
+	movl	%ebx, 2(%ecx)
+	lidt    (%ecx)
+
+	pop	%ebx
+	pop	%ebp
 	RET
 SYM_FUNC_END(startup32_load_idt)
+#endif
 
 /*
  * Check for the correct C-bit position when the startup_32 boot-path is used.
@@ -796,6 +804,7 @@ SYM_FUNC_END(startup32_load_idt)
  * succeed. An incorrect C-bit position will map all memory unencrypted, so that
  * the compare will use the encrypted random data and fail.
  */
+	__HEAD
 SYM_FUNC_START(startup32_check_sev_cbit)
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 	pushl	%eax
Signature · 1 line
2.35.1
merge bot Applied
#22 lore raw

[tip: x86/boot] x86/boot/compressed: Move startup32_load_idt() into .text section

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     c6355995ba471d7ad574174e593192ce805c7e1a
Gitweb:        https://git.kernel.org/tip/c6355995ba471d7ad574174e593192ce805c7e1a
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:11 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00

x86/boot/compressed: Move startup32_load_idt() into .text section

Convert startup32_load_idt() into an ordinary function and move it into the .text section. This involves turning the rva() immediates into ones derived from a local label, and preserving/restoring the %ebp and %ebx as per the calling convention.

Also move the #ifdef to the only existing call site. This makes it clear that the function call does nothing if support for memory encryption is not compiled in.

---
 arch/x86/boot/compressed/head_64.S | 31 ++++++++++++++++++-----------
 1 file changed, 20 insertions(+), 11 deletions(-)
arch/x86/boot/compressed/head_64.S +20 -11
index f9926b3..7aa147f 100644
@@ -118,7 +118,9 @@ SYM_FUNC_START(startup_32)
 1:
 
 	/* Setup Exception handling for SEV-ES */
+#ifdef CONFIG_AMD_MEM_ENCRYPT
 	call	startup32_load_idt
+#endif
 
 	/* Make sure cpu supports long mode. */
 	call	verify_cpu
@@ -716,10 +718,8 @@ SYM_DATA_START(boot32_idt)
 	.quad 0
 	.endr
 SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
-#endif
 
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-	__HEAD
+	.text
 	.code32
 /*
  * Write an IDT entry into boot32_idt
@@ -752,24 +752,32 @@ SYM_FUNC_START_LOCAL(startup32_set_idt_entry)
 
 	RET
 SYM_FUNC_END(startup32_set_idt_entry)
-#endif
 
 SYM_FUNC_START(startup32_load_idt)
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-	leal    rva(boot32_idt)(%ebp), %ecx
+	push	%ebp
+	push	%ebx
+
+	call	1f
+1:	pop	%ebp
+
+	leal    (boot32_idt - 1b)(%ebp), %ebx
 
 	/* #VC handler */
-	leal    rva(startup32_vc_handler)(%ebp), %eax
+	leal    (startup32_vc_handler - 1b)(%ebp), %eax
 	movl    $X86_TRAP_VC, %edx
+	movl	%ebx, %ecx
 	call    startup32_set_idt_entry
 
 	/* Load IDT */
-	leal	rva(boot32_idt)(%ebp), %eax
-	movl	%eax, rva(boot32_idt_desc+2)(%ebp)
-	lidt    rva(boot32_idt_desc)(%ebp)
-#endif
+	leal	(boot32_idt_desc - 1b)(%ebp), %ecx
+	movl	%ebx, 2(%ecx)
+	lidt    (%ecx)
+
+	pop	%ebx
+	pop	%ebp
 	RET
 SYM_FUNC_END(startup32_load_idt)
+#endif
 
 /*
  * Check for the correct C-bit position when the startup_32 boot-path is used.
@@ -788,6 +796,7 @@ SYM_FUNC_END(startup32_load_idt)
  * succeed. An incorrect C-bit position will map all memory unencrypted, so that
  * the compare will use the encrypted random data and fail.
  */
+	__HEAD
 SYM_FUNC_START(startup32_check_sev_cbit)
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 	pushl	%eax
author maintainer
#23 lore raw

PATCH v3 12/17 x86/compressed: move startup32_load_idt() out of head_64.S

In reply to Ard Biesheuvel

Now that startup32_load_idt() has been refactored into an ordinary callable function, move it into mem-encrypt.S where it belongs.

---
 arch/x86/boot/compressed/head_64.S     | 72 --------------------
 arch/x86/boot/compressed/mem_encrypt.S | 72 +++++++++++++++++++-
 2 files changed, 71 insertions(+), 73 deletions(-)
arch/x86/boot/compressed/head_64.S +0 -72
index 2d42852d5b828209..97b2167f128cbefe 100644
@@ -715,78 +715,6 @@ SYM_DATA_START(boot_idt)
 	.endr
 SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end)
 
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-SYM_DATA_START(boot32_idt_desc)
-	.word   boot32_idt_end - boot32_idt - 1
-	.long   0
-SYM_DATA_END(boot32_idt_desc)
-	.balign 8
-SYM_DATA_START(boot32_idt)
-	.rept 32
-	.quad 0
-	.endr
-SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
-
-	.text
-	.code32
-/*
- * Write an IDT entry into boot32_idt
- *
- * Parameters:
- *
- * %eax:	Handler address
- * %edx:	Vector number
- * %ecx:	IDT address
- */
-SYM_FUNC_START_LOCAL(startup32_set_idt_entry)
-	/* IDT entry address to %ecx */
-	leal	(%ecx, %edx, 8), %ecx
-
-	/* Build IDT entry, lower 4 bytes */
-	movl    %eax, %edx
-	andl    $0x0000ffff, %edx		# Target code segment offset [15:0]
-	orl	$(__KERNEL32_CS << 16), %edx	# Target code segment selector
-
-	/* Store lower 4 bytes to IDT */
-	movl    %edx, (%ecx)
-
-	/* Build IDT entry, upper 4 bytes */
-	movl    %eax, %edx
-	andl    $0xffff0000, %edx	# Target code segment offset [31:16]
-	orl     $0x00008e00, %edx	# Present, Type 32-bit Interrupt Gate
-
-	/* Store upper 4 bytes to IDT */
-	movl    %edx, 4(%ecx)
-
-	RET
-SYM_FUNC_END(startup32_set_idt_entry)
-
-SYM_FUNC_START(startup32_load_idt)
-	push	%ebp
-	push	%ebx
-
-	call	1f
-1:	pop	%ebp
-
-	leal    (boot32_idt - 1b)(%ebp), %ebx
-
-	/* #VC handler */
-	leal    (startup32_vc_handler - 1b)(%ebp), %eax
-	movl    $X86_TRAP_VC, %edx
-	movl	%ebx, %ecx
-	call    startup32_set_idt_entry
-
-	/* Load IDT */
-	leal	(boot32_idt_desc - 1b)(%ebp), %ecx
-	movl	%ebx, 2(%ecx)
-	lidt    (%ecx)
-
-	pop	%ebx
-	pop	%ebp
-	RET
-SYM_FUNC_END(startup32_load_idt)
-#endif
-
 /*
  * Check for the correct C-bit position when the startup_32 boot-path is used.
  *
arch/x86/boot/compressed/mem_encrypt.S +71 -1

94 lines not on this page · raw message

Signature · 1 line
2.35.1
merge bot Applied
#24 lore raw

[tip: x86/boot] x86/boot/compressed: Move startup32_load_idt() out of head_64.S

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     9ea813be3d345dfb8ac5bf6fbb29e6a63647a39d
Gitweb:        https://git.kernel.org/tip/9ea813be3d345dfb8ac5bf6fbb29e6a63647a39d
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:12 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00

x86/boot/compressed: Move startup32_load_idt() out of head_64.S

Now that startup32_load_idt() has been refactored into an ordinary callable function, move it into mem-encrypt.S where it belongs.

---
 arch/x86/boot/compressed/head_64.S     | 72 +-------------------------
 arch/x86/boot/compressed/mem_encrypt.S | 72 ++++++++++++++++++++++++-
 2 files changed, 71 insertions(+), 73 deletions(-)
arch/x86/boot/compressed/head_64.S +0 -72
index 7aa147f..16cccc2 100644
@@ -707,78 +707,6 @@ SYM_DATA_START(boot_idt)
 	.endr
 SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end)
 
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-SYM_DATA_START(boot32_idt_desc)
-	.word   boot32_idt_end - boot32_idt - 1
-	.long   0
-SYM_DATA_END(boot32_idt_desc)
-	.balign 8
-SYM_DATA_START(boot32_idt)
-	.rept 32
-	.quad 0
-	.endr
-SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
-
-	.text
-	.code32
-/*
- * Write an IDT entry into boot32_idt
- *
- * Parameters:
- *
- * %eax:	Handler address
- * %edx:	Vector number
- * %ecx:	IDT address
- */
-SYM_FUNC_START_LOCAL(startup32_set_idt_entry)
-	/* IDT entry address to %ecx */
-	leal	(%ecx, %edx, 8), %ecx
-
-	/* Build IDT entry, lower 4 bytes */
-	movl    %eax, %edx
-	andl    $0x0000ffff, %edx		# Target code segment offset [15:0]
-	orl	$(__KERNEL32_CS << 16), %edx	# Target code segment selector
-
-	/* Store lower 4 bytes to IDT */
-	movl    %edx, (%ecx)
-
-	/* Build IDT entry, upper 4 bytes */
-	movl    %eax, %edx
-	andl    $0xffff0000, %edx	# Target code segment offset [31:16]
-	orl     $0x00008e00, %edx	# Present, Type 32-bit Interrupt Gate
-
-	/* Store upper 4 bytes to IDT */
-	movl    %edx, 4(%ecx)
-
-	RET
-SYM_FUNC_END(startup32_set_idt_entry)
-
-SYM_FUNC_START(startup32_load_idt)
-	push	%ebp
-	push	%ebx
-
-	call	1f
-1:	pop	%ebp
-
-	leal    (boot32_idt - 1b)(%ebp), %ebx
-
-	/* #VC handler */
-	leal    (startup32_vc_handler - 1b)(%ebp), %eax
-	movl    $X86_TRAP_VC, %edx
-	movl	%ebx, %ecx
-	call    startup32_set_idt_entry
-
-	/* Load IDT */
-	leal	(boot32_idt_desc - 1b)(%ebp), %ecx
-	movl	%ebx, 2(%ecx)
-	lidt    (%ecx)
-
-	pop	%ebx
-	pop	%ebp
-	RET
-SYM_FUNC_END(startup32_load_idt)
-#endif
-
 /*
  * Check for the correct C-bit position when the startup_32 boot-path is used.
  *
arch/x86/boot/compressed/mem_encrypt.S +71 -1

94 lines not on this page · raw message

author maintainer
#25 lore raw

PATCH v3 13/17 x86/compressed: move startup32_check_sev_cbit() into .text

In reply to Ard Biesheuvel

Move startup32_check_sev_cbit() into the .text section and turn it into an ordinary function using the ordinary 32-bit calling convention, instead of saving/restoring the registers that are known to be live at the only call site. This improves maintainability, and makes it possible to move this function out of head_64.S and into a separate compilation unit that is specific to memory encryption.

Note that this requires the call site to be moved before the mixed mode check, as %eax will be live otherwise.

---
 arch/x86/boot/compressed/head_64.S | 35 +++++++++++---------
 1 file changed, 19 insertions(+), 16 deletions(-)
arch/x86/boot/compressed/head_64.S +19 -16
index 97b2167f128cbefe..272b2e97456f0dcf 100644
@@ -259,6 +259,11 @@ SYM_FUNC_START(startup_32)
 	movl    $__BOOT_TSS, %eax
 	ltr	%ax
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+	/* Check if the C-bit position is correct when SEV is active */
+	call	startup32_check_sev_cbit
+#endif
+
 	/*
 	 * Setup for the jump to 64bit mode
 	 *
@@ -276,8 +281,6 @@ SYM_FUNC_START(startup_32)
 	leal	rva(startup_64_mixed_mode)(%ebp), %eax
 1:
 #endif
-	/* Check if the C-bit position is correct when SEV is active */
-	call	startup32_check_sev_cbit
 
 	pushl	$__KERNEL_CS
 	pushl	%eax
@@ -732,16 +735,17 @@ SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end)
  * succeed. An incorrect C-bit position will map all memory unencrypted, so that
  * the compare will use the encrypted random data and fail.
  */
-	__HEAD
-SYM_FUNC_START(startup32_check_sev_cbit)
 #ifdef CONFIG_AMD_MEM_ENCRYPT
-	pushl	%eax
+	.text
+SYM_FUNC_START(startup32_check_sev_cbit)
 	pushl	%ebx
-	pushl	%ecx
-	pushl	%edx
+	pushl	%ebp
+
+	call	0f
+0:	popl	%ebp
 
 	/* Check for non-zero sev_status */
-	movl	rva(sev_status)(%ebp), %eax
+	movl	(sev_status - 0b)(%ebp), %eax
 	testl	%eax, %eax
 	jz	4f
 
@@ -756,17 +760,18 @@ SYM_FUNC_START(startup32_check_sev_cbit)
 	jnc	2b
 
 	/* Store to memory and keep it in the registers */
-	movl	%eax, rva(sev_check_data)(%ebp)
-	movl	%ebx, rva(sev_check_data+4)(%ebp)
+	leal	(sev_check_data - 0b)(%ebp), %ebp
+	movl	%eax, 0(%ebp)
+	movl	%ebx, 4(%ebp)
 
 	/* Enable paging to see if encryption is active */
 	movl	%cr0, %edx			 /* Backup %cr0 in %edx */
 	movl	$(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */
 	movl	%ecx, %cr0
 
-	cmpl	%eax, rva(sev_check_data)(%ebp)
+	cmpl	%eax, 0(%ebp)
 	jne	3f
-	cmpl	%ebx, rva(sev_check_data+4)(%ebp)
+	cmpl	%ebx, 4(%ebp)
 	jne	3f
 
 	movl	%edx, %cr0	/* Restore previous %cr0 */
@@ -778,13 +783,11 @@ SYM_FUNC_START(startup32_check_sev_cbit)
 	jmp	3b
 
 4:
-	popl	%edx
-	popl	%ecx
+	popl	%ebp
 	popl	%ebx
-	popl	%eax
-#endif
 	RET
 SYM_FUNC_END(startup32_check_sev_cbit)
+#endif
 
 /*
  * Stack and heap for uncompression
Signature · 1 line
2.35.1
merge bot Applied
#26 lore raw

[tip: x86/boot] x86/boot/compressed: Move startup32_check_sev_cbit() into .text

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     b5d854cd4b6a314edd6c15dabc4233b84a0f8e5e
Gitweb:        https://git.kernel.org/tip/b5d854cd4b6a314edd6c15dabc4233b84a0f8e5e
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:13 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00

x86/boot/compressed: Move startup32_check_sev_cbit() into .text

Move startup32_check_sev_cbit() into the .text section and turn it into an ordinary function using the ordinary 32-bit calling convention, instead of saving/restoring the registers that are known to be live at the only call site. This improves maintainability, and makes it possible to move this function out of head_64.S and into a separate compilation unit that is specific to memory encryption.

Note that this requires the call site to be moved before the mixed mode check, as %eax will be live otherwise.

---
 arch/x86/boot/compressed/head_64.S | 35 +++++++++++++++--------------
 1 file changed, 19 insertions(+), 16 deletions(-)
arch/x86/boot/compressed/head_64.S +19 -16
index 16cccc2..30ba541 100644
@@ -251,6 +251,11 @@ SYM_FUNC_START(startup_32)
 	movl    $__BOOT_TSS, %eax
 	ltr	%ax
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+	/* Check if the C-bit position is correct when SEV is active */
+	call	startup32_check_sev_cbit
+#endif
+
 	/*
 	 * Setup for the jump to 64bit mode
 	 *
@@ -268,8 +273,6 @@ SYM_FUNC_START(startup_32)
 	leal	rva(startup_64_mixed_mode)(%ebp), %eax
 1:
 #endif
-	/* Check if the C-bit position is correct when SEV is active */
-	call	startup32_check_sev_cbit
 
 	pushl	$__KERNEL_CS
 	pushl	%eax
@@ -724,16 +727,17 @@ SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end)
  * succeed. An incorrect C-bit position will map all memory unencrypted, so that
  * the compare will use the encrypted random data and fail.
  */
-	__HEAD
-SYM_FUNC_START(startup32_check_sev_cbit)
 #ifdef CONFIG_AMD_MEM_ENCRYPT
-	pushl	%eax
+	.text
+SYM_FUNC_START(startup32_check_sev_cbit)
 	pushl	%ebx
-	pushl	%ecx
-	pushl	%edx
+	pushl	%ebp
+
+	call	0f
+0:	popl	%ebp
 
 	/* Check for non-zero sev_status */
-	movl	rva(sev_status)(%ebp), %eax
+	movl	(sev_status - 0b)(%ebp), %eax
 	testl	%eax, %eax
 	jz	4f
 
@@ -748,17 +752,18 @@ SYM_FUNC_START(startup32_check_sev_cbit)
 	jnc	2b
 
 	/* Store to memory and keep it in the registers */
-	movl	%eax, rva(sev_check_data)(%ebp)
-	movl	%ebx, rva(sev_check_data+4)(%ebp)
+	leal	(sev_check_data - 0b)(%ebp), %ebp
+	movl	%eax, 0(%ebp)
+	movl	%ebx, 4(%ebp)
 
 	/* Enable paging to see if encryption is active */
 	movl	%cr0, %edx			 /* Backup %cr0 in %edx */
 	movl	$(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */
 	movl	%ecx, %cr0
 
-	cmpl	%eax, rva(sev_check_data)(%ebp)
+	cmpl	%eax, 0(%ebp)
 	jne	3f
-	cmpl	%ebx, rva(sev_check_data+4)(%ebp)
+	cmpl	%ebx, 4(%ebp)
 	jne	3f
 
 	movl	%edx, %cr0	/* Restore previous %cr0 */
@@ -770,13 +775,11 @@ SYM_FUNC_START(startup32_check_sev_cbit)
 	jmp	3b
 
 4:
-	popl	%edx
-	popl	%ecx
+	popl	%ebp
 	popl	%ebx
-	popl	%eax
-#endif
 	RET
 SYM_FUNC_END(startup32_check_sev_cbit)
+#endif
 
 /*
  * Stack and heap for uncompression
author maintainer
#27 lore raw

PATCH v3 14/17 x86/compressed: move startup32_check_sev_cbit() out of head_64.S

In reply to Ard Biesheuvel

Now that the startup32_check_sev_cbit() routine can execute from anywhere and behaves like an ordinary function, we no longer need to keep it in head_64.S.

---
 arch/x86/boot/compressed/head_64.S     | 71 --------------------
 arch/x86/boot/compressed/mem_encrypt.S | 68 +++++++++++++++++++
 2 files changed, 68 insertions(+), 71 deletions(-)
arch/x86/boot/compressed/head_64.S +0 -71
index 272b2e97456f0dcf..0cfc8ce273a2731c 100644
@@ -718,77 +718,6 @@ SYM_DATA_START(boot_idt)
 	.endr
 SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end)
 
-/*
- * Check for the correct C-bit position when the startup_32 boot-path is used.
- *
- * The check makes use of the fact that all memory is encrypted when paging is
- * disabled. The function creates 64 bits of random data using the RDRAND
- * instruction. RDRAND is mandatory for SEV guests, so always available. If the
- * hypervisor violates that the kernel will crash right here.
- *
- * The 64 bits of random data are stored to a memory location and at the same
- * time kept in the %eax and %ebx registers. Since encryption is always active
- * when paging is off the random data will be stored encrypted in main memory.
- *
- * Then paging is enabled. When the C-bit position is correct all memory is
- * still mapped encrypted and comparing the register values with memory will
- * succeed. An incorrect C-bit position will map all memory unencrypted, so that
- * the compare will use the encrypted random data and fail.
- */
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-	.text
-SYM_FUNC_START(startup32_check_sev_cbit)
-	pushl	%ebx
-	pushl	%ebp
-
-	call	0f
-0:	popl	%ebp
-
-	/* Check for non-zero sev_status */
-	movl	(sev_status - 0b)(%ebp), %eax
-	testl	%eax, %eax
-	jz	4f
-
-	/*
-	 * Get two 32-bit random values - Don't bail out if RDRAND fails
-	 * because it is better to prevent forward progress if no random value
-	 * can be gathered.
-	 */
-1:	rdrand	%eax
-	jnc	1b
-2:	rdrand	%ebx
-	jnc	2b
-
-	/* Store to memory and keep it in the registers */
-	leal	(sev_check_data - 0b)(%ebp), %ebp
-	movl	%eax, 0(%ebp)
-	movl	%ebx, 4(%ebp)
-
-	/* Enable paging to see if encryption is active */
-	movl	%cr0, %edx			 /* Backup %cr0 in %edx */
-	movl	$(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */
-	movl	%ecx, %cr0
-
-	cmpl	%eax, 0(%ebp)
-	jne	3f
-	cmpl	%ebx, 4(%ebp)
-	jne	3f
-
-	movl	%edx, %cr0	/* Restore previous %cr0 */
-
-	jmp	4f
-
-3:	/* Check failed - hlt the machine */
-	hlt
-	jmp	3b
-
-4:
-	popl	%ebp
-	popl	%ebx
-	RET
-SYM_FUNC_END(startup32_check_sev_cbit)
-#endif
-
 /*
  * Stack and heap for uncompression
  */
arch/x86/boot/compressed/mem_encrypt.S +68 -0

74 lines not on this page · raw message

Signature · 1 line
2.35.1
merge bot Applied
#28 lore raw

[tip: x86/boot] x86/boot/compressed: Move startup32_check_sev_cbit() out of head_64.S

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     9d7eaae6a071ff1f718e0aa5e610bb712f8cc632
Gitweb:        https://git.kernel.org/tip/9d7eaae6a071ff1f718e0aa5e610bb712f8cc632
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:14 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00

x86/boot/compressed: Move startup32_check_sev_cbit() out of head_64.S

Now that the startup32_check_sev_cbit() routine can execute from anywhere and behaves like an ordinary function, it can be moved where it belongs.

---
 arch/x86/boot/compressed/head_64.S     | 71 +-------------------------
 arch/x86/boot/compressed/mem_encrypt.S | 68 ++++++++++++++++++++++++-
 2 files changed, 68 insertions(+), 71 deletions(-)
arch/x86/boot/compressed/head_64.S +0 -71
index 30ba541..db577fb 100644
@@ -711,77 +711,6 @@ SYM_DATA_START(boot_idt)
 SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end)
 
 /*
- * Check for the correct C-bit position when the startup_32 boot-path is used.
- *
- * The check makes use of the fact that all memory is encrypted when paging is
- * disabled. The function creates 64 bits of random data using the RDRAND
- * instruction. RDRAND is mandatory for SEV guests, so always available. If the
- * hypervisor violates that the kernel will crash right here.
- *
- * The 64 bits of random data are stored to a memory location and at the same
- * time kept in the %eax and %ebx registers. Since encryption is always active
- * when paging is off the random data will be stored encrypted in main memory.
- *
- * Then paging is enabled. When the C-bit position is correct all memory is
- * still mapped encrypted and comparing the register values with memory will
- * succeed. An incorrect C-bit position will map all memory unencrypted, so that
- * the compare will use the encrypted random data and fail.
- */
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-	.text
-SYM_FUNC_START(startup32_check_sev_cbit)
-	pushl	%ebx
-	pushl	%ebp
-
-	call	0f
-0:	popl	%ebp
-
-	/* Check for non-zero sev_status */
-	movl	(sev_status - 0b)(%ebp), %eax
-	testl	%eax, %eax
-	jz	4f
-
-	/*
-	 * Get two 32-bit random values - Don't bail out if RDRAND fails
-	 * because it is better to prevent forward progress if no random value
-	 * can be gathered.
-	 */
-1:	rdrand	%eax
-	jnc	1b
-2:	rdrand	%ebx
-	jnc	2b
-
-	/* Store to memory and keep it in the registers */
-	leal	(sev_check_data - 0b)(%ebp), %ebp
-	movl	%eax, 0(%ebp)
-	movl	%ebx, 4(%ebp)
-
-	/* Enable paging to see if encryption is active */
-	movl	%cr0, %edx			 /* Backup %cr0 in %edx */
-	movl	$(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */
-	movl	%ecx, %cr0
-
-	cmpl	%eax, 0(%ebp)
-	jne	3f
-	cmpl	%ebx, 4(%ebp)
-	jne	3f
-
-	movl	%edx, %cr0	/* Restore previous %cr0 */
-
-	jmp	4f
-
-3:	/* Check failed - hlt the machine */
-	hlt
-	jmp	3b
-
-4:
-	popl	%ebp
-	popl	%ebx
-	RET
-SYM_FUNC_END(startup32_check_sev_cbit)
-#endif
-
-/*
  * Stack and heap for uncompression
  */
 	.bss
arch/x86/boot/compressed/mem_encrypt.S +68 -0

74 lines not on this page · raw message

author maintainer
#29 lore raw

PATCH v3 15/17 x86/compressed: adhere to calling convention in get_sev_encryption_bit()

In reply to Ard Biesheuvel

Make get_sev_encryption_bit() follow the ordinary i386 calling convention, and only call it if CONFIG_AMD_MEM_ENCRYPT is actually enabled. This clarifies the calling code, and makes it more maintainable.

---
 arch/x86/boot/compressed/head_64.S     |  5 +++--
 arch/x86/boot/compressed/mem_encrypt.S | 10 ----------
 2 files changed, 3 insertions(+), 12 deletions(-)
arch/x86/boot/compressed/head_64.S +3 -2
index 0cfc8ce273a2731c..dd18216cff5c37e0 100644
@@ -188,12 +188,13 @@ SYM_FUNC_START(startup_32)
   */
 	/*
 	 * If SEV is active then set the encryption mask in the page tables.
-	 * This will insure that when the kernel is copied and decompressed
+	 * This will ensure that when the kernel is copied and decompressed
 	 * it will be done so encrypted.
 	 */
-	call	get_sev_encryption_bit
 	xorl	%edx, %edx
 #ifdef	CONFIG_AMD_MEM_ENCRYPT
+	call	get_sev_encryption_bit
+	xorl	%edx, %edx
 	testl	%eax, %eax
 	jz	1f
 	subl	$32, %eax	/* Encryption bit is always above bit 31 */
arch/x86/boot/compressed/mem_encrypt.S +0 -10
index 14cf04a1ed091655..e69674588a31c81f 100644
@@ -18,12 +18,7 @@
 	.text
 	.code32
 SYM_FUNC_START(get_sev_encryption_bit)
-	xor	%eax, %eax
-
-#ifdef CONFIG_AMD_MEM_ENCRYPT
 	push	%ebx
-	push	%ecx
-	push	%edx
 
 	movl	$0x80000000, %eax	/* CPUID to check the highest leaf */
 	cpuid
@@ -54,12 +49,7 @@ SYM_FUNC_START(get_sev_encryption_bit)
 	xor	%eax, %eax
 
 .Lsev_exit:
-	pop	%edx
-	pop	%ecx
 	pop	%ebx
-
-#endif	/* CONFIG_AMD_MEM_ENCRYPT */
-
 	RET
 SYM_FUNC_END(get_sev_encryption_bit)
 
Signature · 1 line
2.35.1
merge bot Applied
#30 lore raw

[tip: x86/boot] x86/boot/compressed: Adhere to calling convention in get_sev_encryption_bit()

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     30c9ca16a5271ba6f8ad9c86507ff1c789c94677
Gitweb:        https://git.kernel.org/tip/30c9ca16a5271ba6f8ad9c86507ff1c789c94677
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:15 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00

x86/boot/compressed: Adhere to calling convention in get_sev_encryption_bit()

Make get_sev_encryption_bit() follow the ordinary i386 calling convention, and only call it if CONFIG_AMD_MEM_ENCRYPT is actually enabled. This clarifies the calling code, and makes it more maintainable.

---
 arch/x86/boot/compressed/head_64.S     |  5 +++--
 arch/x86/boot/compressed/mem_encrypt.S | 10 ----------
 2 files changed, 3 insertions(+), 12 deletions(-)
arch/x86/boot/compressed/head_64.S +3 -2
index db577fb..6ba2c21 100644
@@ -180,12 +180,13 @@ SYM_FUNC_START(startup_32)
   */
 	/*
 	 * If SEV is active then set the encryption mask in the page tables.
-	 * This will insure that when the kernel is copied and decompressed
+	 * This will ensure that when the kernel is copied and decompressed
 	 * it will be done so encrypted.
 	 */
-	call	get_sev_encryption_bit
 	xorl	%edx, %edx
 #ifdef	CONFIG_AMD_MEM_ENCRYPT
+	call	get_sev_encryption_bit
+	xorl	%edx, %edx
 	testl	%eax, %eax
 	jz	1f
 	subl	$32, %eax	/* Encryption bit is always above bit 31 */
arch/x86/boot/compressed/mem_encrypt.S +0 -10
index 14cf04a..e696745 100644
@@ -18,12 +18,7 @@
 	.text
 	.code32
 SYM_FUNC_START(get_sev_encryption_bit)
-	xor	%eax, %eax
-
-#ifdef CONFIG_AMD_MEM_ENCRYPT
 	push	%ebx
-	push	%ecx
-	push	%edx
 
 	movl	$0x80000000, %eax	/* CPUID to check the highest leaf */
 	cpuid
@@ -54,12 +49,7 @@ SYM_FUNC_START(get_sev_encryption_bit)
 	xor	%eax, %eax
 
 .Lsev_exit:
-	pop	%edx
-	pop	%ecx
 	pop	%ebx
-
-#endif	/* CONFIG_AMD_MEM_ENCRYPT */
-
 	RET
 SYM_FUNC_END(get_sev_encryption_bit)
 
author maintainer
#31 lore raw

PATCH v3 16/17 x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y

In reply to Ard Biesheuvel

Avoid building the mem_encrypt.o object if memory encryption support is not enabled to begin with.

---
 arch/x86/boot/compressed/Makefile      | 2 +-
 arch/x86/boot/compressed/mem_encrypt.S | 2 --
 2 files changed, 1 insertion(+), 3 deletions(-)
arch/x86/boot/compressed/Makefile +1 -1
index cd1a6295b957ea1d..3dc5db651dd0a197 100644
@@ -100,7 +100,7 @@ vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/kaslr.o
 ifdef CONFIG_X86_64
 	vmlinux-objs-y += $(obj)/ident_map_64.o
 	vmlinux-objs-y += $(obj)/idt_64.o $(obj)/idt_handlers_64.o
-	vmlinux-objs-y += $(obj)/mem_encrypt.o
+	vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/mem_encrypt.o
 	vmlinux-objs-y += $(obj)/pgtable_64.o
 	vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/sev.o
 endif
arch/x86/boot/compressed/mem_encrypt.S +0 -2
index e69674588a31c81f..32f7cc8a862547e4 100644
@@ -307,7 +307,6 @@ SYM_FUNC_END(startup32_check_sev_cbit)
 
 	.data
 
-#ifdef CONFIG_AMD_MEM_ENCRYPT
 	.balign	8
 SYM_DATA(sme_me_mask,		.quad 0)
 SYM_DATA(sev_status,		.quad 0)
@@ -323,4 +322,3 @@ SYM_DATA_START_LOCAL(boot32_idt_desc)
 	.word	. - boot32_idt - 1
 	.long	0
 SYM_DATA_END(boot32_idt_desc)
-#endif
Signature · 1 line
2.35.1
merge bot Applied
#32 lore raw

[tip: x86/boot] x86/boot/compressed: Only build mem_encrypt.S if AMD_MEM_ENCRYPT=y

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     61de13df95901bc58456bc5acdbd3c18c66cf859
Gitweb:        https://git.kernel.org/tip/61de13df95901bc58456bc5acdbd3c18c66cf859
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:16 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00

x86/boot/compressed: Only build mem_encrypt.S if AMD_MEM_ENCRYPT=y

Avoid building the mem_encrypt.o object if memory encryption support is not enabled to begin with.

---
 arch/x86/boot/compressed/Makefile      | 2 +-
 arch/x86/boot/compressed/mem_encrypt.S | 2 --
 2 files changed, 1 insertion(+), 3 deletions(-)
arch/x86/boot/compressed/Makefile +1 -1
index cd1a629..3dc5db6 100644
@@ -100,7 +100,7 @@ vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/kaslr.o
 ifdef CONFIG_X86_64
 	vmlinux-objs-y += $(obj)/ident_map_64.o
 	vmlinux-objs-y += $(obj)/idt_64.o $(obj)/idt_handlers_64.o
-	vmlinux-objs-y += $(obj)/mem_encrypt.o
+	vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/mem_encrypt.o
 	vmlinux-objs-y += $(obj)/pgtable_64.o
 	vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/sev.o
 endif
arch/x86/boot/compressed/mem_encrypt.S +0 -2
index e696745..32f7cc8 100644
@@ -307,7 +307,6 @@ SYM_FUNC_END(startup32_check_sev_cbit)
 
 	.data
 
-#ifdef CONFIG_AMD_MEM_ENCRYPT
 	.balign	8
 SYM_DATA(sme_me_mask,		.quad 0)
 SYM_DATA(sev_status,		.quad 0)
@@ -323,4 +322,3 @@ SYM_DATA_START_LOCAL(boot32_idt_desc)
 	.word	. - boot32_idt - 1
 	.long	0
 SYM_DATA_END(boot32_idt_desc)
-#endif
author maintainer
#33 lore raw

PATCH v3 17/17 efi: x86: Make the deprecated EFI handover protocol optional

In reply to Ard Biesheuvel

The EFI handover protocol permits a bootloader to invoke the kernel as a EFI PE/COFF application, while passing a bootparams struct as a third argument to the entrypoint function call.

This has no basis in the UEFI specification, and there are better ways to pass additional data to a UEFI application (UEFI configuration tables, UEFI variables, UEFI protocols) than going around the StartImage() boot service and jumping to a fixed offset in the loaded image, just to call a different function that takes a third parameter.

The reason for handling struct bootparams in the bootloader was that the EFI stub could only load initrd images from the EFI system partition, and so passing it via struct bootparams was needed for loaders like GRUB, which pass the initrd in memory, and may load it from anywhere, including from the network. Another motivation was EFI mixed mode, which could not use the initrd loader in the EFI stub at all due to 32/64 bit incompatibilities (which will be fixed shortly [0]), and could not invoke the ordinary PE/COFF entry point either, for the same reasons.

Given that loaders such as GRUB already carried the bootparams handling in order to implement non-EFI boot, retaining that code and just passing bootparams to the EFI stub was a reasonable choice (although defining an alternate entrypoint could have been avoided.) However, the GRUB side changes never made it upstream, and are only shipped by some of the distros in their downstream versions.

In the meantime, EFI support has been added to other Linux architecture ports, as well as to U-boot and systemd, including arch-agnostic methods for passing initrd images in memory [1], and for doing mixed mode boot [2], none of them requiring anything like the EFI handover protocol. So given that only out-of-tree distro GRUB relies on this, let's permit it to be omitted from the build, in preparation for retiring it completely at a later date. (Note that systemd-boot does have an implementation as well, but only uses it as a fallback for booting images that do not implement the LoadFile2 based initrd loading method, i.e., v5.8 or older)

[0] https://lore.kernel.org/all/[email protected]/
[1] ec93fc371f01 ("efi/libstub: Add support for loading the initrd ...")
[2] 97aa276579b2 ("efi/x86: Add true mixed mode entry point into ...")

---
 arch/x86/Kconfig                   | 17 +++++++++++++++++
 arch/x86/boot/compressed/head_64.S |  4 +++-
 arch/x86/boot/header.S             |  2 +-
 arch/x86/boot/tools/build.c        |  2 ++
 4 files changed, 23 insertions(+), 2 deletions(-)
arch/x86/Kconfig +17 -0
index 8c6da5e42d5a6c25..121f1fdca3145fd2 100644
@@ -1981,6 +1981,23 @@ config EFI_STUB
 
 	  See Documentation/admin-guide/efi-stub.rst for more information.
 
+config EFI_HANDOVER_PROTOCOL
+	bool "EFI handover protocol (DEPRECATED)"
+	depends on EFI_STUB
+	default y
+	help
+	  Select this in order to include support for the deprecated EFI
+	  handover protocol, which defines alternative entry points into the
+	  EFI stub.  This is a practice that has no basis in the UEFI
+	  specification, and requires a priori knowledge on the part of the
+	  bootloader about Linux/x86 specific ways of passing the command line
+	  and initrd, and where in memory those assets may be loaded.
+
+	  If in doubt, say Y. Even though he corresponding support is not
+	  present in upstream GRUB or other bootloaders, most distros build
+	  GRUB with numerous downstream patches applied, and may rely on the
+	  handover protocol as as result.
+
 config EFI_MIXED
 	bool "EFI mixed-mode support"
 	depends on EFI_STUB && X86_64
arch/x86/boot/compressed/head_64.S +3 -1
index dd18216cff5c37e0..a75712991df3e936 100644
@@ -294,7 +294,7 @@ SYM_FUNC_START(startup_32)
 	lret
 SYM_FUNC_END(startup_32)
 
-#ifdef CONFIG_EFI_MIXED
+#if IS_ENABLED(CONFIG_EFI_MIXED) && IS_ENABLED(CONFIG_EFI_HANDOVER_PROTOCOL)
 	.org 0x190
 SYM_FUNC_START(efi32_stub_entry)
 	add	$0x4, %esp		/* Discard return address */
@@ -524,7 +524,9 @@ trampoline_return:
 SYM_CODE_END(startup_64)
 
 #ifdef CONFIG_EFI_STUB
+#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
 	.org 0x390
+#endif
 SYM_FUNC_START(efi64_stub_entry)
 	and	$~0xf, %rsp			/* realign the stack */
 	movq	%rdx, %rbx			/* save boot_params pointer */
arch/x86/boot/header.S +1 -1
index f912d777013052ea..d31982509654dcb1 100644
@@ -406,7 +406,7 @@ xloadflags:
 # define XLF1 0
 #endif
 
-#ifdef CONFIG_EFI_STUB
+#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
 # ifdef CONFIG_EFI_MIXED
 #  define XLF23 (XLF_EFI_HANDOVER_32|XLF_EFI_HANDOVER_64)
 # else
arch/x86/boot/tools/build.c +2 -0
index a3725ad46c5a0b49..bd247692b70174f0 100644
@@ -290,6 +290,7 @@ static void efi_stub_entry_update(void)
 {
 	unsigned long addr = efi32_stub_entry;
 
+#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
 #ifdef CONFIG_X86_64
 	/* Yes, this is really how we defined it :( */
 	addr = efi64_stub_entry - 0x200;
@@ -298,6 +299,7 @@ static void efi_stub_entry_update(void)
 #ifdef CONFIG_EFI_MIXED
 	if (efi32_stub_entry != addr)
 		die("32-bit and 64-bit EFI entry points do not match\n");
+#endif
 #endif
 	put_unaligned_le32(addr, &buf[0x264]);
 }
Signature · 1 line
2.35.1
#34 lore raw

Re: [PATCH v3 17/17] efi: x86: Make the deprecated EFI handover protocol optional

Quoted 11 lines · Ard Biesheuvel
arch/x86/Kconfig +3 -0
index 8c6da5e42d5a6c25..121f1fdca3145fd2 100644
@@ -1981,6 +1981,23 @@ config EFI_STUB
 
 	  See Documentation/admin-guide/efi-stub.rst for more information.
 
+config EFI_HANDOVER_PROTOCOL
+	bool "EFI handover protocol (DEPRECATED)"
+	depends on EFI_STUB

arch/x86/Kconfig config EFI_STUB · line 1996

Quoted 10 lines
arch/x86/Kconfig +10 -0
+	default y
+	help
+	  Select this in order to include support for the deprecated EFI
+	  handover protocol, which defines alternative entry points into the
+	  EFI stub.  This is a practice that has no basis in the UEFI
+	  specification, and requires a priori knowledge on the part of the
+	  bootloader about Linux/x86 specific ways of passing the command line
+	  and initrd, and where in memory those assets may be loaded.
+
+	  If in doubt, say Y. Even though he corresponding support is not

the

Quoted 3 lines
arch/x86/Kconfig +3 -0
+	  present in upstream GRUB or other bootloaders, most distros build
+	  GRUB with numerous downstream patches applied, and may rely on the
+	  handover protocol as as result.
Signature · 1 line
~Randy
merge bot Applied
#35 lore raw

[tip: x86/boot] x86/efi: Make the deprecated EFI handover protocol optional

In reply to Ard Biesheuvel
The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     cc3fdda2876e58a7e83e558ab51853cf106afb6a
Gitweb:        https://git.kernel.org/tip/cc3fdda2876e58a7e83e558ab51853cf106afb6a
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Tue, 22 Nov 2022 17:10:17 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00

x86/efi: Make the deprecated EFI handover protocol optional

The EFI handover protocol permits a bootloader to invoke the kernel as a EFI PE/COFF application, while passing a bootparams struct as a third argument to the entrypoint function call.

This has no basis in the UEFI specification, and there are better ways to pass additional data to a UEFI application (UEFI configuration tables, UEFI variables, UEFI protocols) than going around the StartImage() boot service and jumping to a fixed offset in the loaded image, just to call a different function that takes a third parameter.

The reason for handling struct bootparams in the bootloader was that the EFI stub could only load initrd images from the EFI system partition, and so passing it via struct bootparams was needed for loaders like GRUB, which pass the initrd in memory, and may load it from anywhere, including from the network. Another motivation was EFI mixed mode, which could not use the initrd loader in the EFI stub at all due to 32/64 bit incompatibilities (which will be fixed shortly [0]), and could not invoke the ordinary PE/COFF entry point either, for the same reasons.

Given that loaders such as GRUB already carried the bootparams handling in order to implement non-EFI boot, retaining that code and just passing bootparams to the EFI stub was a reasonable choice (although defining an alternate entrypoint could have been avoided.) However, the GRUB side changes never made it upstream, and are only shipped by some of the distros in their downstream versions.

In the meantime, EFI support has been added to other Linux architecture ports, as well as to U-boot and systemd, including arch-agnostic methods for passing initrd images in memory [1], and for doing mixed mode boot [2], none of them requiring anything like the EFI handover protocol. So given that only out-of-tree distro GRUB relies on this, let's permit it to be omitted from the build, in preparation for retiring it completely at a later date. (Note that systemd-boot does have an implementation as well, but only uses it as a fallback for booting images that do not implement the LoadFile2 based initrd loading method, i.e., v5.8 or older)

[0] https://lore.kernel.org/all/[email protected]/
[1] ec93fc371f01 ("efi/libstub: Add support for loading the initrd from a device path")
[2] 97aa276579b2 ("efi/x86: Add true mixed mode entry point into .compat section")

---
 arch/x86/Kconfig                   | 17 +++++++++++++++++
 arch/x86/boot/compressed/head_64.S |  4 +++-
 arch/x86/boot/header.S             |  2 +-
 arch/x86/boot/tools/build.c        |  2 ++
 4 files changed, 23 insertions(+), 2 deletions(-)
arch/x86/Kconfig +17 -0
index 67745ce..a0fb836 100644
@@ -1980,6 +1980,23 @@ config EFI_STUB
 
 	  See Documentation/admin-guide/efi-stub.rst for more information.
 
+config EFI_HANDOVER_PROTOCOL
+	bool "EFI handover protocol (DEPRECATED)"
+	depends on EFI_STUB
+	default y
+	help
+	  Select this in order to include support for the deprecated EFI
+	  handover protocol, which defines alternative entry points into the
+	  EFI stub.  This is a practice that has no basis in the UEFI
+	  specification, and requires a priori knowledge on the part of the
+	  bootloader about Linux/x86 specific ways of passing the command line
+	  and initrd, and where in memory those assets may be loaded.
+
+	  If in doubt, say Y. Even though the corresponding support is not
+	  present in upstream GRUB or other bootloaders, most distros build
+	  GRUB with numerous downstream patches applied, and may rely on the
+	  handover protocol as as result.
+
 config EFI_MIXED
 	bool "EFI mixed-mode support"
 	depends on EFI_STUB && X86_64
arch/x86/boot/compressed/head_64.S +3 -1
index 6ba2c21..d4c4281 100644
@@ -286,7 +286,7 @@ SYM_FUNC_START(startup_32)
 	lret
 SYM_FUNC_END(startup_32)
 
-#ifdef CONFIG_EFI_MIXED
+#if IS_ENABLED(CONFIG_EFI_MIXED) && IS_ENABLED(CONFIG_EFI_HANDOVER_PROTOCOL)
 	.org 0x190
 SYM_FUNC_START(efi32_stub_entry)
 	add	$0x4, %esp		/* Discard return address */
@@ -516,7 +516,9 @@ trampoline_return:
 SYM_CODE_END(startup_64)
 
 #ifdef CONFIG_EFI_STUB
+#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
 	.org 0x390
+#endif
 SYM_FUNC_START(efi64_stub_entry)
 	and	$~0xf, %rsp			/* realign the stack */
 	movq	%rdx, %rbx			/* save boot_params pointer */
arch/x86/boot/header.S +1 -1
index f912d77..d319825 100644
@@ -406,7 +406,7 @@ xloadflags:
 # define XLF1 0
 #endif
 
-#ifdef CONFIG_EFI_STUB
+#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
 # ifdef CONFIG_EFI_MIXED
 #  define XLF23 (XLF_EFI_HANDOVER_32|XLF_EFI_HANDOVER_64)
 # else
arch/x86/boot/tools/build.c +2 -0
index a3725ad..bd24769 100644
@@ -290,6 +290,7 @@ static void efi_stub_entry_update(void)
 {
 	unsigned long addr = efi32_stub_entry;
 
+#ifdef CONFIG_EFI_HANDOVER_PROTOCOL
 #ifdef CONFIG_X86_64
 	/* Yes, this is really how we defined it :( */
 	addr = efi64_stub_entry - 0x200;
@@ -299,6 +300,7 @@ static void efi_stub_entry_update(void)
 	if (efi32_stub_entry != addr)
 		die("32-bit and 64-bit EFI entry points do not match\n");
 #endif
+#endif
 	put_unaligned_le32(addr, &buf[0x264]);
 }
 
#36 lore raw
In reply to Ard Biesheuvel
Quoted 14 lines · Ard Biesheuvel

After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.

Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/master

Changes since v1:
- at Boris's request, split the patches into smaller ones that are easier to review

Quoted 5 lines
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Michael Roth <[email protected]>

This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in arch/x86/boot/compressed/head_64.S of startup_64).

Here is the Qemu register dump:
RAX=00000000029cc260 RBX=ffffffffdd98c000 RCX=0000000000000010 RDX=0000000000000002 RSI=000000003dec1000 RDI=0000000000000000 RBP=ffffffffdb000000 RSP=ffffffffde36e000 R8 =000000003dec1410 R9 =000000003dec13fc R10=000000000000006c R11=0000000000000000 R12=0000000000000000 R13=0000000000000001 R14=0000000000000004 R15=000000003eacdf44 RIP=0000000002000263 RFL=00200002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 00000000 00000000
CS =0038 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA]
SS =0000 0000000000000000 00000000 00000000
DS =0000 0000000000000000 00000000 00000000
FS =0000 0000000000000000 00000000 00000000
GS =0000 0000000000000000 00000000 00000000
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT= 00000000029cc270 0000002f
IDT= 000000003f55e018 00000fff
CR0=80010033 CR2=ffffffffde36dff8 CR3=000000003fc01000 CR4=00000668 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000d00

Thanks,
Tom

Quoted 38 lines

Ard Biesheuvel (17):
x86/compressed: efi-mixed: rename efi_thunk_64.S to efi-mixed.S x86/compressed: efi-mixed: move 32-bit entrypoint code into .text
section
x86/compressed: efi-mixed: move bootargs parsing out of 32-bit startup
code
x86/compressed: efi-mixed: move efi32_pe_entry into .text section x86/compressed: efi-mixed: move efi32_entry out of head_64.S x86/compressed: efi-mixed: move efi32_pe_entry() out of head_64.S x86/compressed: efi: merge multiple definitions of image_offset into
one
x86/compressed: efi-mixed: simplify IDT/GDT preserve/restore x86/compressed: avoid touching ECX in startup32_set_idt_entry() x86/compressed: pull global variable ref up into startup32_load_idt() x86/compressed: move startup32_load_idt() into .text section x86/compressed: move startup32_load_idt() out of head_64.S x86/compressed: move startup32_check_sev_cbit() into .text x86/compressed: move startup32_check_sev_cbit() out of head_64.S x86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y
efi: x86: Make the deprecated EFI handover protocol optional

  arch/x86/Kconfig                        |  17 +
  arch/x86/boot/compressed/Makefile       |   8 +-
  arch/x86/boot/compressed/efi_mixed.S    | 351 ++++++++++++++++++++
  arch/x86/boot/compressed/efi_thunk_64.S | 195 -----------
  arch/x86/boot/compressed/head_32.S      |   4 -
  arch/x86/boot/compressed/head_64.S      | 303 +----------------
  arch/x86/boot/compressed/mem_encrypt.S  | 152 ++++++++-
  arch/x86/boot/header.S                  |   2 +-
  arch/x86/boot/tools/build.c             |   2 +
  drivers/firmware/efi/libstub/x86-stub.c |   2 +-
  10 files changed, 533 insertions(+), 503 deletions(-)

create mode 100644 arch/x86/boot/compressed/efi_mixed.S delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S

author maintainer
#37 lore raw
Quoted 22 lines · Tom Lendacky
Quoted 14 lines · Ard Biesheuvel

After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.

Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/master

Changes since v1:
- at Boris's request, split the patches into smaller ones that are easier to review

Quoted 5 lines
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Michael Roth <[email protected]>
Quoted 5 lines

This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in arch/x86/boot/compressed/head_64.S of startup_64).

Thanks for the report.

So the mystery here (at least to me) is that all the changes are to the 32-bit code, and startup_64 reloads the stack pointer from the symbol

Does your config have CONFIG_EFI_MIXED enabled?

Can I reproduce this fully emulated with QEMU? Or do I need a SEV host?

Quoted 62 lines

Here is the Qemu register dump:
RAX=00000000029cc260 RBX=ffffffffdd98c000 RCX=0000000000000010 RDX=0000000000000002 RSI=000000003dec1000 RDI=0000000000000000 RBP=ffffffffdb000000 RSP=ffffffffde36e000 R8 =000000003dec1410 R9 =000000003dec13fc R10=000000000000006c R11=0000000000000000 R12=0000000000000000 R13=0000000000000001 R14=0000000000000004 R15=000000003eacdf44 RIP=0000000002000263 RFL=00200002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 00000000 00000000
CS =0038 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA]
SS =0000 0000000000000000 00000000 00000000
DS =0000 0000000000000000 00000000 00000000
FS =0000 0000000000000000 00000000 00000000
GS =0000 0000000000000000 00000000 00000000
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT= 00000000029cc270 0000002f
IDT= 000000003f55e018 00000fff
CR0=80010033 CR2=ffffffffde36dff8 CR3=000000003fc01000 CR4=00000668 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000d00

Thanks,
Tom

Quoted 38 lines

Ard Biesheuvel (17):
x86/compressed: efi-mixed: rename efi_thunk_64.S to efi-mixed.S x86/compressed: efi-mixed: move 32-bit entrypoint code into .text
section
x86/compressed: efi-mixed: move bootargs parsing out of 32-bit startup
code
x86/compressed: efi-mixed: move efi32_pe_entry into .text section x86/compressed: efi-mixed: move efi32_entry out of head_64.S x86/compressed: efi-mixed: move efi32_pe_entry() out of head_64.S x86/compressed: efi: merge multiple definitions of image_offset into
one
x86/compressed: efi-mixed: simplify IDT/GDT preserve/restore x86/compressed: avoid touching ECX in startup32_set_idt_entry() x86/compressed: pull global variable ref up into startup32_load_idt() x86/compressed: move startup32_load_idt() into .text section x86/compressed: move startup32_load_idt() out of head_64.S x86/compressed: move startup32_check_sev_cbit() into .text x86/compressed: move startup32_check_sev_cbit() out of head_64.S x86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y
efi: x86: Make the deprecated EFI handover protocol optional

  arch/x86/Kconfig                        |  17 +
  arch/x86/boot/compressed/Makefile       |   8 +-
  arch/x86/boot/compressed/efi_mixed.S    | 351 ++++++++++++++++++++
  arch/x86/boot/compressed/efi_thunk_64.S | 195 -----------
  arch/x86/boot/compressed/head_32.S      |   4 -
  arch/x86/boot/compressed/head_64.S      | 303 +----------------
  arch/x86/boot/compressed/mem_encrypt.S  | 152 ++++++++-
  arch/x86/boot/header.S                  |   2 +-
  arch/x86/boot/tools/build.c             |   2 +
  drivers/firmware/efi/libstub/x86-stub.c |   2 +-
  10 files changed, 533 insertions(+), 503 deletions(-)

create mode 100644 arch/x86/boot/compressed/efi_mixed.S delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S

author maintainer
#38 lore raw
Quoted 32 lines · Ard Biesheuvel
Quoted 22 lines · Tom Lendacky
Quoted 14 lines · Ard Biesheuvel

After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.

Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/master

Changes since v1:
- at Boris's request, split the patches into smaller ones that are easier to review

Quoted 5 lines
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Michael Roth <[email protected]>
Quoted 5 lines

This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in arch/x86/boot/compressed/head_64.S of startup_64).

Thanks for the report.

Quoted 8 lines

So the mystery here (at least to me) is that all the changes are to the 32-bit code, and startup_64 reloads the stack pointer from the symbol

Does your config have CONFIG_EFI_MIXED enabled?

Can I reproduce this fully emulated with QEMU? Or do I need a SEV host?

Also, mind giving this a quick spin?

(quoted diff) +0 -0

b/drivers/firmware/efi/libstub/x86-stub.c
index cb5f0befee57..1af11d34bc6c 100644

drivers/firmware/efi/libstub/x86-stub.c +1 -1
@@ -23,7 +23,7 @@
 
 const efi_system_table_t *efi_system_table;
 const efi_dxe_services_table_t *efi_dxe_table;
-u32 image_offset;
+u32 __section(".data") image_offset;
 static efi_loaded_image_t *image = NULL;
 
 static efi_status_t

Thanks,
Ard.

#39 lore raw
Quoted 37 lines · Ard Biesheuvel
Quoted 36 lines · Ard Biesheuvel
Quoted 22 lines · Tom Lendacky
Quoted 14 lines · Ard Biesheuvel

After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.

Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/master

Changes since v1:
- at Boris's request, split the patches into smaller ones that are easier to review

Quoted 5 lines
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Michael Roth <[email protected]>
Quoted 5 lines

This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in arch/x86/boot/compressed/head_64.S of startup_64).

Thanks for the report.

So the mystery here (at least to me) is that all the changes are to the 32-bit code, and startup_64 reloads the stack pointer from the symbol

Quoted 6 lines
Quoted 4 lines

Does your config have CONFIG_EFI_MIXED enabled?

Can I reproduce this fully emulated with QEMU? Or do I need a SEV host?

Also, mind giving this a quick spin?

Just saw this after I sent out my email. Yes, this fixes it.

Thanks,
Tom

Quoted 18 lines
drivers/firmware/efi/libstub/x86-stub.c +1 -1
index cb5f0befee57..1af11d34bc6c 100644
@@ -23,7 +23,7 @@
 
  const efi_system_table_t *efi_system_table;
  const efi_dxe_services_table_t *efi_dxe_table;
-u32 image_offset;
+u32 __section(".data") image_offset;
  static efi_loaded_image_t *image = NULL;
 
  static efi_status_t

Thanks,
Ard.

author maintainer
#40 lore raw
Quoted 39 lines · Tom Lendacky
Quoted 37 lines · Ard Biesheuvel
Quoted 36 lines · Ard Biesheuvel
Quoted 22 lines · Tom Lendacky
Quoted 14 lines · Ard Biesheuvel

After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.

Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/master

Changes since v1:
- at Boris's request, split the patches into smaller ones that are easier to review

Quoted 5 lines
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Michael Roth <[email protected]>
Quoted 5 lines

This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in arch/x86/boot/compressed/head_64.S of startup_64).

Thanks for the report.

So the mystery here (at least to me) is that all the changes are to the 32-bit code, and startup_64 reloads the stack pointer from the symbol

Quoted 9 lines
Quoted 6 lines
Quoted 4 lines

Does your config have CONFIG_EFI_MIXED enabled?

Can I reproduce this fully emulated with QEMU? Or do I need a SEV host?

Also, mind giving this a quick spin?

Just saw this after I sent out my email. Yes, this fixes it.

Excellent, thanks for testing.

#41 lore raw
In reply to Ard Biesheuvel
Quoted 29 lines · Ard Biesheuvel
Quoted 22 lines · Tom Lendacky
Quoted 14 lines · Ard Biesheuvel

After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.

Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/master

Changes since v1:
- at Boris's request, split the patches into smaller ones that are easier to review

Quoted 5 lines
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Michael Roth <[email protected]>
Quoted 5 lines

This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in arch/x86/boot/compressed/head_64.S of startup_64).

Quoted 5 lines

Thanks for the report.

So the mystery here (at least to me) is that all the changes are to the 32-bit code, and startup_64 reloads the stack pointer from the symbol

I bisected it to:

99b7f4b23d9f ("x86/boot/compressed, efi: Merge multiple definitions of image_offset into one")

And doing the following fixed it:

drivers/firmware/efi/libstub/x86-stub.c +1 -1
index cb5f0befee57..a0bfd31358ba 100644
@@ -23,7 +23,7 @@
 
 const efi_system_table_t *efi_system_table;
 const efi_dxe_services_table_t *efi_dxe_table;
-u32 image_offset;
+u32 image_offset __section(".data");
 static efi_loaded_image_t *image = NULL;
 
 static efi_status_t

I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put image_offset in the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".

Thanks,
Tom

Quoted 67 lines

Does your config have CONFIG_EFI_MIXED enabled?

Can I reproduce this fully emulated with QEMU? Or do I need a SEV host?

Quoted 62 lines

Here is the Qemu register dump:
RAX=00000000029cc260 RBX=ffffffffdd98c000 RCX=0000000000000010 RDX=0000000000000002 RSI=000000003dec1000 RDI=0000000000000000 RBP=ffffffffdb000000 RSP=ffffffffde36e000 R8 =000000003dec1410 R9 =000000003dec13fc R10=000000000000006c R11=0000000000000000 R12=0000000000000000 R13=0000000000000001 R14=0000000000000004 R15=000000003eacdf44 RIP=0000000002000263 RFL=00200002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 00000000 00000000
CS =0038 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA]
SS =0000 0000000000000000 00000000 00000000
DS =0000 0000000000000000 00000000 00000000
FS =0000 0000000000000000 00000000 00000000
GS =0000 0000000000000000 00000000 00000000
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT= 00000000029cc270 0000002f
IDT= 000000003f55e018 00000fff
CR0=80010033 CR2=ffffffffde36dff8 CR3=000000003fc01000 CR4=00000668 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000d00

Thanks,
Tom

Quoted 38 lines

Ard Biesheuvel (17):
x86/compressed: efi-mixed: rename efi_thunk_64.S to efi-mixed.S x86/compressed: efi-mixed: move 32-bit entrypoint code into .text
section
x86/compressed: efi-mixed: move bootargs parsing out of 32-bit startup
code
x86/compressed: efi-mixed: move efi32_pe_entry into .text section x86/compressed: efi-mixed: move efi32_entry out of head_64.S x86/compressed: efi-mixed: move efi32_pe_entry() out of head_64.S x86/compressed: efi: merge multiple definitions of image_offset into
one
x86/compressed: efi-mixed: simplify IDT/GDT preserve/restore x86/compressed: avoid touching ECX in startup32_set_idt_entry() x86/compressed: pull global variable ref up into startup32_load_idt() x86/compressed: move startup32_load_idt() into .text section x86/compressed: move startup32_load_idt() out of head_64.S x86/compressed: move startup32_check_sev_cbit() into .text x86/compressed: move startup32_check_sev_cbit() out of head_64.S x86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y
efi: x86: Make the deprecated EFI handover protocol optional

   arch/x86/Kconfig                        |  17 +
   arch/x86/boot/compressed/Makefile       |   8 +-
   arch/x86/boot/compressed/efi_mixed.S    | 351 ++++++++++++++++++++
   arch/x86/boot/compressed/efi_thunk_64.S | 195 -----------
   arch/x86/boot/compressed/head_32.S      |   4 -
   arch/x86/boot/compressed/head_64.S      | 303 +----------------
   arch/x86/boot/compressed/mem_encrypt.S  | 152 ++++++++-
   arch/x86/boot/header.S                  |   2 +-
   arch/x86/boot/tools/build.c             |   2 +
   drivers/firmware/efi/libstub/x86-stub.c |   2 +-
   10 files changed, 533 insertions(+), 503 deletions(-)

create mode 100644 arch/x86/boot/compressed/efi_mixed.S delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S

maintainer
#42 lore raw
Quoted 18 lines · Tom Lendacky

I bisected it to:

99b7f4b23d9f ("x86/boot/compressed, efi: Merge multiple definitions of image_offset into one")

And doing the following fixed it:

drivers/firmware/efi/libstub/x86-stub.c +1 -1
index cb5f0befee57..a0bfd31358ba 100644
@@ -23,7 +23,7 @@
 const efi_system_table_t *efi_system_table;
 const efi_dxe_services_table_t *efi_dxe_table;
-u32 image_offset;
+u32 image_offset __section(".data");
 static efi_loaded_image_t *image = NULL;
 static efi_status_t
 
Quoted 4 lines

I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put image_offset in the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".

Thank you both. I'll refresh the set and run it here tomorrow too, to make sure it boots here too.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
maintainer
#43 lore raw
In reply to Tom Lendacky
Quoted 12 lines · Tom Lendacky
drivers/firmware/efi/libstub/x86-stub.c +1 -1
index cb5f0befee57..a0bfd31358ba 100644
@@ -23,7 +23,7 @@
 const efi_system_table_t *efi_system_table;
 const efi_dxe_services_table_t *efi_dxe_table;
-u32 image_offset;
+u32 image_offset __section(".data");
 static efi_loaded_image_t *image = NULL;
 static efi_status_t
 
Quoted 4 lines

I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put image_offset in the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".

Hmm, two points about this:

1. Can we do

u32 image_offset __bss_decrypted;

here instead? We have this special section just for that fun and it self-documents this way.

2. Also, why does my SEV-ES guest boot just fine without that change?

[    0.000000] Linux version 6.1.0-rc6+ (root@ml) (gcc (Debian 11.3.0-1) 11.3.0, GNU ld (GNU Binutils for Debian) 2.38) #1 SMP PREEMPT_DYNAMIC Wed Nov 23 11:27:17 CET 2022
...
[    0.336132] Memory Encryption Features active: AMD SEV SEV-ES

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#44 lore raw
Quoted 21 lines · Borislav Petkov
Quoted 12 lines · Tom Lendacky
drivers/firmware/efi/libstub/x86-stub.c +1 -1
index cb5f0befee57..a0bfd31358ba 100644
@@ -23,7 +23,7 @@
 const efi_system_table_t *efi_system_table;
 const efi_dxe_services_table_t *efi_dxe_table;
-u32 image_offset;
+u32 image_offset __section(".data");
 static efi_loaded_image_t *image = NULL;
 static efi_status_t
 
Quoted 4 lines

I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put image_offset in the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".

Hmm, two points about this:

Quoted 7 lines

1. Can we do

u32 image_offset __bss_decrypted;

here instead? We have this special section just for that fun and it self-documents this way.

The patch moves it from .data to .bss inadvertently, and I am not convinced Tom's analysis is entirely accurate: we may simply have garbage in image_offset if we access it before .bss gets cleared.

Quoted 2 lines

2. Also, why does my SEV-ES guest boot just fine without that change?

Indeed, so it needs to be in .data

Quoted 11 lines
[    0.000000] Linux version 6.1.0-rc6+ (root@ml) (gcc (Debian 11.3.0-1) 11.3.0, GNU ld (GNU Binutils for Debian) 2.38) #1 SMP PREEMPT_DYNAMIC Wed Nov 23 11:27:17 CET 2022
...
[    0.336132] Memory Encryption Features active: AMD SEV SEV-ES

Thx.

--
Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette

maintainer
#45 lore raw
Quoted 3 lines · Ard Biesheuvel

The patch moves it from .data to .bss inadvertently, and I am not convinced Tom's analysis is entirely accurate: we may simply have garbage in image_offset if we access it before .bss gets cleared.

That should not be too hard to find out: add an endless loop in asm in the guest right after the first image_offset access:

1:
jmp 1b

and then dump its value.

Or Tom might have an even better solution.

But looking at the code, BSS clearing happens later, at .Lrelocated and the EFI stub comes before it. AFAICT.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#46 lore raw
Quoted 12 lines · Borislav Petkov
Quoted 3 lines · Ard Biesheuvel

The patch moves it from .data to .bss inadvertently, and I am not convinced Tom's analysis is entirely accurate: we may simply have garbage in image_offset if we access it before .bss gets cleared.

That should not be too hard to find out: add an endless loop in asm in the guest right after the first image_offset access:

1:
jmp 1b

Quoted 7 lines

and then dump its value.

Or Tom might have an even better solution.

But looking at the code, BSS clearing happens later, at .Lrelocated and the EFI stub comes before it. AFAICT.

Indeed. And moving it back into .data makes the most sense in any case
- the point of the patch is to drop the duplicate definitions from asm code, not to move it into a different section.

The reason I hadn't spotted this is because my boot chain always sets the value of image_offset during the boot, and does not rely on the statically initialized value at all.

#47 lore raw
In reply to Ard Biesheuvel
Quoted 26 lines · Ard Biesheuvel
Quoted 25 lines · Borislav Petkov
Quoted 12 lines · Tom Lendacky
drivers/firmware/efi/libstub/x86-stub.c +1 -1
index cb5f0befee57..a0bfd31358ba 100644
@@ -23,7 +23,7 @@
  const efi_system_table_t *efi_system_table;
  const efi_dxe_services_table_t *efi_dxe_table;
-u32 image_offset;
+u32 image_offset __section(".data");
  static efi_loaded_image_t *image = NULL;
  static efi_status_t
 
Quoted 4 lines

I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put image_offset in the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".

Hmm, two points about this:

1. Can we do

u32 image_offset __bss_decrypted;

Quoted 7 lines
Quoted 3 lines

here instead? We have this special section just for that fun and it self-documents this way.

The patch moves it from .data to .bss inadvertently, and I am not convinced Tom's analysis is entirely accurate: we may simply have garbage in image_offset if we access it before .bss gets cleared.

When running non-encrypted, I imagine all the memory is cleared to zero as part of Qemu allocating it. As soon as you put an SEV guest on top of that, host made zeroes will not appear as zeroes to the SEV guest, rather they will be decrypted and end up looking like ciphertext (hence the random values I kept seeing in image_offset). The SEV guest must explicitly clear it, which is why having it in .bss doesn't work for SEV.

Thanks,
Tom

Quoted 18 lines
Quoted 3 lines

2. Also, why does my SEV-ES guest boot just fine without that change?

Indeed, so it needs to be in .data

Quoted 11 lines
[    0.000000] Linux version 6.1.0-rc6+ (root@ml) (gcc (Debian 11.3.0-1) 11.3.0, GNU ld (GNU Binutils for Debian) 2.38) #1 SMP PREEMPT_DYNAMIC Wed Nov 23 11:27:17 CET 2022
...
[    0.336132] Memory Encryption Features active: AMD SEV SEV-ES

Thx.

--
Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette

author maintainer
#48 lore raw
Quoted 36 lines · Tom Lendacky
Quoted 26 lines · Ard Biesheuvel
Quoted 25 lines · Borislav Petkov
Quoted 12 lines · Tom Lendacky
drivers/firmware/efi/libstub/x86-stub.c +1 -1
index cb5f0befee57..a0bfd31358ba 100644
@@ -23,7 +23,7 @@
  const efi_system_table_t *efi_system_table;
  const efi_dxe_services_table_t *efi_dxe_table;
-u32 image_offset;
+u32 image_offset __section(".data");
  static efi_loaded_image_t *image = NULL;
  static efi_status_t
 
Quoted 4 lines

I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put image_offset in the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".

Hmm, two points about this:

1. Can we do

u32 image_offset __bss_decrypted;

Quoted 7 lines
Quoted 3 lines

here instead? We have this special section just for that fun and it self-documents this way.

The patch moves it from .data to .bss inadvertently, and I am not convinced Tom's analysis is entirely accurate: we may simply have garbage in image_offset if we access it before .bss gets cleared.

Quoted 7 lines

When running non-encrypted, I imagine all the memory is cleared to zero as part of Qemu allocating it. As soon as you put an SEV guest on top of that, host made zeroes will not appear as zeroes to the SEV guest, rather they will be decrypted and end up looking like ciphertext (hence the random values I kept seeing in image_offset). The SEV guest must explicitly clear it, which is why having it in .bss doesn't work for SEV.

Yes, QEMU will probably clear it, but GRUB, shim, etc do a terrible job at implementing image loading correctly, i.e., not bothering to parse the PE/COFF header at all, but just copying the image into memory and invoke it at a fixed offset of 0x290 bytes into the image (the famous EFI handover protocol, see the last patch in this series if you want to know more :-))

This also means that the balance of the file size vs the image size (where BSS lives) is completely ignored, and we actually have to relocate the image from the EFI stub in such cases, because we cannot be sure that the BSS does not overlap with memory that is already in use, given the top down allocation strategy the EFI usually employs.

In summary, I guess there are multiple way how we might end up in this situation, and simply putting the variable back into .data where it came from is probably the best approach here.

#49 lore raw
In reply to Borislav Petkov
Quoted 20 lines · Borislav Petkov
Quoted 12 lines · Tom Lendacky
drivers/firmware/efi/libstub/x86-stub.c +1 -1
index cb5f0befee57..a0bfd31358ba 100644
@@ -23,7 +23,7 @@
  const efi_system_table_t *efi_system_table;
  const efi_dxe_services_table_t *efi_dxe_table;
-u32 image_offset;
+u32 image_offset __section(".data");
  static efi_loaded_image_t *image = NULL;
  static efi_status_t
 
Quoted 4 lines

I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put image_offset in the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".

Hmm, two points about this:

Quoted 6 lines

1. Can we do

u32 image_offset __bss_decrypted;

here instead? We have this special section just for that fun and it self-documents this way.

Yes, but __bss_decrypted is for the main kernel, not the decompression kernel.

The original value was in the .data section of the assembler (before the patch moved it), which gets initialized when loaded. Having it in the .bss section where you hope that memory was zeroed before hand is the issue.

Quoted 6 lines

2. Also, why does my SEV-ES guest boot just fine without that change?

[    0.000000] Linux version 6.1.0-rc6+ (root@ml) (gcc (Debian 11.3.0-1) 11.3.0, GNU ld (GNU Binutils for Debian) 2.38) #1 SMP PREEMPT_DYNAMIC Wed Nov 23 11:27:17 CET 2022
...
[    0.336132] Memory Encryption Features active: AMD SEV SEV-ES

Are you booting directly using the -kernel option on Qemu or going through the bootloader. It was only when using Grub that the problem appeared for me.

Thanks,
Tom

Quoted 3 lines

Thx.