x86: head_64.S spring cleaning
Ard Biesheuvel
maintainerv3 cover x86: head_64.S spring cleaning
After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.
Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/master
Changes since v1:
- at Boris's request, split the patches into smaller ones that are easier to review
Ard Biesheuvel (17):
x86/compressed: efi-mixed: rename efi_thunk_64.S to efi-mixed.S x86/compressed: efi-mixed: move 32-bit entrypoint code into .text
section
x86/compressed: efi-mixed: move bootargs parsing out of 32-bit startup
code
x86/compressed: efi-mixed: move efi32_pe_entry into .text section x86/compressed: efi-mixed: move efi32_entry out of head_64.S x86/compressed: efi-mixed: move efi32_pe_entry() out of head_64.S x86/compressed: efi: merge multiple definitions of image_offset into
one
x86/compressed: efi-mixed: simplify IDT/GDT preserve/restore x86/compressed: avoid touching ECX in startup32_set_idt_entry() x86/compressed: pull global variable ref up into startup32_load_idt() x86/compressed: move startup32_load_idt() into .text section x86/compressed: move startup32_load_idt() out of head_64.S x86/compressed: move startup32_check_sev_cbit() into .text x86/compressed: move startup32_check_sev_cbit() out of head_64.S x86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y
efi: x86: Make the deprecated EFI handover protocol optional
arch/x86/Kconfig | 17 + arch/x86/boot/compressed/Makefile | 8 +- arch/x86/boot/compressed/efi_mixed.S | 351 ++++++++++++++++++++ arch/x86/boot/compressed/efi_thunk_64.S | 195 ----------- arch/x86/boot/compressed/head_32.S | 4 - arch/x86/boot/compressed/head_64.S | 303 +---------------- arch/x86/boot/compressed/mem_encrypt.S | 152 ++++++++- arch/x86/boot/header.S | 2 +- arch/x86/boot/tools/build.c | 2 + drivers/firmware/efi/libstub/x86-stub.c | 2 +- 10 files changed, 533 insertions(+), 503 deletions(-) create mode 100644 arch/x86/boot/compressed/efi_mixed.S delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S
Signature · 1 line
2.35.1
Ard Biesheuvel
maintainerPATCH v3 1/17 x86/compressed: efi-mixed: rename efi_thunk_64.S to efi-mixed.S
In preparation for moving the mixed mode specific code out of head_64.S, rename the existing file to clarify that it contains more than just the mixed mode thunk.
While at it, clean up the Makefile rules that add it to the build.
---
arch/x86/boot/compressed/Makefile | 6 +++---
arch/x86/boot/compressed/{efi_thunk_64.S => efi_mixed.S} | 0
2 files changed, 3 insertions(+), 3 deletions(-)arch/x86/boot/compressed/Makefile +3 -3
@@ -108,11 +108,11 @@ endif vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o $(obj)/tdcall.o -vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o vmlinux-objs-$(CONFIG_EFI) += $(obj)/efi.o -efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.a +vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_mixed.o +vmlinux-objs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a -$(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE +$(obj)/vmlinux: $(vmlinux-objs-y) FORCE $(call if_changed,ld) OBJCOPYFLAGS_vmlinux.bin := -R .comment -S
arch/x86/boot/compressed/efi_mixed.S renamed from arch/x86/boot/compressed/efi_thunk_64.S +0 -0
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Rename efi_thunk_64.S to efi-mixed.S
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: cb8bda8ad4438b4bcfcf89697fc84803fb210017 Gitweb: https://git.kernel.org/tip/cb8bda8ad4438b4bcfcf89697fc84803fb210017 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:01 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Tue, 22 Nov 2022 18:54:44 +01:00
x86/boot/compressed: Rename efi_thunk_64.S to efi-mixed.S
In preparation for moving the mixed mode specific code out of head_64.S, rename the existing file to clarify that it contains more than just the mixed mode thunk.
While at it, clean up the Makefile rules that add it to the build.
--- arch/x86/boot/compressed/Makefile | 6 +- arch/x86/boot/compressed/efi_mixed.S | 195 +++++++++++++++++++++++- arch/x86/boot/compressed/efi_thunk_64.S | 195 +----------------------- 3 files changed, 198 insertions(+), 198 deletions(-) create mode 100644 arch/x86/boot/compressed/efi_mixed.S delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S
arch/x86/boot/compressed/Makefile +3 -3
@@ -108,11 +108,11 @@ endif vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o $(obj)/tdcall.o -vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o vmlinux-objs-$(CONFIG_EFI) += $(obj)/efi.o -efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.a +vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_mixed.o +vmlinux-objs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a -$(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE +$(obj)/vmlinux: $(vmlinux-objs-y) FORCE $(call if_changed,ld) OBJCOPYFLAGS_vmlinux.bin := -R .comment -S
arch/x86/boot/compressed/efi_mixed.S new file +195 -0
195 lines not on this page · raw message
arch/x86/boot/compressed/efi_thunk_64.S deleted +0 -195
195 lines not on this page · raw message
Ard Biesheuvel
maintainerPATCH v3 2/17 x86/compressed: efi-mixed: move 32-bit entrypoint code into .text section
In reply to Ard BiesheuvelMove the code that stores the arguments passed to the EFI entrypoint into the .text section, so that it can be moved into a separate compilation unit in a subsequent patch.
--- arch/x86/boot/compressed/head_64.S | 48 ++++++++++++++------ 1 file changed, 34 insertions(+), 14 deletions(-)
arch/x86/boot/compressed/head_64.S +34 -14
@@ -311,24 +311,41 @@ SYM_FUNC_START(efi32_stub_entry) popl %ecx popl %edx popl %esi + jmp efi32_entry +SYM_FUNC_END(efi32_stub_entry) + .text +/* + * This is the common EFI stub entry point for mixed mode. + * + * Arguments: %ecx image handle + * %edx EFI system table pointer + * %esi struct bootparams pointer (or NULL when not using + * the EFI handover protocol) + * + * Since this is the point of no return for ordinary execution, no registers + * are considered live except for the function parameters. [Note that the EFI + * stub may still exit and return to the firmware using the Exit() EFI boot + * service.] + */ +SYM_FUNC_START_LOCAL(efi32_entry) call 1f -1: pop %ebp - subl $ rva(1b), %ebp - - movl %esi, rva(efi32_boot_args+8)(%ebp) -SYM_INNER_LABEL(efi32_pe_stub_entry, SYM_L_LOCAL) - movl %ecx, rva(efi32_boot_args)(%ebp) - movl %edx, rva(efi32_boot_args+4)(%ebp) - movb $0, rva(efi_is64)(%ebp) +1: pop %ebx /* Save firmware GDTR and code/data selectors */ - sgdtl rva(efi32_boot_gdt)(%ebp) - movw %cs, rva(efi32_boot_cs)(%ebp) - movw %ds, rva(efi32_boot_ds)(%ebp) + sgdtl (efi32_boot_gdt - 1b)(%ebx) + movw %cs, (efi32_boot_cs - 1b)(%ebx) + movw %ds, (efi32_boot_ds - 1b)(%ebx) /* Store firmware IDT descriptor */ - sidtl rva(efi32_boot_idt)(%ebp) + sidtl (efi32_boot_idt - 1b)(%ebx) + + /* Store boot arguments */ + leal (efi32_boot_args - 1b)(%ebx), %ebx + movl %ecx, 0(%ebx) + movl %edx, 4(%ebx) + movl %esi, 8(%ebx) + movb $0x0, 12(%ebx) // efi_is64 /* Disable paging */ movl %cr0, %eax
@@ -336,7 +353,8 @@ SYM_INNER_LABEL(efi32_pe_stub_entry, SYM_L_LOCAL) movl %eax, %cr0 jmp startup_32 -SYM_FUNC_END(efi32_stub_entry) +SYM_FUNC_END(efi32_entry) + __HEAD #endif .code64
@@ -839,7 +857,9 @@ SYM_FUNC_START(efi32_pe_entry) */ subl %esi, %ebx movl %ebx, rva(image_offset)(%ebp) // save image_offset - jmp efi32_pe_stub_entry + xorl %esi, %esi + jmp efi32_entry // pass %ecx, %edx, %esi + // no other registers remain live 2: popl %edi // restore callee-save registers popl %ebx
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Move 32-bit entrypoint code into .text section
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: e2ab9eab324cdf240de89741e4a1aa79919f0196 Gitweb: https://git.kernel.org/tip/e2ab9eab324cdf240de89741e4a1aa79919f0196 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:02 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Tue, 22 Nov 2022 19:03:56 +01:00
x86/boot/compressed: Move 32-bit entrypoint code into .text section
Move the code that stores the arguments passed to the EFI entrypoint into the .text section, so that it can be moved into a separate compilation unit in a subsequent patch.
--- arch/x86/boot/compressed/head_64.S | 48 ++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 14 deletions(-)
arch/x86/boot/compressed/head_64.S +34 -14
@@ -303,24 +303,41 @@ SYM_FUNC_START(efi32_stub_entry) popl %ecx popl %edx popl %esi + jmp efi32_entry +SYM_FUNC_END(efi32_stub_entry) + .text +/* + * This is the common EFI stub entry point for mixed mode. + * + * Arguments: %ecx image handle + * %edx EFI system table pointer + * %esi struct bootparams pointer (or NULL when not using + * the EFI handover protocol) + * + * Since this is the point of no return for ordinary execution, no registers + * are considered live except for the function parameters. [Note that the EFI + * stub may still exit and return to the firmware using the Exit() EFI boot + * service.] + */ +SYM_FUNC_START_LOCAL(efi32_entry) call 1f -1: pop %ebp - subl $ rva(1b), %ebp - - movl %esi, rva(efi32_boot_args+8)(%ebp) -SYM_INNER_LABEL(efi32_pe_stub_entry, SYM_L_LOCAL) - movl %ecx, rva(efi32_boot_args)(%ebp) - movl %edx, rva(efi32_boot_args+4)(%ebp) - movb $0, rva(efi_is64)(%ebp) +1: pop %ebx /* Save firmware GDTR and code/data selectors */ - sgdtl rva(efi32_boot_gdt)(%ebp) - movw %cs, rva(efi32_boot_cs)(%ebp) - movw %ds, rva(efi32_boot_ds)(%ebp) + sgdtl (efi32_boot_gdt - 1b)(%ebx) + movw %cs, (efi32_boot_cs - 1b)(%ebx) + movw %ds, (efi32_boot_ds - 1b)(%ebx) /* Store firmware IDT descriptor */ - sidtl rva(efi32_boot_idt)(%ebp) + sidtl (efi32_boot_idt - 1b)(%ebx) + + /* Store boot arguments */ + leal (efi32_boot_args - 1b)(%ebx), %ebx + movl %ecx, 0(%ebx) + movl %edx, 4(%ebx) + movl %esi, 8(%ebx) + movb $0x0, 12(%ebx) // efi_is64 /* Disable paging */ movl %cr0, %eax
@@ -328,7 +345,8 @@ SYM_INNER_LABEL(efi32_pe_stub_entry, SYM_L_LOCAL) movl %eax, %cr0 jmp startup_32 -SYM_FUNC_END(efi32_stub_entry) +SYM_FUNC_END(efi32_entry) + __HEAD #endif .code64
@@ -831,7 +849,9 @@ SYM_FUNC_START(efi32_pe_entry) */ subl %esi, %ebx movl %ebx, rva(image_offset)(%ebp) // save image_offset - jmp efi32_pe_stub_entry + xorl %esi, %esi + jmp efi32_entry // pass %ecx, %edx, %esi + // no other registers remain live 2: popl %edi // restore callee-save registers popl %ebx
Ard Biesheuvel
maintainerPATCH v3 3/17 x86/compressed: efi-mixed: move bootargs parsing out of 32-bit startup code
In reply to Ard BiesheuvelMove the logic that chooses between the different EFI entrypoints out of the 32-bit boot path, and into a 64-bit helper that can perform the same task much more cleanly. While at it, document the mixed mode boot flow in a code comment.
--- arch/x86/boot/compressed/efi_mixed.S | 43 ++++++++++++++++++++ arch/x86/boot/compressed/head_64.S | 24 ++--------- 2 files changed, 47 insertions(+), 20 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +43 -0
@@ -22,6 +22,49 @@ .code64 .text +/* + * When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode() + * is the first thing that runs after switching to long mode. Depending on + * whether the EFI handover protocol or the compat entry point was used to + * enter the kernel, it will either branch to the 64-bit EFI handover + * entrypoint at offset 0x390 in the image, or to the 64-bit EFI PE/COFF + * entrypoint efi_pe_entry(). In the former case, the bootloader must provide a + * struct bootparams pointer as the third argument, so the presence of such a + * pointer is used to disambiguate. + * + * +--------------+ + * +------------------+ +------------+ +------>| efi_pe_entry | + * | efi32_pe_entry |---->| | | +-----------+--+ + * +------------------+ | | +------+----------------+ | + * | startup_32 |---->| startup_64_mixed_mode | | + * +------------------+ | | +------+----------------+ V + * | efi32_stub_entry |---->| | | +------------------+ + * +------------------+ +------------+ +---->| efi64_stub_entry | + * +-------------+----+ + * +------------+ +----------+ | + * | startup_64 |<----| efi_main |<--------------+ + * +------------+ +----------+ + */ +SYM_FUNC_START(startup_64_mixed_mode) + lea efi32_boot_args(%rip), %rdx + mov 0(%rdx), %edi + mov 4(%rdx), %esi + mov 8(%rdx), %edx // saved bootparams pointer + test %edx, %edx + jnz efi64_stub_entry + /* + * efi_pe_entry uses MS calling convention, which requires 32 bytes of + * shadow space on the stack even if all arguments are passed in + * registers. We also need an additional 8 bytes for the space that + * would be occupied by the return address, and this also results in + * the correct stack alignment for entry. + */ + sub $40, %rsp + mov %rdi, %rcx // MS calling convention + mov %rsi, %rdx + jmp efi_pe_entry +SYM_FUNC_END(startup_64_mixed_mode) + SYM_FUNC_START(__efi64_thunk) push %rbp push %rbx
arch/x86/boot/compressed/head_64.S +4 -20
@@ -269,25 +269,9 @@ SYM_FUNC_START(startup_32) */ leal rva(startup_64)(%ebp), %eax #ifdef CONFIG_EFI_MIXED - movl rva(efi32_boot_args)(%ebp), %edi - testl %edi, %edi - jz 1f - leal rva(efi64_stub_entry)(%ebp), %eax - movl rva(efi32_boot_args+4)(%ebp), %esi - movl rva(efi32_boot_args+8)(%ebp), %edx // saved bootparams pointer - testl %edx, %edx - jnz 1f - /* - * efi_pe_entry uses MS calling convention, which requires 32 bytes of - * shadow space on the stack even if all arguments are passed in - * registers. We also need an additional 8 bytes for the space that - * would be occupied by the return address, and this also results in - * the correct stack alignment for entry. - */ - subl $40, %esp - leal rva(efi_pe_entry)(%ebp), %eax - movl %edi, %ecx // MS calling convention - movl %esi, %edx + cmpb $1, rva(efi_is64)(%ebp) + je 1f + leal rva(startup_64_mixed_mode)(%ebp), %eax 1: #endif /* Check if the C-bit position is correct when SEV is active */
@@ -787,7 +771,7 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) SYM_DATA(image_offset, .long 0) #endif #ifdef CONFIG_EFI_MIXED -SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0) +SYM_DATA(efi32_boot_args, .long 0, 0, 0) SYM_DATA(efi_is64, .byte 1) #define ST32_boottime 60 // offsetof(efi_system_table_32_t, boottime)
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Move bootargs parsing out of 32-bit startup code
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 5c3a85f35b583259cf5ca0344cd79c8899ba1bb7 Gitweb: https://git.kernel.org/tip/5c3a85f35b583259cf5ca0344cd79c8899ba1bb7 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:03 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Tue, 22 Nov 2022 19:21:26 +01:00
x86/boot/compressed: Move bootargs parsing out of 32-bit startup code
Move the logic that chooses between the different EFI entrypoints out of the 32-bit boot path, and into a 64-bit helper that can perform the same task much more cleanly. While at it, document the mixed mode boot flow in a code comment.
--- arch/x86/boot/compressed/efi_mixed.S | 43 +++++++++++++++++++++++++++- arch/x86/boot/compressed/head_64.S | 24 ++------------- 2 files changed, 47 insertions(+), 20 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +43 -0
@@ -22,6 +22,49 @@ .code64 .text +/* + * When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode() + * is the first thing that runs after switching to long mode. Depending on + * whether the EFI handover protocol or the compat entry point was used to + * enter the kernel, it will either branch to the 64-bit EFI handover + * entrypoint at offset 0x390 in the image, or to the 64-bit EFI PE/COFF + * entrypoint efi_pe_entry(). In the former case, the bootloader must provide a + * struct bootparams pointer as the third argument, so the presence of such a + * pointer is used to disambiguate. + * + * +--------------+ + * +------------------+ +------------+ +------>| efi_pe_entry | + * | efi32_pe_entry |---->| | | +-----------+--+ + * +------------------+ | | +------+----------------+ | + * | startup_32 |---->| startup_64_mixed_mode | | + * +------------------+ | | +------+----------------+ V + * | efi32_stub_entry |---->| | | +------------------+ + * +------------------+ +------------+ +---->| efi64_stub_entry | + * +-------------+----+ + * +------------+ +----------+ | + * | startup_64 |<----| efi_main |<--------------+ + * +------------+ +----------+ + */ +SYM_FUNC_START(startup_64_mixed_mode) + lea efi32_boot_args(%rip), %rdx + mov 0(%rdx), %edi + mov 4(%rdx), %esi + mov 8(%rdx), %edx // saved bootparams pointer + test %edx, %edx + jnz efi64_stub_entry + /* + * efi_pe_entry uses MS calling convention, which requires 32 bytes of + * shadow space on the stack even if all arguments are passed in + * registers. We also need an additional 8 bytes for the space that + * would be occupied by the return address, and this also results in + * the correct stack alignment for entry. + */ + sub $40, %rsp + mov %rdi, %rcx // MS calling convention + mov %rsi, %rdx + jmp efi_pe_entry +SYM_FUNC_END(startup_64_mixed_mode) + SYM_FUNC_START(__efi64_thunk) push %rbp push %rbx
arch/x86/boot/compressed/head_64.S +4 -20
@@ -261,25 +261,9 @@ SYM_FUNC_START(startup_32) */ leal rva(startup_64)(%ebp), %eax #ifdef CONFIG_EFI_MIXED - movl rva(efi32_boot_args)(%ebp), %edi - testl %edi, %edi - jz 1f - leal rva(efi64_stub_entry)(%ebp), %eax - movl rva(efi32_boot_args+4)(%ebp), %esi - movl rva(efi32_boot_args+8)(%ebp), %edx // saved bootparams pointer - testl %edx, %edx - jnz 1f - /* - * efi_pe_entry uses MS calling convention, which requires 32 bytes of - * shadow space on the stack even if all arguments are passed in - * registers. We also need an additional 8 bytes for the space that - * would be occupied by the return address, and this also results in - * the correct stack alignment for entry. - */ - subl $40, %esp - leal rva(efi_pe_entry)(%ebp), %eax - movl %edi, %ecx // MS calling convention - movl %esi, %edx + cmpb $1, rva(efi_is64)(%ebp) + je 1f + leal rva(startup_64_mixed_mode)(%ebp), %eax 1: #endif /* Check if the C-bit position is correct when SEV is active */
@@ -779,7 +763,7 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) SYM_DATA(image_offset, .long 0) #endif #ifdef CONFIG_EFI_MIXED -SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0) +SYM_DATA(efi32_boot_args, .long 0, 0, 0) SYM_DATA(efi_is64, .byte 1) #define ST32_boottime 60 // offsetof(efi_system_table_32_t, boottime)
Ard Biesheuvel
maintainerPATCH v3 4/17 x86/compressed: efi-mixed: move efi32_pe_entry into .text section
In reply to Ard BiesheuvelMove efi32_pe_entry() into the .text section, so that it can be moved out of head_64.S and into a separate compilation unit in a subsequent patch.
--- arch/x86/boot/compressed/head_64.S | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)
arch/x86/boot/compressed/head_64.S +5 -6
@@ -778,7 +778,7 @@ SYM_DATA(efi_is64, .byte 1) #define BS32_handle_protocol 88 // offsetof(efi_boot_services_32_t, handle_protocol) #define LI32_image_base 32 // offsetof(efi_loaded_image_32_t, image_base) - __HEAD + .text .code32 SYM_FUNC_START(efi32_pe_entry) /*
@@ -800,12 +800,11 @@ SYM_FUNC_START(efi32_pe_entry) call 1f 1: pop %ebx - subl $ rva(1b), %ebx /* Get the loaded image protocol pointer from the image handle */ leal -4(%ebp), %eax pushl %eax // &loaded_image - leal rva(loaded_image_proto)(%ebx), %eax + leal (loaded_image_proto - 1b)(%ebx), %eax pushl %eax // pass the GUID address pushl 8(%ebp) // pass the image handle
@@ -834,13 +833,13 @@ SYM_FUNC_START(efi32_pe_entry) movl 12(%ebp), %edx // sys_table movl -4(%ebp), %esi // loaded_image movl LI32_image_base(%esi), %esi // loaded_image->image_base - movl %ebx, %ebp // startup_32 for efi32_pe_stub_entry + leal (startup_32 - 1b)(%ebx), %ebp // runtime address of startup_32 /* * We need to set the image_offset variable here since startup_32() will * use it before we get to the 64-bit efi_pe_entry() in C code. */ - subl %esi, %ebx - movl %ebx, rva(image_offset)(%ebp) // save image_offset + subl %esi, %ebp // calculate image_offset + movl %ebp, (image_offset - 1b)(%ebx) // save image_offset xorl %esi, %esi jmp efi32_entry // pass %ecx, %edx, %esi // no other registers remain live
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Move efi32_pe_entry into .text section
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 91592b5c0c2f076ff9d8cc0c14aa563448ac9fc4 Gitweb: https://git.kernel.org/tip/91592b5c0c2f076ff9d8cc0c14aa563448ac9fc4 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:04 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Tue, 22 Nov 2022 19:23:01 +01:00
x86/boot/compressed: Move efi32_pe_entry into .text section
Move efi32_pe_entry() into the .text section, so that it can be moved out of head_64.S and into a separate compilation unit in a subsequent patch.
--- arch/x86/boot/compressed/head_64.S | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)
arch/x86/boot/compressed/head_64.S +5 -6
@@ -770,7 +770,7 @@ SYM_DATA(efi_is64, .byte 1) #define BS32_handle_protocol 88 // offsetof(efi_boot_services_32_t, handle_protocol) #define LI32_image_base 32 // offsetof(efi_loaded_image_32_t, image_base) - __HEAD + .text .code32 SYM_FUNC_START(efi32_pe_entry) /*
@@ -792,12 +792,11 @@ SYM_FUNC_START(efi32_pe_entry) call 1f 1: pop %ebx - subl $ rva(1b), %ebx /* Get the loaded image protocol pointer from the image handle */ leal -4(%ebp), %eax pushl %eax // &loaded_image - leal rva(loaded_image_proto)(%ebx), %eax + leal (loaded_image_proto - 1b)(%ebx), %eax pushl %eax // pass the GUID address pushl 8(%ebp) // pass the image handle
@@ -826,13 +825,13 @@ SYM_FUNC_START(efi32_pe_entry) movl 12(%ebp), %edx // sys_table movl -4(%ebp), %esi // loaded_image movl LI32_image_base(%esi), %esi // loaded_image->image_base - movl %ebx, %ebp // startup_32 for efi32_pe_stub_entry + leal (startup_32 - 1b)(%ebx), %ebp // runtime address of startup_32 /* * We need to set the image_offset variable here since startup_32() will * use it before we get to the 64-bit efi_pe_entry() in C code. */ - subl %esi, %ebx - movl %ebx, rva(image_offset)(%ebp) // save image_offset + subl %esi, %ebp // calculate image_offset + movl %ebp, (image_offset - 1b)(%ebx) // save image_offset xorl %esi, %esi jmp efi32_entry // pass %ecx, %edx, %esi // no other registers remain live
Ard Biesheuvel
maintainerPATCH v3 5/17 x86/compressed: efi-mixed: move efi32_entry out of head_64.S
In reply to Ard BiesheuvelMove the efi32_entry() routine out of head_64.S and into efi-mixed.S, which reduces clutter in the complicated startup routines. It also permits linkage of some symbols used by code to be made local.
--- arch/x86/boot/compressed/efi_mixed.S | 57 ++++++++++++++++---- arch/x86/boot/compressed/head_64.S | 45 ---------------- 2 files changed, 47 insertions(+), 55 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +47 -10
@@ -105,7 +105,7 @@ SYM_FUNC_START(__efi64_thunk) /* * Switch to IDT and GDT with 32-bit segments. This is the firmware GDT * and IDT that was installed when the kernel started executing. The - * pointers were saved at the EFI stub entry point in head_64.S. + * pointers were saved by the efi32_entry() routine below. * * Pass the saved DS selector to the 32-bit code, and use far return to * restore the saved CS selector.
@@ -217,22 +217,59 @@ SYM_FUNC_START_LOCAL(efi_enter32) lret SYM_FUNC_END(efi_enter32) +/* + * This is the common EFI stub entry point for mixed mode. + * + * Arguments: %ecx image handle + * %edx EFI system table pointer + * %esi struct bootparams pointer (or NULL when not using + * the EFI handover protocol) + * + * Since this is the point of no return for ordinary execution, no registers + * are considered live except for the function parameters. [Note that the EFI + * stub may still exit and return to the firmware using the Exit() EFI boot + * service.] + */ +SYM_FUNC_START(efi32_entry) + call 1f +1: pop %ebx + + /* Save firmware GDTR and code/data selectors */ + sgdtl (efi32_boot_gdt - 1b)(%ebx) + movw %cs, (efi32_boot_cs - 1b)(%ebx) + movw %ds, (efi32_boot_ds - 1b)(%ebx) + + /* Store firmware IDT descriptor */ + sidtl (efi32_boot_idt - 1b)(%ebx) + + /* Store boot arguments */ + leal (efi32_boot_args - 1b)(%ebx), %ebx + movl %ecx, 0(%ebx) + movl %edx, 4(%ebx) + movl %esi, 8(%ebx) + movb $0x0, 12(%ebx) // efi_is64 + + /* Disable paging */ + movl %cr0, %eax + btrl $X86_CR0_PG_BIT, %eax + movl %eax, %cr0 + + jmp startup_32 +SYM_FUNC_END(efi32_entry) + .data .balign 8 -SYM_DATA_START(efi32_boot_gdt) +SYM_DATA_START_LOCAL(efi32_boot_gdt) .word 0 .quad 0 SYM_DATA_END(efi32_boot_gdt) -SYM_DATA_START(efi32_boot_idt) +SYM_DATA_START_LOCAL(efi32_boot_idt) .word 0 .quad 0 SYM_DATA_END(efi32_boot_idt) -SYM_DATA_START(efi32_boot_cs) - .word 0 -SYM_DATA_END(efi32_boot_cs) - -SYM_DATA_START(efi32_boot_ds) - .word 0 -SYM_DATA_END(efi32_boot_ds) +SYM_DATA_LOCAL(efi32_boot_cs, .word 0) +SYM_DATA_LOCAL(efi32_boot_ds, .word 0) +SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0) +SYM_DATA(efi_is64, .byte 1)
arch/x86/boot/compressed/head_64.S +0 -45
57 lines not on this page · raw message
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Move efi32_entry out of head_64.S
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 73a6dec80e2acedaef3ca603d4b5799049f6e9f8 Gitweb: https://git.kernel.org/tip/73a6dec80e2acedaef3ca603d4b5799049f6e9f8 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:05 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Tue, 22 Nov 2022 19:24:09 +01:00
x86/boot/compressed: Move efi32_entry out of head_64.S
Move the efi32_entry() routine out of head_64.S and into efi-mixed.S, which reduces clutter in the complicated startup routines. It also permits linkage of some symbols used by code to be made local.
--- arch/x86/boot/compressed/efi_mixed.S | 57 ++++++++++++++++++++++----- arch/x86/boot/compressed/head_64.S | 45 +--------------------- 2 files changed, 47 insertions(+), 55 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +47 -10
@@ -105,7 +105,7 @@ SYM_FUNC_START(__efi64_thunk) /* * Switch to IDT and GDT with 32-bit segments. This is the firmware GDT * and IDT that was installed when the kernel started executing. The - * pointers were saved at the EFI stub entry point in head_64.S. + * pointers were saved by the efi32_entry() routine below. * * Pass the saved DS selector to the 32-bit code, and use far return to * restore the saved CS selector.
@@ -217,22 +217,59 @@ SYM_FUNC_START_LOCAL(efi_enter32) lret SYM_FUNC_END(efi_enter32) +/* + * This is the common EFI stub entry point for mixed mode. + * + * Arguments: %ecx image handle + * %edx EFI system table pointer + * %esi struct bootparams pointer (or NULL when not using + * the EFI handover protocol) + * + * Since this is the point of no return for ordinary execution, no registers + * are considered live except for the function parameters. [Note that the EFI + * stub may still exit and return to the firmware using the Exit() EFI boot + * service.] + */ +SYM_FUNC_START(efi32_entry) + call 1f +1: pop %ebx + + /* Save firmware GDTR and code/data selectors */ + sgdtl (efi32_boot_gdt - 1b)(%ebx) + movw %cs, (efi32_boot_cs - 1b)(%ebx) + movw %ds, (efi32_boot_ds - 1b)(%ebx) + + /* Store firmware IDT descriptor */ + sidtl (efi32_boot_idt - 1b)(%ebx) + + /* Store boot arguments */ + leal (efi32_boot_args - 1b)(%ebx), %ebx + movl %ecx, 0(%ebx) + movl %edx, 4(%ebx) + movl %esi, 8(%ebx) + movb $0x0, 12(%ebx) // efi_is64 + + /* Disable paging */ + movl %cr0, %eax + btrl $X86_CR0_PG_BIT, %eax + movl %eax, %cr0 + + jmp startup_32 +SYM_FUNC_END(efi32_entry) + .data .balign 8 -SYM_DATA_START(efi32_boot_gdt) +SYM_DATA_START_LOCAL(efi32_boot_gdt) .word 0 .quad 0 SYM_DATA_END(efi32_boot_gdt) -SYM_DATA_START(efi32_boot_idt) +SYM_DATA_START_LOCAL(efi32_boot_idt) .word 0 .quad 0 SYM_DATA_END(efi32_boot_idt) -SYM_DATA_START(efi32_boot_cs) - .word 0 -SYM_DATA_END(efi32_boot_cs) - -SYM_DATA_START(efi32_boot_ds) - .word 0 -SYM_DATA_END(efi32_boot_ds) +SYM_DATA_LOCAL(efi32_boot_cs, .word 0) +SYM_DATA_LOCAL(efi32_boot_ds, .word 0) +SYM_DATA_LOCAL(efi32_boot_args, .long 0, 0, 0) +SYM_DATA(efi_is64, .byte 1)
arch/x86/boot/compressed/head_64.S +0 -45
57 lines not on this page · raw message
Ard Biesheuvel
maintainerPATCH v3 6/17 x86/compressed: efi-mixed: move efi32_pe_entry() out of head_64.S
In reply to Ard BiesheuvelMove the implementation of efi32_pe_entry() into efi-mixed.S, which is a more suitable location that only gets built if EFI mixed mode is actually enabled.
--- arch/x86/boot/compressed/efi_mixed.S | 82 ++++++++++++++++++ arch/x86/boot/compressed/head_64.S | 87 +------------------- 2 files changed, 83 insertions(+), 86 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +82 -0
@@ -257,6 +257,88 @@ SYM_FUNC_START(efi32_entry) jmp startup_32 SYM_FUNC_END(efi32_entry) +#define ST32_boottime 60 // offsetof(efi_system_table_32_t, boottime) +#define BS32_handle_protocol 88 // offsetof(efi_boot_services_32_t, handle_protocol) +#define LI32_image_base 32 // offsetof(efi_loaded_image_32_t, image_base) + +/* + * efi_status_t efi32_pe_entry(efi_handle_t image_handle, + * efi_system_table_32_t *sys_table) + */ +SYM_FUNC_START(efi32_pe_entry) + pushl %ebp + movl %esp, %ebp + pushl %eax // dummy push to allocate loaded_image + + pushl %ebx // save callee-save registers + pushl %edi + + call verify_cpu // check for long mode support + testl %eax, %eax + movl $0x80000003, %eax // EFI_UNSUPPORTED + jnz 2f + + call 1f +1: pop %ebx + + /* Get the loaded image protocol pointer from the image handle */ + leal -4(%ebp), %eax + pushl %eax // &loaded_image + leal (loaded_image_proto - 1b)(%ebx), %eax + pushl %eax // pass the GUID address + pushl 8(%ebp) // pass the image handle + + /* + * Note the alignment of the stack frame. + * sys_table + * handle <-- 16-byte aligned on entry by ABI + * return address + * frame pointer + * loaded_image <-- local variable + * saved %ebx <-- 16-byte aligned here + * saved %edi + * &loaded_image + * &loaded_image_proto + * handle <-- 16-byte aligned for call to handle_protocol + */ + + movl 12(%ebp), %eax // sys_table + movl ST32_boottime(%eax), %eax // sys_table->boottime + call *BS32_handle_protocol(%eax) // sys_table->boottime->handle_protocol + addl $12, %esp // restore argument space + testl %eax, %eax + jnz 2f + + movl 8(%ebp), %ecx // image_handle + movl 12(%ebp), %edx // sys_table + movl -4(%ebp), %esi // loaded_image + movl LI32_image_base(%esi), %esi // loaded_image->image_base + leal (startup_32 - 1b)(%ebx), %ebp // runtime address of startup_32 + /* + * We need to set the image_offset variable here since startup_32() will + * use it before we get to the 64-bit efi_pe_entry() in C code. + */ + subl %esi, %ebp // calculate image_offset + movl %ebp, (image_offset - 1b)(%ebx) // save image_offset + xorl %esi, %esi + jmp efi32_entry // pass %ecx, %edx, %esi + // no other registers remain live + +2: popl %edi // restore callee-save registers + popl %ebx + leave + RET +SYM_FUNC_END(efi32_pe_entry) + + .section ".rodata" + /* EFI loaded image protocol GUID */ + .balign 4 +SYM_DATA_START_LOCAL(loaded_image_proto) + .long 0x5b1b31a1 + .word 0x9562, 0x11d2 + .byte 0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b +SYM_DATA_END(loaded_image_proto) + .data .balign 8 SYM_DATA_START_LOCAL(efi32_boot_gdt)
arch/x86/boot/compressed/head_64.S +1 -86
99 lines not on this page · raw message
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Move efi32_pe_entry() out of head_64.S
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 7f22ca396778fea9332d83ec2359dbe8396e9a06 Gitweb: https://git.kernel.org/tip/7f22ca396778fea9332d83ec2359dbe8396e9a06 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:06 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Tue, 22 Nov 2022 19:24:57 +01:00
x86/boot/compressed: Move efi32_pe_entry() out of head_64.S
Move the implementation of efi32_pe_entry() into efi-mixed.S, which is a more suitable location that only gets built if EFI mixed mode is actually enabled.
--- arch/x86/boot/compressed/efi_mixed.S | 82 +++++++++++++++++++++++++- arch/x86/boot/compressed/head_64.S | 87 +--------------------------- 2 files changed, 83 insertions(+), 86 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +82 -0
@@ -257,6 +257,88 @@ SYM_FUNC_START(efi32_entry) jmp startup_32 SYM_FUNC_END(efi32_entry) +#define ST32_boottime 60 // offsetof(efi_system_table_32_t, boottime) +#define BS32_handle_protocol 88 // offsetof(efi_boot_services_32_t, handle_protocol) +#define LI32_image_base 32 // offsetof(efi_loaded_image_32_t, image_base) + +/* + * efi_status_t efi32_pe_entry(efi_handle_t image_handle, + * efi_system_table_32_t *sys_table) + */ +SYM_FUNC_START(efi32_pe_entry) + pushl %ebp + movl %esp, %ebp + pushl %eax // dummy push to allocate loaded_image + + pushl %ebx // save callee-save registers + pushl %edi + + call verify_cpu // check for long mode support + testl %eax, %eax + movl $0x80000003, %eax // EFI_UNSUPPORTED + jnz 2f + + call 1f +1: pop %ebx + + /* Get the loaded image protocol pointer from the image handle */ + leal -4(%ebp), %eax + pushl %eax // &loaded_image + leal (loaded_image_proto - 1b)(%ebx), %eax + pushl %eax // pass the GUID address + pushl 8(%ebp) // pass the image handle + + /* + * Note the alignment of the stack frame. + * sys_table + * handle <-- 16-byte aligned on entry by ABI + * return address + * frame pointer + * loaded_image <-- local variable + * saved %ebx <-- 16-byte aligned here + * saved %edi + * &loaded_image + * &loaded_image_proto + * handle <-- 16-byte aligned for call to handle_protocol + */ + + movl 12(%ebp), %eax // sys_table + movl ST32_boottime(%eax), %eax // sys_table->boottime + call *BS32_handle_protocol(%eax) // sys_table->boottime->handle_protocol + addl $12, %esp // restore argument space + testl %eax, %eax + jnz 2f + + movl 8(%ebp), %ecx // image_handle + movl 12(%ebp), %edx // sys_table + movl -4(%ebp), %esi // loaded_image + movl LI32_image_base(%esi), %esi // loaded_image->image_base + leal (startup_32 - 1b)(%ebx), %ebp // runtime address of startup_32 + /* + * We need to set the image_offset variable here since startup_32() will + * use it before we get to the 64-bit efi_pe_entry() in C code. + */ + subl %esi, %ebp // calculate image_offset + movl %ebp, (image_offset - 1b)(%ebx) // save image_offset + xorl %esi, %esi + jmp efi32_entry // pass %ecx, %edx, %esi + // no other registers remain live + +2: popl %edi // restore callee-save registers + popl %ebx + leave + RET +SYM_FUNC_END(efi32_pe_entry) + + .section ".rodata" + /* EFI loaded image protocol GUID */ + .balign 4 +SYM_DATA_START_LOCAL(loaded_image_proto) + .long 0x5b1b31a1 + .word 0x9562, 0x11d2 + .byte 0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b +SYM_DATA_END(loaded_image_proto) + .data .balign 8 SYM_DATA_START_LOCAL(efi32_boot_gdt)
arch/x86/boot/compressed/head_64.S +1 -86
99 lines not on this page · raw message
Ard Biesheuvel
maintainerPATCH v3 7/17 x86/compressed: efi: merge multiple definitions of image_offset into one
In reply to Ard BiesheuvelThere is no need for head_32.S and head_64.S both declaring a copy of the globale 'image_offset' variable, so drop those and make the extern C declaration the definition.
--- arch/x86/boot/compressed/head_32.S | 4 ---- arch/x86/boot/compressed/head_64.S | 4 ---- drivers/firmware/efi/libstub/x86-stub.c | 2 +- 3 files changed, 1 insertion(+), 9 deletions(-)
arch/x86/boot/compressed/head_32.S +0 -4
@@ -208,10 +208,6 @@ SYM_DATA_START_LOCAL(gdt) .quad 0x00cf92000000ffff /* __KERNEL_DS */ SYM_DATA_END_LABEL(gdt, SYM_L_LOCAL, gdt_end) -#ifdef CONFIG_EFI_STUB -SYM_DATA(image_offset, .long 0) -#endif - /* * Stack and heap for uncompression */
arch/x86/boot/compressed/head_64.S +0 -4
@@ -726,10 +726,6 @@ SYM_DATA_START(boot32_idt) SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) #endif -#ifdef CONFIG_EFI_STUB -SYM_DATA(image_offset, .long 0) -#endif - #ifdef CONFIG_AMD_MEM_ENCRYPT __HEAD .code32
drivers/firmware/efi/libstub/x86-stub.c +1 -1
@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -extern u32 image_offset; +u32 image_offset;static efi_loaded_image_t *image =NULL ; static efi_status_t
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed, efi: Merge multiple definitions of image_offset into one
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 4b52016247aeaa55ca3e3bc2e03cd91114c145c2 Gitweb: https://git.kernel.org/tip/4b52016247aeaa55ca3e3bc2e03cd91114c145c2 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:07 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:55:55 +01:00
x86/boot/compressed, efi: Merge multiple definitions of image_offset into one
There is no need for head_32.S and head_64.S both declaring a copy of the global 'image_offset' variable, so drop those and make the extern C declaration the definition.
When image_offset is moved to the .c file, it needs to be placed particularly in the .data section because it lands by default in the .bss section which is cleared too late, in .Lrelocated, before the first access to it and thus garbage gets read, leading to SEV guests exploding in early boot.
This happens only when the SEV guest kernel is loaded through grub. If supplied with qemu's -kernel command line option, that memory is always cleared upfront by qemu and all is fine there.
[ bp: Expand commit message with SEV aspect. ]
--- arch/x86/boot/compressed/head_32.S | 4 ---- arch/x86/boot/compressed/head_64.S | 4 ---- drivers/firmware/efi/libstub/x86-stub.c | 2 +- 3 files changed, 1 insertion(+), 9 deletions(-)
arch/x86/boot/compressed/head_32.S +0 -4
@@ -208,10 +208,6 @@ SYM_DATA_START_LOCAL(gdt) .quad 0x00cf92000000ffff /* __KERNEL_DS */ SYM_DATA_END_LABEL(gdt, SYM_L_LOCAL, gdt_end) -#ifdef CONFIG_EFI_STUB -SYM_DATA(image_offset, .long 0) -#endif - /* * Stack and heap for uncompression */
arch/x86/boot/compressed/head_64.S +0 -4
@@ -718,10 +718,6 @@ SYM_DATA_START(boot32_idt) SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) #endif -#ifdef CONFIG_EFI_STUB -SYM_DATA(image_offset, .long 0) -#endif - #ifdef CONFIG_AMD_MEM_ENCRYPT __HEAD .code32
drivers/firmware/efi/libstub/x86-stub.c +1 -1
@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -extern u32 image_offset; +u32 image_offset __section(".data");static efi_loaded_image_t *image =NULL ; static efi_status_t
Ard Biesheuvel
maintainerPATCH v3 8/17 x86/compressed: efi-mixed: simplify IDT/GDT preserve/restore
In reply to Ard BiesheuvelTweak the asm and remove some redundant instructions. While at it, fix the associated comment for style and correctness.
--- arch/x86/boot/compressed/efi_mixed.S | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +7 -13
@@ -96,24 +96,20 @@ SYM_FUNC_START(__efi64_thunk) leaq 0x20(%rsp), %rbx sgdt (%rbx) - - addq $16, %rbx - sidt (%rbx) + sidt 16(%rbx) leaq 1f(%rip), %rbp /* - * Switch to IDT and GDT with 32-bit segments. This is the firmware GDT - * and IDT that was installed when the kernel started executing. The - * pointers were saved by the efi32_entry() routine below. + * Switch to IDT and GDT with 32-bit segments. These are the firmware + * GDT and IDT that were installed when the kernel started executing. + * The pointers were saved by the efi32_entry() routine below. * * Pass the saved DS selector to the 32-bit code, and use far return to * restore the saved CS selector. */ - leaq efi32_boot_idt(%rip), %rax - lidt (%rax) - leaq efi32_boot_gdt(%rip), %rax - lgdt (%rax) + lidt efi32_boot_idt(%rip) + lgdt efi32_boot_gdt(%rip) movzwl efi32_boot_ds(%rip), %edx movzwq efi32_boot_cs(%rip), %rax
@@ -187,9 +183,7 @@ SYM_FUNC_START_LOCAL(efi_enter32) */ cli - lidtl (%ebx) - subl $16, %ebx - + lidtl 16(%ebx) lgdtl (%ebx) movl %cr4, %eax
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Simplify IDT/GDT preserve/restore in the EFI thunk
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 630f337f0c4fd80390e8600adcab31550aea33df Gitweb: https://git.kernel.org/tip/630f337f0c4fd80390e8600adcab31550aea33df Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:08 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00
x86/boot/compressed: Simplify IDT/GDT preserve/restore in the EFI thunk
Tweak the asm and remove some redundant instructions. While at it, fix the associated comment for style and correctness.
--- arch/x86/boot/compressed/efi_mixed.S | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +7 -13
@@ -96,24 +96,20 @@ SYM_FUNC_START(__efi64_thunk) leaq 0x20(%rsp), %rbx sgdt (%rbx) - - addq $16, %rbx - sidt (%rbx) + sidt 16(%rbx) leaq 1f(%rip), %rbp /* - * Switch to IDT and GDT with 32-bit segments. This is the firmware GDT - * and IDT that was installed when the kernel started executing. The - * pointers were saved by the efi32_entry() routine below. + * Switch to IDT and GDT with 32-bit segments. These are the firmware + * GDT and IDT that were installed when the kernel started executing. + * The pointers were saved by the efi32_entry() routine below. * * Pass the saved DS selector to the 32-bit code, and use far return to * restore the saved CS selector. */ - leaq efi32_boot_idt(%rip), %rax - lidt (%rax) - leaq efi32_boot_gdt(%rip), %rax - lgdt (%rax) + lidt efi32_boot_idt(%rip) + lgdt efi32_boot_gdt(%rip) movzwl efi32_boot_ds(%rip), %edx movzwq efi32_boot_cs(%rip), %rax
@@ -187,9 +183,7 @@ SYM_FUNC_START_LOCAL(efi_enter32) */ cli - lidtl (%ebx) - subl $16, %ebx - + lidtl 16(%ebx) lgdtl (%ebx) movl %cr4, %eax
Ard Biesheuvel
maintainerPATCH v3 9/17 x86/compressed: avoid touching ECX in startup32_set_idt_entry()
In reply to Ard BiesheuvelAvoid touching register %ecx in startup32_set_idt_entry(), by folding the MOV, SHL and ORL instructions into a single ORL which no longer requires a temp register.
This permits ECX to be used as a function argument in a subsequent patch.
--- arch/x86/boot/compressed/head_64.S | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
arch/x86/boot/compressed/head_64.S +2 -6
@@ -741,7 +741,6 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) */ SYM_FUNC_START(startup32_set_idt_entry) push %ebx - push %ecx /* IDT entry address to %ebx */ leal rva(boot32_idt)(%ebp), %ebx
@@ -750,10 +749,8 @@ SYM_FUNC_START(startup32_set_idt_entry) /* Build IDT entry, lower 4 bytes */ movl %eax, %edx - andl $0x0000ffff, %edx # Target code segment offset [15:0] - movl $__KERNEL32_CS, %ecx # Target code segment selector - shl $16, %ecx - orl %ecx, %edx + andl $0x0000ffff, %edx # Target code segment offset [15:0] + orl $(__KERNEL32_CS << 16), %edx # Target code segment selector /* Store lower 4 bytes to IDT */ movl %edx, (%ebx)
@@ -766,7 +763,6 @@ SYM_FUNC_START(startup32_set_idt_entry) /* Store upper 4 bytes to IDT */ movl %edx, 4(%ebx) - pop %ecx pop %ebx RET SYM_FUNC_END(startup32_set_idt_entry)
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Avoid touching ECX in startup32_set_idt_entry()
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 6aac80a8da46d70f2ae7ff97c9f45a15c7c9b3ef Gitweb: https://git.kernel.org/tip/6aac80a8da46d70f2ae7ff97c9f45a15c7c9b3ef Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:09 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00
x86/boot/compressed: Avoid touching ECX in startup32_set_idt_entry()
Avoid touching register %ecx in startup32_set_idt_entry(), by folding the MOV, SHL and ORL instructions into a single ORL which no longer requires a temp register.
This permits ECX to be used as a function argument in a subsequent patch.
--- arch/x86/boot/compressed/head_64.S | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
arch/x86/boot/compressed/head_64.S +2 -6
@@ -733,7 +733,6 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) */ SYM_FUNC_START(startup32_set_idt_entry) push %ebx - push %ecx /* IDT entry address to %ebx */ leal rva(boot32_idt)(%ebp), %ebx
@@ -742,10 +741,8 @@ SYM_FUNC_START(startup32_set_idt_entry) /* Build IDT entry, lower 4 bytes */ movl %eax, %edx - andl $0x0000ffff, %edx # Target code segment offset [15:0] - movl $__KERNEL32_CS, %ecx # Target code segment selector - shl $16, %ecx - orl %ecx, %edx + andl $0x0000ffff, %edx # Target code segment offset [15:0] + orl $(__KERNEL32_CS << 16), %edx # Target code segment selector /* Store lower 4 bytes to IDT */ movl %edx, (%ebx)
@@ -758,7 +755,6 @@ SYM_FUNC_START(startup32_set_idt_entry) /* Store upper 4 bytes to IDT */ movl %edx, 4(%ebx) - pop %ecx pop %ebx RET SYM_FUNC_END(startup32_set_idt_entry)
Ard Biesheuvel
maintainerPATCH v3 10/17 x86/compressed: pull global variable ref up into startup32_load_idt()
In reply to Ard BiesheuvelIn preparation for moving startup32_load_idt() out of head_64.S and turning it into an ordinary function using the ordinary 32-bit calling convention, pull the global variable reference to boot32_idt up into startup32_load_idt() so that startup32_set_idt_entry() does not need to discover its own runtime physical address, which will no longer be correlated with startup_32 once this code is moved into .text.
While at it, give startup32_set_idt_entry() static linkage.
--- arch/x86/boot/compressed/head_64.S | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-)
arch/x86/boot/compressed/head_64.S +8 -12
@@ -736,16 +736,11 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) * * %eax: Handler address * %edx: Vector number - * - * Physical offset is expected in %ebp + * %ecx: IDT address */ -SYM_FUNC_START(startup32_set_idt_entry) - push %ebx - - /* IDT entry address to %ebx */ - leal rva(boot32_idt)(%ebp), %ebx - shl $3, %edx - addl %edx, %ebx +SYM_FUNC_START_LOCAL(startup32_set_idt_entry) + /* IDT entry address to %ecx */ + leal (%ecx, %edx, 8), %ecx /* Build IDT entry, lower 4 bytes */ movl %eax, %edx
@@ -753,7 +748,7 @@ SYM_FUNC_START(startup32_set_idt_entry) orl $(__KERNEL32_CS << 16), %edx # Target code segment selector /* Store lower 4 bytes to IDT */ - movl %edx, (%ebx) + movl %edx, (%ecx) /* Build IDT entry, upper 4 bytes */ movl %eax, %edx
@@ -761,15 +756,16 @@ SYM_FUNC_START(startup32_set_idt_entry) orl $0x00008e00, %edx # Present, Type 32-bit Interrupt Gate /* Store upper 4 bytes to IDT */ - movl %edx, 4(%ebx) + movl %edx, 4(%ecx) - pop %ebx RET SYM_FUNC_END(startup32_set_idt_entry) #endif SYM_FUNC_START(startup32_load_idt) #ifdef CONFIG_AMD_MEM_ENCRYPT + leal rva(boot32_idt)(%ebp), %ecx + /* #VC handler */ leal rva(startup32_vc_handler)(%ebp), %eax movl $X86_TRAP_VC, %edx
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Pull global variable reference into startup32_load_idt()
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: d73a257f7f86871c3aac24dc20538e3983096647 Gitweb: https://git.kernel.org/tip/d73a257f7f86871c3aac24dc20538e3983096647 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:10 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00
x86/boot/compressed: Pull global variable reference into startup32_load_idt()
In preparation for moving startup32_load_idt() out of head_64.S and turning it into an ordinary function using the ordinary 32-bit calling convention, pull the global variable reference to boot32_idt up into startup32_load_idt() so that startup32_set_idt_entry() does not need to discover its own runtime physical address, which will no longer be correlated with startup_32 once this code is moved into .text.
While at it, give startup32_set_idt_entry() static linkage.
--- arch/x86/boot/compressed/head_64.S | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-)
arch/x86/boot/compressed/head_64.S +8 -12
@@ -728,16 +728,11 @@ SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) * * %eax: Handler address * %edx: Vector number - * - * Physical offset is expected in %ebp + * %ecx: IDT address */ -SYM_FUNC_START(startup32_set_idt_entry) - push %ebx - - /* IDT entry address to %ebx */ - leal rva(boot32_idt)(%ebp), %ebx - shl $3, %edx - addl %edx, %ebx +SYM_FUNC_START_LOCAL(startup32_set_idt_entry) + /* IDT entry address to %ecx */ + leal (%ecx, %edx, 8), %ecx /* Build IDT entry, lower 4 bytes */ movl %eax, %edx
@@ -745,7 +740,7 @@ SYM_FUNC_START(startup32_set_idt_entry) orl $(__KERNEL32_CS << 16), %edx # Target code segment selector /* Store lower 4 bytes to IDT */ - movl %edx, (%ebx) + movl %edx, (%ecx) /* Build IDT entry, upper 4 bytes */ movl %eax, %edx
@@ -753,15 +748,16 @@ SYM_FUNC_START(startup32_set_idt_entry) orl $0x00008e00, %edx # Present, Type 32-bit Interrupt Gate /* Store upper 4 bytes to IDT */ - movl %edx, 4(%ebx) + movl %edx, 4(%ecx) - pop %ebx RET SYM_FUNC_END(startup32_set_idt_entry) #endif SYM_FUNC_START(startup32_load_idt) #ifdef CONFIG_AMD_MEM_ENCRYPT + leal rva(boot32_idt)(%ebp), %ecx + /* #VC handler */ leal rva(startup32_vc_handler)(%ebp), %eax movl $X86_TRAP_VC, %edx
Ard Biesheuvel
maintainerPATCH v3 11/17 x86/compressed: move startup32_load_idt() into .text section
In reply to Ard BiesheuvelConvert startup32_load_idt() into an ordinary function and move it into the .text section. This involves turning the rva() immediates into ones derived from a local label, and preserving/restoring the %ebp and %ebx as per the calling convention.
Also move the #ifdef to the only existing call site. This makes it clear that the function call does nothing if support for memory encryption is not compiled in.
--- arch/x86/boot/compressed/head_64.S | 31 +++++++++++++------- 1 file changed, 20 insertions(+), 11 deletions(-)
arch/x86/boot/compressed/head_64.S +20 -11
@@ -126,7 +126,9 @@ SYM_FUNC_START(startup_32) 1: /* Setup Exception handling for SEV-ES */ +#ifdef CONFIG_AMD_MEM_ENCRYPT call startup32_load_idt +#endif /* Make sure cpu supports long mode. */ call verify_cpu
@@ -724,10 +726,8 @@ SYM_DATA_START(boot32_idt) .quad 0 .endr SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) -#endif -#ifdef CONFIG_AMD_MEM_ENCRYPT - __HEAD + .text .code32 /* * Write an IDT entry into boot32_idt
@@ -760,24 +760,32 @@ SYM_FUNC_START_LOCAL(startup32_set_idt_entry) RET SYM_FUNC_END(startup32_set_idt_entry) -#endif SYM_FUNC_START(startup32_load_idt) -#ifdef CONFIG_AMD_MEM_ENCRYPT - leal rva(boot32_idt)(%ebp), %ecx + push %ebp + push %ebx + + call 1f +1: pop %ebp + + leal (boot32_idt - 1b)(%ebp), %ebx /* #VC handler */ - leal rva(startup32_vc_handler)(%ebp), %eax + leal (startup32_vc_handler - 1b)(%ebp), %eax movl $X86_TRAP_VC, %edx + movl %ebx, %ecx call startup32_set_idt_entry /* Load IDT */ - leal rva(boot32_idt)(%ebp), %eax - movl %eax, rva(boot32_idt_desc+2)(%ebp) - lidt rva(boot32_idt_desc)(%ebp) -#endif + leal (boot32_idt_desc - 1b)(%ebp), %ecx + movl %ebx, 2(%ecx) + lidt (%ecx) + + pop %ebx + pop %ebp RET SYM_FUNC_END(startup32_load_idt) +#endif /* * Check for the correct C-bit position when the startup_32 boot-path is used.
@@ -796,6 +804,7 @@ SYM_FUNC_END(startup32_load_idt) * succeed. An incorrect C-bit position will map all memory unencrypted, so that * the compare will use the encrypted random data and fail. */ + __HEAD SYM_FUNC_START(startup32_check_sev_cbit) #ifdef CONFIG_AMD_MEM_ENCRYPT pushl %eax
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Move startup32_load_idt() into .text section
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: c6355995ba471d7ad574174e593192ce805c7e1a Gitweb: https://git.kernel.org/tip/c6355995ba471d7ad574174e593192ce805c7e1a Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:11 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00
x86/boot/compressed: Move startup32_load_idt() into .text section
Convert startup32_load_idt() into an ordinary function and move it into the .text section. This involves turning the rva() immediates into ones derived from a local label, and preserving/restoring the %ebp and %ebx as per the calling convention.
Also move the #ifdef to the only existing call site. This makes it clear that the function call does nothing if support for memory encryption is not compiled in.
--- arch/x86/boot/compressed/head_64.S | 31 ++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-)
arch/x86/boot/compressed/head_64.S +20 -11
@@ -118,7 +118,9 @@ SYM_FUNC_START(startup_32) 1: /* Setup Exception handling for SEV-ES */ +#ifdef CONFIG_AMD_MEM_ENCRYPT call startup32_load_idt +#endif /* Make sure cpu supports long mode. */ call verify_cpu
@@ -716,10 +718,8 @@ SYM_DATA_START(boot32_idt) .quad 0 .endr SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) -#endif -#ifdef CONFIG_AMD_MEM_ENCRYPT - __HEAD + .text .code32 /* * Write an IDT entry into boot32_idt
@@ -752,24 +752,32 @@ SYM_FUNC_START_LOCAL(startup32_set_idt_entry) RET SYM_FUNC_END(startup32_set_idt_entry) -#endif SYM_FUNC_START(startup32_load_idt) -#ifdef CONFIG_AMD_MEM_ENCRYPT - leal rva(boot32_idt)(%ebp), %ecx + push %ebp + push %ebx + + call 1f +1: pop %ebp + + leal (boot32_idt - 1b)(%ebp), %ebx /* #VC handler */ - leal rva(startup32_vc_handler)(%ebp), %eax + leal (startup32_vc_handler - 1b)(%ebp), %eax movl $X86_TRAP_VC, %edx + movl %ebx, %ecx call startup32_set_idt_entry /* Load IDT */ - leal rva(boot32_idt)(%ebp), %eax - movl %eax, rva(boot32_idt_desc+2)(%ebp) - lidt rva(boot32_idt_desc)(%ebp) -#endif + leal (boot32_idt_desc - 1b)(%ebp), %ecx + movl %ebx, 2(%ecx) + lidt (%ecx) + + pop %ebx + pop %ebp RET SYM_FUNC_END(startup32_load_idt) +#endif /* * Check for the correct C-bit position when the startup_32 boot-path is used.
@@ -788,6 +796,7 @@ SYM_FUNC_END(startup32_load_idt) * succeed. An incorrect C-bit position will map all memory unencrypted, so that * the compare will use the encrypted random data and fail. */ + __HEAD SYM_FUNC_START(startup32_check_sev_cbit) #ifdef CONFIG_AMD_MEM_ENCRYPT pushl %eax
Ard Biesheuvel
maintainerPATCH v3 12/17 x86/compressed: move startup32_load_idt() out of head_64.S
In reply to Ard BiesheuvelNow that startup32_load_idt() has been refactored into an ordinary callable function, move it into mem-encrypt.S where it belongs.
--- arch/x86/boot/compressed/head_64.S | 72 -------------------- arch/x86/boot/compressed/mem_encrypt.S | 72 +++++++++++++++++++- 2 files changed, 71 insertions(+), 73 deletions(-)
arch/x86/boot/compressed/head_64.S +0 -72
@@ -715,78 +715,6 @@ SYM_DATA_START(boot_idt) .endr SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end) -#ifdef CONFIG_AMD_MEM_ENCRYPT -SYM_DATA_START(boot32_idt_desc) - .word boot32_idt_end - boot32_idt - 1 - .long 0 -SYM_DATA_END(boot32_idt_desc) - .balign 8 -SYM_DATA_START(boot32_idt) - .rept 32 - .quad 0 - .endr -SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) - - .text - .code32 -/* - * Write an IDT entry into boot32_idt - * - * Parameters: - * - * %eax: Handler address - * %edx: Vector number - * %ecx: IDT address - */ -SYM_FUNC_START_LOCAL(startup32_set_idt_entry) - /* IDT entry address to %ecx */ - leal (%ecx, %edx, 8), %ecx - - /* Build IDT entry, lower 4 bytes */ - movl %eax, %edx - andl $0x0000ffff, %edx # Target code segment offset [15:0] - orl $(__KERNEL32_CS << 16), %edx # Target code segment selector - - /* Store lower 4 bytes to IDT */ - movl %edx, (%ecx) - - /* Build IDT entry, upper 4 bytes */ - movl %eax, %edx - andl $0xffff0000, %edx # Target code segment offset [31:16] - orl $0x00008e00, %edx # Present, Type 32-bit Interrupt Gate - - /* Store upper 4 bytes to IDT */ - movl %edx, 4(%ecx) - - RET -SYM_FUNC_END(startup32_set_idt_entry) - -SYM_FUNC_START(startup32_load_idt) - push %ebp - push %ebx - - call 1f -1: pop %ebp - - leal (boot32_idt - 1b)(%ebp), %ebx - - /* #VC handler */ - leal (startup32_vc_handler - 1b)(%ebp), %eax - movl $X86_TRAP_VC, %edx - movl %ebx, %ecx - call startup32_set_idt_entry - - /* Load IDT */ - leal (boot32_idt_desc - 1b)(%ebp), %ecx - movl %ebx, 2(%ecx) - lidt (%ecx) - - pop %ebx - pop %ebp - RET -SYM_FUNC_END(startup32_load_idt) -#endif - /* * Check for the correct C-bit position when the startup_32 boot-path is used. *
arch/x86/boot/compressed/mem_encrypt.S +71 -1
94 lines not on this page · raw message
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Move startup32_load_idt() out of head_64.S
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 9ea813be3d345dfb8ac5bf6fbb29e6a63647a39d Gitweb: https://git.kernel.org/tip/9ea813be3d345dfb8ac5bf6fbb29e6a63647a39d Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:12 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00
x86/boot/compressed: Move startup32_load_idt() out of head_64.S
Now that startup32_load_idt() has been refactored into an ordinary callable function, move it into mem-encrypt.S where it belongs.
--- arch/x86/boot/compressed/head_64.S | 72 +------------------------- arch/x86/boot/compressed/mem_encrypt.S | 72 ++++++++++++++++++++++++- 2 files changed, 71 insertions(+), 73 deletions(-)
arch/x86/boot/compressed/head_64.S +0 -72
@@ -707,78 +707,6 @@ SYM_DATA_START(boot_idt) .endr SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end) -#ifdef CONFIG_AMD_MEM_ENCRYPT -SYM_DATA_START(boot32_idt_desc) - .word boot32_idt_end - boot32_idt - 1 - .long 0 -SYM_DATA_END(boot32_idt_desc) - .balign 8 -SYM_DATA_START(boot32_idt) - .rept 32 - .quad 0 - .endr -SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) - - .text - .code32 -/* - * Write an IDT entry into boot32_idt - * - * Parameters: - * - * %eax: Handler address - * %edx: Vector number - * %ecx: IDT address - */ -SYM_FUNC_START_LOCAL(startup32_set_idt_entry) - /* IDT entry address to %ecx */ - leal (%ecx, %edx, 8), %ecx - - /* Build IDT entry, lower 4 bytes */ - movl %eax, %edx - andl $0x0000ffff, %edx # Target code segment offset [15:0] - orl $(__KERNEL32_CS << 16), %edx # Target code segment selector - - /* Store lower 4 bytes to IDT */ - movl %edx, (%ecx) - - /* Build IDT entry, upper 4 bytes */ - movl %eax, %edx - andl $0xffff0000, %edx # Target code segment offset [31:16] - orl $0x00008e00, %edx # Present, Type 32-bit Interrupt Gate - - /* Store upper 4 bytes to IDT */ - movl %edx, 4(%ecx) - - RET -SYM_FUNC_END(startup32_set_idt_entry) - -SYM_FUNC_START(startup32_load_idt) - push %ebp - push %ebx - - call 1f -1: pop %ebp - - leal (boot32_idt - 1b)(%ebp), %ebx - - /* #VC handler */ - leal (startup32_vc_handler - 1b)(%ebp), %eax - movl $X86_TRAP_VC, %edx - movl %ebx, %ecx - call startup32_set_idt_entry - - /* Load IDT */ - leal (boot32_idt_desc - 1b)(%ebp), %ecx - movl %ebx, 2(%ecx) - lidt (%ecx) - - pop %ebx - pop %ebp - RET -SYM_FUNC_END(startup32_load_idt) -#endif - /* * Check for the correct C-bit position when the startup_32 boot-path is used. *
arch/x86/boot/compressed/mem_encrypt.S +71 -1
94 lines not on this page · raw message
Ard Biesheuvel
maintainerPATCH v3 13/17 x86/compressed: move startup32_check_sev_cbit() into .text
In reply to Ard BiesheuvelMove startup32_check_sev_cbit() into the .text section and turn it into an ordinary function using the ordinary 32-bit calling convention, instead of saving/restoring the registers that are known to be live at the only call site. This improves maintainability, and makes it possible to move this function out of head_64.S and into a separate compilation unit that is specific to memory encryption.
Note that this requires the call site to be moved before the mixed mode check, as %eax will be live otherwise.
--- arch/x86/boot/compressed/head_64.S | 35 +++++++++++--------- 1 file changed, 19 insertions(+), 16 deletions(-)
arch/x86/boot/compressed/head_64.S +19 -16
@@ -259,6 +259,11 @@ SYM_FUNC_START(startup_32) movl $__BOOT_TSS, %eax ltr %ax +#ifdef CONFIG_AMD_MEM_ENCRYPT + /* Check if the C-bit position is correct when SEV is active */ + call startup32_check_sev_cbit +#endif + /* * Setup for the jump to 64bit mode *
@@ -276,8 +281,6 @@ SYM_FUNC_START(startup_32) leal rva(startup_64_mixed_mode)(%ebp), %eax 1: #endif - /* Check if the C-bit position is correct when SEV is active */ - call startup32_check_sev_cbit pushl $__KERNEL_CS pushl %eax
@@ -732,16 +735,17 @@ SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end) * succeed. An incorrect C-bit position will map all memory unencrypted, so that * the compare will use the encrypted random data and fail. */ - __HEAD -SYM_FUNC_START(startup32_check_sev_cbit) #ifdef CONFIG_AMD_MEM_ENCRYPT - pushl %eax + .text +SYM_FUNC_START(startup32_check_sev_cbit) pushl %ebx - pushl %ecx - pushl %edx + pushl %ebp + + call 0f +0: popl %ebp /* Check for non-zero sev_status */ - movl rva(sev_status)(%ebp), %eax + movl (sev_status - 0b)(%ebp), %eax testl %eax, %eax jz 4f
@@ -756,17 +760,18 @@ SYM_FUNC_START(startup32_check_sev_cbit) jnc 2b /* Store to memory and keep it in the registers */ - movl %eax, rva(sev_check_data)(%ebp) - movl %ebx, rva(sev_check_data+4)(%ebp) + leal (sev_check_data - 0b)(%ebp), %ebp + movl %eax, 0(%ebp) + movl %ebx, 4(%ebp) /* Enable paging to see if encryption is active */ movl %cr0, %edx /* Backup %cr0 in %edx */ movl $(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */ movl %ecx, %cr0 - cmpl %eax, rva(sev_check_data)(%ebp) + cmpl %eax, 0(%ebp) jne 3f - cmpl %ebx, rva(sev_check_data+4)(%ebp) + cmpl %ebx, 4(%ebp) jne 3f movl %edx, %cr0 /* Restore previous %cr0 */
@@ -778,13 +783,11 @@ SYM_FUNC_START(startup32_check_sev_cbit) jmp 3b 4: - popl %edx - popl %ecx + popl %ebp popl %ebx - popl %eax -#endif RET SYM_FUNC_END(startup32_check_sev_cbit) +#endif /* * Stack and heap for uncompression
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Move startup32_check_sev_cbit() into .text
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: b5d854cd4b6a314edd6c15dabc4233b84a0f8e5e Gitweb: https://git.kernel.org/tip/b5d854cd4b6a314edd6c15dabc4233b84a0f8e5e Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:13 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00
x86/boot/compressed: Move startup32_check_sev_cbit() into .text
Move startup32_check_sev_cbit() into the .text section and turn it into an ordinary function using the ordinary 32-bit calling convention, instead of saving/restoring the registers that are known to be live at the only call site. This improves maintainability, and makes it possible to move this function out of head_64.S and into a separate compilation unit that is specific to memory encryption.
Note that this requires the call site to be moved before the mixed mode check, as %eax will be live otherwise.
--- arch/x86/boot/compressed/head_64.S | 35 +++++++++++++++-------------- 1 file changed, 19 insertions(+), 16 deletions(-)
arch/x86/boot/compressed/head_64.S +19 -16
@@ -251,6 +251,11 @@ SYM_FUNC_START(startup_32) movl $__BOOT_TSS, %eax ltr %ax +#ifdef CONFIG_AMD_MEM_ENCRYPT + /* Check if the C-bit position is correct when SEV is active */ + call startup32_check_sev_cbit +#endif + /* * Setup for the jump to 64bit mode *
@@ -268,8 +273,6 @@ SYM_FUNC_START(startup_32) leal rva(startup_64_mixed_mode)(%ebp), %eax 1: #endif - /* Check if the C-bit position is correct when SEV is active */ - call startup32_check_sev_cbit pushl $__KERNEL_CS pushl %eax
@@ -724,16 +727,17 @@ SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end) * succeed. An incorrect C-bit position will map all memory unencrypted, so that * the compare will use the encrypted random data and fail. */ - __HEAD -SYM_FUNC_START(startup32_check_sev_cbit) #ifdef CONFIG_AMD_MEM_ENCRYPT - pushl %eax + .text +SYM_FUNC_START(startup32_check_sev_cbit) pushl %ebx - pushl %ecx - pushl %edx + pushl %ebp + + call 0f +0: popl %ebp /* Check for non-zero sev_status */ - movl rva(sev_status)(%ebp), %eax + movl (sev_status - 0b)(%ebp), %eax testl %eax, %eax jz 4f
@@ -748,17 +752,18 @@ SYM_FUNC_START(startup32_check_sev_cbit) jnc 2b /* Store to memory and keep it in the registers */ - movl %eax, rva(sev_check_data)(%ebp) - movl %ebx, rva(sev_check_data+4)(%ebp) + leal (sev_check_data - 0b)(%ebp), %ebp + movl %eax, 0(%ebp) + movl %ebx, 4(%ebp) /* Enable paging to see if encryption is active */ movl %cr0, %edx /* Backup %cr0 in %edx */ movl $(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */ movl %ecx, %cr0 - cmpl %eax, rva(sev_check_data)(%ebp) + cmpl %eax, 0(%ebp) jne 3f - cmpl %ebx, rva(sev_check_data+4)(%ebp) + cmpl %ebx, 4(%ebp) jne 3f movl %edx, %cr0 /* Restore previous %cr0 */
@@ -770,13 +775,11 @@ SYM_FUNC_START(startup32_check_sev_cbit) jmp 3b 4: - popl %edx - popl %ecx + popl %ebp popl %ebx - popl %eax -#endif RET SYM_FUNC_END(startup32_check_sev_cbit) +#endif /* * Stack and heap for uncompression
Ard Biesheuvel
maintainerPATCH v3 14/17 x86/compressed: move startup32_check_sev_cbit() out of head_64.S
In reply to Ard BiesheuvelNow that the startup32_check_sev_cbit() routine can execute from anywhere and behaves like an ordinary function, we no longer need to keep it in head_64.S.
--- arch/x86/boot/compressed/head_64.S | 71 -------------------- arch/x86/boot/compressed/mem_encrypt.S | 68 +++++++++++++++++++ 2 files changed, 68 insertions(+), 71 deletions(-)
arch/x86/boot/compressed/head_64.S +0 -71
@@ -718,77 +718,6 @@ SYM_DATA_START(boot_idt) .endr SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end) -/* - * Check for the correct C-bit position when the startup_32 boot-path is used. - * - * The check makes use of the fact that all memory is encrypted when paging is - * disabled. The function creates 64 bits of random data using the RDRAND - * instruction. RDRAND is mandatory for SEV guests, so always available. If the - * hypervisor violates that the kernel will crash right here. - * - * The 64 bits of random data are stored to a memory location and at the same - * time kept in the %eax and %ebx registers. Since encryption is always active - * when paging is off the random data will be stored encrypted in main memory. - * - * Then paging is enabled. When the C-bit position is correct all memory is - * still mapped encrypted and comparing the register values with memory will - * succeed. An incorrect C-bit position will map all memory unencrypted, so that - * the compare will use the encrypted random data and fail. - */ -#ifdef CONFIG_AMD_MEM_ENCRYPT - .text -SYM_FUNC_START(startup32_check_sev_cbit) - pushl %ebx - pushl %ebp - - call 0f -0: popl %ebp - - /* Check for non-zero sev_status */ - movl (sev_status - 0b)(%ebp), %eax - testl %eax, %eax - jz 4f - - /* - * Get two 32-bit random values - Don't bail out if RDRAND fails - * because it is better to prevent forward progress if no random value - * can be gathered. - */ -1: rdrand %eax - jnc 1b -2: rdrand %ebx - jnc 2b - - /* Store to memory and keep it in the registers */ - leal (sev_check_data - 0b)(%ebp), %ebp - movl %eax, 0(%ebp) - movl %ebx, 4(%ebp) - - /* Enable paging to see if encryption is active */ - movl %cr0, %edx /* Backup %cr0 in %edx */ - movl $(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */ - movl %ecx, %cr0 - - cmpl %eax, 0(%ebp) - jne 3f - cmpl %ebx, 4(%ebp) - jne 3f - - movl %edx, %cr0 /* Restore previous %cr0 */ - - jmp 4f - -3: /* Check failed - hlt the machine */ - hlt - jmp 3b - -4: - popl %ebp - popl %ebx - RET -SYM_FUNC_END(startup32_check_sev_cbit) -#endif - /* * Stack and heap for uncompression */
arch/x86/boot/compressed/mem_encrypt.S +68 -0
74 lines not on this page · raw message
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Move startup32_check_sev_cbit() out of head_64.S
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 9d7eaae6a071ff1f718e0aa5e610bb712f8cc632 Gitweb: https://git.kernel.org/tip/9d7eaae6a071ff1f718e0aa5e610bb712f8cc632 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:14 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00
x86/boot/compressed: Move startup32_check_sev_cbit() out of head_64.S
Now that the startup32_check_sev_cbit() routine can execute from anywhere and behaves like an ordinary function, it can be moved where it belongs.
--- arch/x86/boot/compressed/head_64.S | 71 +------------------------- arch/x86/boot/compressed/mem_encrypt.S | 68 ++++++++++++++++++++++++- 2 files changed, 68 insertions(+), 71 deletions(-)
arch/x86/boot/compressed/head_64.S +0 -71
@@ -711,77 +711,6 @@ SYM_DATA_START(boot_idt) SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end) /* - * Check for the correct C-bit position when the startup_32 boot-path is used. - * - * The check makes use of the fact that all memory is encrypted when paging is - * disabled. The function creates 64 bits of random data using the RDRAND - * instruction. RDRAND is mandatory for SEV guests, so always available. If the - * hypervisor violates that the kernel will crash right here. - * - * The 64 bits of random data are stored to a memory location and at the same - * time kept in the %eax and %ebx registers. Since encryption is always active - * when paging is off the random data will be stored encrypted in main memory. - * - * Then paging is enabled. When the C-bit position is correct all memory is - * still mapped encrypted and comparing the register values with memory will - * succeed. An incorrect C-bit position will map all memory unencrypted, so that - * the compare will use the encrypted random data and fail. - */ -#ifdef CONFIG_AMD_MEM_ENCRYPT - .text -SYM_FUNC_START(startup32_check_sev_cbit) - pushl %ebx - pushl %ebp - - call 0f -0: popl %ebp - - /* Check for non-zero sev_status */ - movl (sev_status - 0b)(%ebp), %eax - testl %eax, %eax - jz 4f - - /* - * Get two 32-bit random values - Don't bail out if RDRAND fails - * because it is better to prevent forward progress if no random value - * can be gathered. - */ -1: rdrand %eax - jnc 1b -2: rdrand %ebx - jnc 2b - - /* Store to memory and keep it in the registers */ - leal (sev_check_data - 0b)(%ebp), %ebp - movl %eax, 0(%ebp) - movl %ebx, 4(%ebp) - - /* Enable paging to see if encryption is active */ - movl %cr0, %edx /* Backup %cr0 in %edx */ - movl $(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */ - movl %ecx, %cr0 - - cmpl %eax, 0(%ebp) - jne 3f - cmpl %ebx, 4(%ebp) - jne 3f - - movl %edx, %cr0 /* Restore previous %cr0 */ - - jmp 4f - -3: /* Check failed - hlt the machine */ - hlt - jmp 3b - -4: - popl %ebp - popl %ebx - RET -SYM_FUNC_END(startup32_check_sev_cbit) -#endif - -/* * Stack and heap for uncompression */ .bss
arch/x86/boot/compressed/mem_encrypt.S +68 -0
74 lines not on this page · raw message
Ard Biesheuvel
maintainerPATCH v3 15/17 x86/compressed: adhere to calling convention in get_sev_encryption_bit()
In reply to Ard BiesheuvelMake get_sev_encryption_bit() follow the ordinary i386 calling convention, and only call it if CONFIG_AMD_MEM_ENCRYPT is actually enabled. This clarifies the calling code, and makes it more maintainable.
--- arch/x86/boot/compressed/head_64.S | 5 +++-- arch/x86/boot/compressed/mem_encrypt.S | 10 ---------- 2 files changed, 3 insertions(+), 12 deletions(-)
arch/x86/boot/compressed/head_64.S +3 -2
@@ -188,12 +188,13 @@ SYM_FUNC_START(startup_32) */ /* * If SEV is active then set the encryption mask in the page tables. - * This will insure that when the kernel is copied and decompressed + * This will ensure that when the kernel is copied and decompressed * it will be done so encrypted. */ - call get_sev_encryption_bit xorl %edx, %edx #ifdef CONFIG_AMD_MEM_ENCRYPT + call get_sev_encryption_bit + xorl %edx, %edx testl %eax, %eax jz 1f subl $32, %eax /* Encryption bit is always above bit 31 */
arch/x86/boot/compressed/mem_encrypt.S +0 -10
@@ -18,12 +18,7 @@ .text .code32 SYM_FUNC_START(get_sev_encryption_bit) - xor %eax, %eax - -#ifdef CONFIG_AMD_MEM_ENCRYPT push %ebx - push %ecx - push %edx movl $0x80000000, %eax /* CPUID to check the highest leaf */ cpuid
@@ -54,12 +49,7 @@ SYM_FUNC_START(get_sev_encryption_bit) xor %eax, %eax .Lsev_exit: - pop %edx - pop %ecx pop %ebx - -#endif /* CONFIG_AMD_MEM_ENCRYPT */ - RET SYM_FUNC_END(get_sev_encryption_bit)
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Adhere to calling convention in get_sev_encryption_bit()
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 30c9ca16a5271ba6f8ad9c86507ff1c789c94677 Gitweb: https://git.kernel.org/tip/30c9ca16a5271ba6f8ad9c86507ff1c789c94677 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:15 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00
x86/boot/compressed: Adhere to calling convention in get_sev_encryption_bit()
Make get_sev_encryption_bit() follow the ordinary i386 calling convention, and only call it if CONFIG_AMD_MEM_ENCRYPT is actually enabled. This clarifies the calling code, and makes it more maintainable.
--- arch/x86/boot/compressed/head_64.S | 5 +++-- arch/x86/boot/compressed/mem_encrypt.S | 10 ---------- 2 files changed, 3 insertions(+), 12 deletions(-)
arch/x86/boot/compressed/head_64.S +3 -2
@@ -180,12 +180,13 @@ SYM_FUNC_START(startup_32) */ /* * If SEV is active then set the encryption mask in the page tables. - * This will insure that when the kernel is copied and decompressed + * This will ensure that when the kernel is copied and decompressed * it will be done so encrypted. */ - call get_sev_encryption_bit xorl %edx, %edx #ifdef CONFIG_AMD_MEM_ENCRYPT + call get_sev_encryption_bit + xorl %edx, %edx testl %eax, %eax jz 1f subl $32, %eax /* Encryption bit is always above bit 31 */
arch/x86/boot/compressed/mem_encrypt.S +0 -10
@@ -18,12 +18,7 @@ .text .code32 SYM_FUNC_START(get_sev_encryption_bit) - xor %eax, %eax - -#ifdef CONFIG_AMD_MEM_ENCRYPT push %ebx - push %ecx - push %edx movl $0x80000000, %eax /* CPUID to check the highest leaf */ cpuid
@@ -54,12 +49,7 @@ SYM_FUNC_START(get_sev_encryption_bit) xor %eax, %eax .Lsev_exit: - pop %edx - pop %ecx pop %ebx - -#endif /* CONFIG_AMD_MEM_ENCRYPT */ - RET SYM_FUNC_END(get_sev_encryption_bit)
Ard Biesheuvel
maintainerPATCH v3 16/17 x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y
In reply to Ard BiesheuvelAvoid building the mem_encrypt.o object if memory encryption support is not enabled to begin with.
--- arch/x86/boot/compressed/Makefile | 2 +- arch/x86/boot/compressed/mem_encrypt.S | 2 -- 2 files changed, 1 insertion(+), 3 deletions(-)
arch/x86/boot/compressed/Makefile +1 -1
@@ -100,7 +100,7 @@ vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/kaslr.o ifdef CONFIG_X86_64 vmlinux-objs-y += $(obj)/ident_map_64.o vmlinux-objs-y += $(obj)/idt_64.o $(obj)/idt_handlers_64.o - vmlinux-objs-y += $(obj)/mem_encrypt.o + vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/mem_encrypt.o vmlinux-objs-y += $(obj)/pgtable_64.o vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/sev.o endif
arch/x86/boot/compressed/mem_encrypt.S +0 -2
@@ -307,7 +307,6 @@ SYM_FUNC_END(startup32_check_sev_cbit) .data -#ifdef CONFIG_AMD_MEM_ENCRYPT .balign 8 SYM_DATA(sme_me_mask, .quad 0) SYM_DATA(sev_status, .quad 0)
@@ -323,4 +322,3 @@ SYM_DATA_START_LOCAL(boot32_idt_desc) .word . - boot32_idt - 1 .long 0 SYM_DATA_END(boot32_idt_desc) -#endif
Signature · 1 line
2.35.1
[tip: x86/boot] x86/boot/compressed: Only build mem_encrypt.S if AMD_MEM_ENCRYPT=y
The following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: 61de13df95901bc58456bc5acdbd3c18c66cf859 Gitweb: https://git.kernel.org/tip/61de13df95901bc58456bc5acdbd3c18c66cf859 Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:16 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00
x86/boot/compressed: Only build mem_encrypt.S if AMD_MEM_ENCRYPT=y
Avoid building the mem_encrypt.o object if memory encryption support is not enabled to begin with.
--- arch/x86/boot/compressed/Makefile | 2 +- arch/x86/boot/compressed/mem_encrypt.S | 2 -- 2 files changed, 1 insertion(+), 3 deletions(-)
arch/x86/boot/compressed/Makefile +1 -1
@@ -100,7 +100,7 @@ vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/kaslr.o ifdef CONFIG_X86_64 vmlinux-objs-y += $(obj)/ident_map_64.o vmlinux-objs-y += $(obj)/idt_64.o $(obj)/idt_handlers_64.o - vmlinux-objs-y += $(obj)/mem_encrypt.o + vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/mem_encrypt.o vmlinux-objs-y += $(obj)/pgtable_64.o vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/sev.o endif
arch/x86/boot/compressed/mem_encrypt.S +0 -2
@@ -307,7 +307,6 @@ SYM_FUNC_END(startup32_check_sev_cbit) .data -#ifdef CONFIG_AMD_MEM_ENCRYPT .balign 8 SYM_DATA(sme_me_mask, .quad 0) SYM_DATA(sev_status, .quad 0)
@@ -323,4 +322,3 @@ SYM_DATA_START_LOCAL(boot32_idt_desc) .word . - boot32_idt - 1 .long 0 SYM_DATA_END(boot32_idt_desc) -#endif
Ard Biesheuvel
maintainerPATCH v3 17/17 efi: x86: Make the deprecated EFI handover protocol optional
In reply to Ard BiesheuvelThe EFI handover protocol permits a bootloader to invoke the kernel as a EFI PE/COFF application, while passing a bootparams struct as a third argument to the entrypoint function call.
This has no basis in the UEFI specification, and there are better ways to pass additional data to a UEFI application (UEFI configuration tables, UEFI variables, UEFI protocols) than going around the StartImage() boot service and jumping to a fixed offset in the loaded image, just to call a different function that takes a third parameter.
The reason for handling struct bootparams in the bootloader was that the EFI stub could only load initrd images from the EFI system partition, and so passing it via struct bootparams was needed for loaders like GRUB, which pass the initrd in memory, and may load it from anywhere, including from the network. Another motivation was EFI mixed mode, which could not use the initrd loader in the EFI stub at all due to 32/64 bit incompatibilities (which will be fixed shortly [0]), and could not invoke the ordinary PE/COFF entry point either, for the same reasons.
Given that loaders such as GRUB already carried the bootparams handling in order to implement non-EFI boot, retaining that code and just passing bootparams to the EFI stub was a reasonable choice (although defining an alternate entrypoint could have been avoided.) However, the GRUB side changes never made it upstream, and are only shipped by some of the distros in their downstream versions.
In the meantime, EFI support has been added to other Linux architecture ports, as well as to U-boot and systemd, including arch-agnostic methods for passing initrd images in memory [1], and for doing mixed mode boot [2], none of them requiring anything like the EFI handover protocol. So given that only out-of-tree distro GRUB relies on this, let's permit it to be omitted from the build, in preparation for retiring it completely at a later date. (Note that systemd-boot does have an implementation as well, but only uses it as a fallback for booting images that do not implement the LoadFile2 based initrd loading method, i.e., v5.8 or older)
[0] https://lore.kernel.org/all/[email protected]/
[1] ec93fc371f01 ("efi/libstub: Add support for loading the initrd ...")
[2] 97aa276579b2 ("efi/x86: Add true mixed mode entry point into ...")
--- arch/x86/Kconfig | 17 +++++++++++++++++ arch/x86/boot/compressed/head_64.S | 4 +++- arch/x86/boot/header.S | 2 +- arch/x86/boot/tools/build.c | 2 ++ 4 files changed, 23 insertions(+), 2 deletions(-)
arch/x86/Kconfig +17 -0
@@ -1981,6 +1981,23 @@ config EFI_STUB See Documentation/admin-guide/efi-stub.rst for more information. +config EFI_HANDOVER_PROTOCOL + bool "EFI handover protocol (DEPRECATED)" + depends on EFI_STUB + default y + help + Select this in order to include support for the deprecated EFI + handover protocol, which defines alternative entry points into the + EFI stub. This is a practice that has no basis in the UEFI + specification, and requires a priori knowledge on the part of the + bootloader about Linux/x86 specific ways of passing the command line + and initrd, and where in memory those assets may be loaded. + + If in doubt, say Y. Even though he corresponding support is not + present in upstream GRUB or other bootloaders, most distros build + GRUB with numerous downstream patches applied, and may rely on the + handover protocol as as result. + config EFI_MIXED bool "EFI mixed-mode support" depends on EFI_STUB && X86_64
arch/x86/boot/compressed/head_64.S +3 -1
@@ -294,7 +294,7 @@ SYM_FUNC_START(startup_32) lret SYM_FUNC_END(startup_32) -#ifdef CONFIG_EFI_MIXED +#if IS_ENABLED(CONFIG_EFI_MIXED) && IS_ENABLED(CONFIG_EFI_HANDOVER_PROTOCOL) .org 0x190 SYM_FUNC_START(efi32_stub_entry) add $0x4, %esp /* Discard return address */
@@ -524,7 +524,9 @@ trampoline_return: SYM_CODE_END(startup_64) #ifdef CONFIG_EFI_STUB +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL .org 0x390 +#endif SYM_FUNC_START(efi64_stub_entry) and $~0xf, %rsp /* realign the stack */ movq %rdx, %rbx /* save boot_params pointer */
arch/x86/boot/header.S +1 -1
@@ -406,7 +406,7 @@ xloadflags: # define XLF1 0 #endif -#ifdef CONFIG_EFI_STUB +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL # ifdef CONFIG_EFI_MIXED # define XLF23 (XLF_EFI_HANDOVER_32|XLF_EFI_HANDOVER_64) # else
arch/x86/boot/tools/build.c +2 -0
@@ -290,6 +290,7 @@ static void efi_stub_entry_update(void) {unsigned long addr = efi32_stub_entry; +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL #ifdef CONFIG_X86_64 /* Yes, this is really how we defined it :( */ addr = efi64_stub_entry -0x200 ;
@@ -298,6 +299,7 @@ static void efi_stub_entry_update(void)#ifdef CONFIG_EFI_MIXED if (efi32_stub_entry != addr)die ("32-bit and 64-bit EFI entry points do not match\n" ); +#endif #endif put_unaligned_le32 (addr, &buf[0x264 ]); }
Signature · 1 line
2.35.1
Re: [PATCH v3 17/17] efi: x86: Make the deprecated EFI handover protocol optional
Quoted 11 lines · Ard Biesheuvel
arch/x86/Kconfig +3
-0@@ -1981,6 +1981,23 @@ config EFI_STUB See Documentation/admin-guide/efi-stub.rst for more information. +config EFI_HANDOVER_PROTOCOL + bool "EFI handover protocol (DEPRECATED)" + depends on EFI_STUB
arch/x86/Kconfig config EFI_STUB · line 1996
Quoted 10 lines
arch/x86/Kconfig +10
-0+ default y + help + Select this in order to include support for the deprecated EFI + handover protocol, which defines alternative entry points into the + EFI stub. This is a practice that has no basis in the UEFI + specification, and requires a priori knowledge on the part of the + bootloader about Linux/x86 specific ways of passing the command line + and initrd, and where in memory those assets may be loaded. + + If in doubt, say Y. Even though he corresponding support is not
the
Quoted 3 lines
arch/x86/Kconfig +3
-0+ present in upstream GRUB or other bootloaders, most distros build + GRUB with numerous downstream patches applied, and may rely on the + handover protocol as as result.
Signature · 1 line
~Randy
[tip: x86/boot] x86/efi: Make the deprecated EFI handover protocol optional
In reply to Ard BiesheuvelThe following commit has been merged into the x86/boot branch of tip:
The following commit has been merged into the x86/boot branch of tip:
Commit-ID: cc3fdda2876e58a7e83e558ab51853cf106afb6a Gitweb: https://git.kernel.org/tip/cc3fdda2876e58a7e83e558ab51853cf106afb6a Author: Ard Biesheuvel <[email protected]> AuthorDate: Tue, 22 Nov 2022 17:10:17 +01:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 24 Nov 2022 08:57:41 +01:00
x86/efi: Make the deprecated EFI handover protocol optional
The EFI handover protocol permits a bootloader to invoke the kernel as a EFI PE/COFF application, while passing a bootparams struct as a third argument to the entrypoint function call.
This has no basis in the UEFI specification, and there are better ways to pass additional data to a UEFI application (UEFI configuration tables, UEFI variables, UEFI protocols) than going around the StartImage() boot service and jumping to a fixed offset in the loaded image, just to call a different function that takes a third parameter.
The reason for handling struct bootparams in the bootloader was that the EFI stub could only load initrd images from the EFI system partition, and so passing it via struct bootparams was needed for loaders like GRUB, which pass the initrd in memory, and may load it from anywhere, including from the network. Another motivation was EFI mixed mode, which could not use the initrd loader in the EFI stub at all due to 32/64 bit incompatibilities (which will be fixed shortly [0]), and could not invoke the ordinary PE/COFF entry point either, for the same reasons.
Given that loaders such as GRUB already carried the bootparams handling in order to implement non-EFI boot, retaining that code and just passing bootparams to the EFI stub was a reasonable choice (although defining an alternate entrypoint could have been avoided.) However, the GRUB side changes never made it upstream, and are only shipped by some of the distros in their downstream versions.
In the meantime, EFI support has been added to other Linux architecture ports, as well as to U-boot and systemd, including arch-agnostic methods for passing initrd images in memory [1], and for doing mixed mode boot [2], none of them requiring anything like the EFI handover protocol. So given that only out-of-tree distro GRUB relies on this, let's permit it to be omitted from the build, in preparation for retiring it completely at a later date. (Note that systemd-boot does have an implementation as well, but only uses it as a fallback for booting images that do not implement the LoadFile2 based initrd loading method, i.e., v5.8 or older)
[0] https://lore.kernel.org/all/[email protected]/
[1] ec93fc371f01 ("efi/libstub: Add support for loading the initrd from a device path")
[2] 97aa276579b2 ("efi/x86: Add true mixed mode entry point into .compat section")
--- arch/x86/Kconfig | 17 +++++++++++++++++ arch/x86/boot/compressed/head_64.S | 4 +++- arch/x86/boot/header.S | 2 +- arch/x86/boot/tools/build.c | 2 ++ 4 files changed, 23 insertions(+), 2 deletions(-)
arch/x86/Kconfig +17 -0
@@ -1980,6 +1980,23 @@ config EFI_STUB See Documentation/admin-guide/efi-stub.rst for more information. +config EFI_HANDOVER_PROTOCOL + bool "EFI handover protocol (DEPRECATED)" + depends on EFI_STUB + default y + help + Select this in order to include support for the deprecated EFI + handover protocol, which defines alternative entry points into the + EFI stub. This is a practice that has no basis in the UEFI + specification, and requires a priori knowledge on the part of the + bootloader about Linux/x86 specific ways of passing the command line + and initrd, and where in memory those assets may be loaded. + + If in doubt, say Y. Even though the corresponding support is not + present in upstream GRUB or other bootloaders, most distros build + GRUB with numerous downstream patches applied, and may rely on the + handover protocol as as result. + config EFI_MIXED bool "EFI mixed-mode support" depends on EFI_STUB && X86_64
arch/x86/boot/compressed/head_64.S +3 -1
@@ -286,7 +286,7 @@ SYM_FUNC_START(startup_32) lret SYM_FUNC_END(startup_32) -#ifdef CONFIG_EFI_MIXED +#if IS_ENABLED(CONFIG_EFI_MIXED) && IS_ENABLED(CONFIG_EFI_HANDOVER_PROTOCOL) .org 0x190 SYM_FUNC_START(efi32_stub_entry) add $0x4, %esp /* Discard return address */
@@ -516,7 +516,9 @@ trampoline_return: SYM_CODE_END(startup_64) #ifdef CONFIG_EFI_STUB +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL .org 0x390 +#endif SYM_FUNC_START(efi64_stub_entry) and $~0xf, %rsp /* realign the stack */ movq %rdx, %rbx /* save boot_params pointer */
arch/x86/boot/header.S +1 -1
@@ -406,7 +406,7 @@ xloadflags: # define XLF1 0 #endif -#ifdef CONFIG_EFI_STUB +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL # ifdef CONFIG_EFI_MIXED # define XLF23 (XLF_EFI_HANDOVER_32|XLF_EFI_HANDOVER_64) # else
arch/x86/boot/tools/build.c +2 -0
@@ -290,6 +290,7 @@ static void efi_stub_entry_update(void) {unsigned long addr = efi32_stub_entry; +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL #ifdef CONFIG_X86_64 /* Yes, this is really how we defined it :( */ addr = efi64_stub_entry -0x200 ;
@@ -299,6 +300,7 @@ static void efi_stub_entry_update(void)if (efi32_stub_entry != addr)die ("32-bit and 64-bit EFI entry points do not match\n" );#endif +#endif put_unaligned_le32 (addr, &buf[0x264 ]); }
Quoted 14 lines · Ard Biesheuvel
After doing some cleanup work on the EFI code in
head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/masterChanges since v1:
- at Boris's request, split the patches into smaller ones that are easier to review
Quoted 5 lines
Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc Michael Roth <[email protected]>
This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in arch/x86/boot/compressed/head_64.S of startup_64).
Here is the Qemu register dump:
RAX=00000000029cc260 RBX=ffffffffdd98c000 RCX=0000000000000010 RDX=0000000000000002 RSI=000000003dec1000 RDI=0000000000000000 RBP=ffffffffdb000000 RSP=ffffffffde36e000 R8 =000000003dec1410 R9 =000000003dec13fc R10=000000000000006c R11=0000000000000000 R12=0000000000000000 R13=0000000000000001 R14=0000000000000004 R15=000000003eacdf44 RIP=0000000002000263 RFL=00200002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 00000000 00000000
CS =0038 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA]
SS =0000 0000000000000000 00000000 00000000
DS =0000 0000000000000000 00000000 00000000
FS =0000 0000000000000000 00000000 00000000
GS =0000 0000000000000000 00000000 00000000
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT= 00000000029cc270 0000002f
IDT= 000000003f55e018 00000fff
CR0=80010033 CR2=ffffffffde36dff8 CR3=000000003fc01000 CR4=00000668 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000d00
Thanks,
Tom
Quoted 38 lines
Ard Biesheuvel (17):
x86/compressed: efi-mixed: renameefi_thunk_64.Stoefi-mixed.Sx86/compressed: efi-mixed: move 32-bit entrypoint code into .text
section
x86/compressed: efi-mixed: move bootargs parsing out of 32-bit startup
code
x86/compressed: efi-mixed: moveefi32_pe_entryinto .text section x86/compressed: efi-mixed: moveefi32_entryout ofhead_64.Sx86/compressed: efi-mixed: moveefi32_pe_entry()out ofhead_64.Sx86/compressed: efi: merge multiple definitions ofimage_offsetinto
one
x86/compressed: efi-mixed: simplify IDT/GDT preserve/restore x86/compressed: avoid touching ECX instartup32_set_idt_entry()x86/compressed: pull global variable ref up intostartup32_load_idt()x86/compressed: movestartup32_load_idt()into .text section x86/compressed: movestartup32_load_idt()out ofhead_64.Sx86/compressed: movestartup32_check_sev_cbit()into .text x86/compressed: movestartup32_check_sev_cbit()out ofhead_64.Sx86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only buildmem_encrypt.Sif AMD_MEM_ENCRYPT=y
efi: x86: Make the deprecated EFI handover protocol optionalarch/x86/Kconfig | 17 + arch/x86/boot/compressed/Makefile | 8 +- arch/x86/boot/compressed/efi_mixed.S | 351 ++++++++++++++++++++ arch/x86/boot/compressed/efi_thunk_64.S | 195 ----------- arch/x86/boot/compressed/head_32.S | 4 - arch/x86/boot/compressed/head_64.S | 303 +---------------- arch/x86/boot/compressed/mem_encrypt.S | 152 ++++++++- arch/x86/boot/header.S | 2 +- arch/x86/boot/tools/build.c | 2 + drivers/firmware/efi/libstub/x86-stub.c | 2 +- 10 files changed, 533 insertions(+), 503 deletions(-)create mode 100644
arch/x86/boot/compressed/efi_mixed.Sdelete mode 100644arch/x86/boot/compressed/efi_thunk_64.S
Ard Biesheuvel
maintainerQuoted 22 lines · Tom Lendacky
Quoted 14 lines · Ard Biesheuvel
After doing some cleanup work on the EFI code in
head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/masterChanges since v1:
- at Boris's request, split the patches into smaller ones that are easier to reviewQuoted 5 lines
Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc Michael Roth <[email protected]>
Quoted 5 lines
This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in
arch/x86/boot/compressed/head_64.Sofstartup_64).
Thanks for the report.
So the mystery here (at least to me) is that all the changes are to the 32-bit code, and startup_64 reloads the stack pointer from the symbol
Does your config have CONFIG_EFI_MIXED enabled?
Can I reproduce this fully emulated with QEMU? Or do I need a SEV host?
Quoted 62 lines
Here is the Qemu register dump:
RAX=00000000029cc260 RBX=ffffffffdd98c000 RCX=0000000000000010 RDX=0000000000000002 RSI=000000003dec1000 RDI=0000000000000000 RBP=ffffffffdb000000 RSP=ffffffffde36e000 R8 =000000003dec1410 R9 =000000003dec13fc R10=000000000000006c R11=0000000000000000 R12=0000000000000000 R13=0000000000000001 R14=0000000000000004 R15=000000003eacdf44 RIP=0000000002000263 RFL=00200002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 00000000 00000000
CS =0038 0000000000000000 ffffffff00a09b00DPL=0 CS64 [-RA]
SS =0000 0000000000000000 00000000 00000000
DS =0000 0000000000000000 00000000 00000000
FS =0000 0000000000000000 00000000 00000000
GS =0000 0000000000000000 00000000 00000000
LDT=0000 00000000000000000000ffff00008200 DPL=0 LDT
TR =0000 00000000000000000000ffff00008b00DPL=0 TSS64-busy
GDT=00000000029cc2700000002f
IDT=000000003f55e01800000fff
CR0=80010033 CR2=ffffffffde36dff8 CR3=000000003fc01000 CR4=00000668 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000d00Thanks,
TomQuoted 38 lines
Ard Biesheuvel (17):
x86/compressed: efi-mixed: renameefi_thunk_64.Stoefi-mixed.Sx86/compressed: efi-mixed: move 32-bit entrypoint code into .text
section
x86/compressed: efi-mixed: move bootargs parsing out of 32-bit startup
code
x86/compressed: efi-mixed: moveefi32_pe_entryinto .text section x86/compressed: efi-mixed: moveefi32_entryout ofhead_64.Sx86/compressed: efi-mixed: moveefi32_pe_entry()out ofhead_64.Sx86/compressed: efi: merge multiple definitions ofimage_offsetinto
one
x86/compressed: efi-mixed: simplify IDT/GDT preserve/restore x86/compressed: avoid touching ECX instartup32_set_idt_entry()x86/compressed: pull global variable ref up intostartup32_load_idt()x86/compressed: movestartup32_load_idt()into .text section x86/compressed: movestartup32_load_idt()out ofhead_64.Sx86/compressed: movestartup32_check_sev_cbit()into .text x86/compressed: movestartup32_check_sev_cbit()out ofhead_64.Sx86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only buildmem_encrypt.Sif AMD_MEM_ENCRYPT=y
efi: x86: Make the deprecated EFI handover protocol optionalarch/x86/Kconfig | 17 + arch/x86/boot/compressed/Makefile | 8 +- arch/x86/boot/compressed/efi_mixed.S | 351 ++++++++++++++++++++ arch/x86/boot/compressed/efi_thunk_64.S | 195 ----------- arch/x86/boot/compressed/head_32.S | 4 - arch/x86/boot/compressed/head_64.S | 303 +---------------- arch/x86/boot/compressed/mem_encrypt.S | 152 ++++++++- arch/x86/boot/header.S | 2 +- arch/x86/boot/tools/build.c | 2 + drivers/firmware/efi/libstub/x86-stub.c | 2 +- 10 files changed, 533 insertions(+), 503 deletions(-)create mode 100644
arch/x86/boot/compressed/efi_mixed.Sdelete mode 100644arch/x86/boot/compressed/efi_thunk_64.S
Ard Biesheuvel
maintainerQuoted 32 lines · Ard Biesheuvel
Quoted 22 lines · Tom Lendacky
Quoted 14 lines · Ard Biesheuvel
After doing some cleanup work on the EFI code in
head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/masterChanges since v1:
- at Boris's request, split the patches into smaller ones that are easier to reviewQuoted 5 lines
Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc Michael Roth <[email protected]>Quoted 5 lines
This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in
arch/x86/boot/compressed/head_64.Sofstartup_64).Thanks for the report.
Quoted 8 lines
So the mystery here (at least to me) is that all the changes are to the 32-bit code, and
startup_64reloads the stack pointer from the symbolDoes your config have
CONFIG_EFI_MIXEDenabled?Can I reproduce this fully emulated with QEMU? Or do I need a SEV host?
Also, mind giving this a quick spin?
(quoted diff) +0 -0
b/drivers/firmware/efi/libstub/x86-stub.c
index cb5f0befee57..1af11d34bc6c 100644
drivers/firmware/efi/libstub/x86-stub.c +1 -1
@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -u32 image_offset; +u32 __section(".data") image_offset;static efi_loaded_image_t *image =NULL ; static efi_status_t
Thanks,
Ard.
Quoted 37 lines · Ard Biesheuvel
Quoted 36 lines · Ard Biesheuvel
Quoted 22 lines · Tom Lendacky
Quoted 14 lines · Ard Biesheuvel
After doing some cleanup work on the EFI code in
head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/masterChanges since v1:
- at Boris's request, split the patches into smaller ones that are easier to reviewQuoted 5 lines
Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc Michael Roth <[email protected]>Quoted 5 lines
This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in
arch/x86/boot/compressed/head_64.Sofstartup_64).Thanks for the report.
So the mystery here (at least to me) is that all the changes are to the 32-bit code, and
startup_64reloads the stack pointer from the symbol
Quoted 6 lines
Quoted 4 lines
Does your config have
CONFIG_EFI_MIXEDenabled?Can I reproduce this fully emulated with QEMU? Or do I need a SEV host?
Also, mind giving this a quick spin?
Just saw this after I sent out my email. Yes, this fixes it.
Thanks,
Tom
Quoted 18 lines
drivers/firmware/efi/libstub/x86-stub.c +1
-1@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -u32 image_offset; +u32 __section(".data") image_offset;static efi_loaded_image_t *image =NULL ; static efi_status_tThanks,
Ard.
Ard Biesheuvel
maintainerQuoted 39 lines · Tom Lendacky
Quoted 37 lines · Ard Biesheuvel
Quoted 36 lines · Ard Biesheuvel
Quoted 22 lines · Tom Lendacky
Quoted 14 lines · Ard Biesheuvel
After doing some cleanup work on the EFI code in
head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/masterChanges since v1:
- at Boris's request, split the patches into smaller ones that are easier to reviewQuoted 5 lines
Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc Michael Roth <[email protected]>Quoted 5 lines
This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in
arch/x86/boot/compressed/head_64.Sofstartup_64).Thanks for the report.
So the mystery here (at least to me) is that all the changes are to the 32-bit code, and
startup_64reloads the stack pointer from the symbol
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
Does your config have
CONFIG_EFI_MIXEDenabled?Can I reproduce this fully emulated with QEMU? Or do I need a SEV host?
Also, mind giving this a quick spin?
Just saw this after I sent out my email. Yes, this fixes it.
Excellent, thanks for testing.
Quoted 29 lines · Ard Biesheuvel
Quoted 22 lines · Tom Lendacky
Quoted 14 lines · Ard Biesheuvel
After doing some cleanup work on the EFI code in
head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.Changes since v2:
- add some clarifying comments to the EFI mixed mode changes
- include patch to make the EFI handover protocol optional that was sent out separately before
- rebase onto tip/masterChanges since v1:
- at Boris's request, split the patches into smaller ones that are easier to reviewQuoted 5 lines
Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc Michael Roth <[email protected]>Quoted 5 lines
This causes an SEV guest to blow up on boot in the early boot code. It looks like the stack pointer is not valid and it triple faults on a pushq instruction (pushq $__KERNEL_CS in
arch/x86/boot/compressed/head_64.Sofstartup_64).
Quoted 5 lines
Thanks for the report.
So the mystery here (at least to me) is that all the changes are to the 32-bit code, and
startup_64reloads the stack pointer from the symbol
I bisected it to:
99b7f4b23d9f ("x86/boot/compressed, efi: Merge multiple definitions of image_offset into one")
And doing the following fixed it:
drivers/firmware/efi/libstub/x86-stub.c +1 -1
@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -u32 image_offset; +u32 image_offset __section(".data");static efi_loaded_image_t *image =NULL ; static efi_status_t
I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put image_offset in the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".
Thanks,
Tom
Quoted 67 lines
Does your config have
CONFIG_EFI_MIXEDenabled?Can I reproduce this fully emulated with QEMU? Or do I need a SEV host?
Quoted 62 lines
Here is the Qemu register dump:
RAX=00000000029cc260 RBX=ffffffffdd98c000 RCX=0000000000000010 RDX=0000000000000002 RSI=000000003dec1000 RDI=0000000000000000 RBP=ffffffffdb000000 RSP=ffffffffde36e000 R8 =000000003dec1410 R9 =000000003dec13fc R10=000000000000006c R11=0000000000000000 R12=0000000000000000 R13=0000000000000001 R14=0000000000000004 R15=000000003eacdf44 RIP=0000000002000263 RFL=00200002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 00000000 00000000
CS =0038 0000000000000000 ffffffff00a09b00DPL=0 CS64 [-RA]
SS =0000 0000000000000000 00000000 00000000
DS =0000 0000000000000000 00000000 00000000
FS =0000 0000000000000000 00000000 00000000
GS =0000 0000000000000000 00000000 00000000
LDT=0000 00000000000000000000ffff00008200 DPL=0 LDT
TR =0000 00000000000000000000ffff00008b00DPL=0 TSS64-busy
GDT=00000000029cc2700000002f
IDT=000000003f55e01800000fff
CR0=80010033 CR2=ffffffffde36dff8 CR3=000000003fc01000 CR4=00000668 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000d00Thanks,
TomQuoted 38 lines
Ard Biesheuvel (17):
x86/compressed: efi-mixed: renameefi_thunk_64.Stoefi-mixed.Sx86/compressed: efi-mixed: move 32-bit entrypoint code into .text
section
x86/compressed: efi-mixed: move bootargs parsing out of 32-bit startup
code
x86/compressed: efi-mixed: moveefi32_pe_entryinto .text section x86/compressed: efi-mixed: moveefi32_entryout ofhead_64.Sx86/compressed: efi-mixed: moveefi32_pe_entry()out ofhead_64.Sx86/compressed: efi: merge multiple definitions ofimage_offsetinto
one
x86/compressed: efi-mixed: simplify IDT/GDT preserve/restore x86/compressed: avoid touching ECX instartup32_set_idt_entry()x86/compressed: pull global variable ref up intostartup32_load_idt()x86/compressed: movestartup32_load_idt()into .text section x86/compressed: movestartup32_load_idt()out ofhead_64.Sx86/compressed: movestartup32_check_sev_cbit()into .text x86/compressed: movestartup32_check_sev_cbit()out ofhead_64.Sx86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only buildmem_encrypt.Sif AMD_MEM_ENCRYPT=y
efi: x86: Make the deprecated EFI handover protocol optionalarch/x86/Kconfig | 17 + arch/x86/boot/compressed/Makefile | 8 +- arch/x86/boot/compressed/efi_mixed.S | 351 ++++++++++++++++++++ arch/x86/boot/compressed/efi_thunk_64.S | 195 ----------- arch/x86/boot/compressed/head_32.S | 4 - arch/x86/boot/compressed/head_64.S | 303 +---------------- arch/x86/boot/compressed/mem_encrypt.S | 152 ++++++++- arch/x86/boot/header.S | 2 +- arch/x86/boot/tools/build.c | 2 + drivers/firmware/efi/libstub/x86-stub.c | 2 +- 10 files changed, 533 insertions(+), 503 deletions(-)create mode 100644
arch/x86/boot/compressed/efi_mixed.Sdelete mode 100644arch/x86/boot/compressed/efi_thunk_64.S
Borislav Petkov
maintainerQuoted 18 lines · Tom Lendacky
I bisected it to:
99b7f4b23d9f("x86/boot/compressed, efi: Merge multiple definitions ofimage_offsetinto one")And doing the following fixed it:
drivers/firmware/efi/libstub/x86-stub.c +1
-1@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -u32 image_offset; +u32 image_offset __section(".data");static efi_loaded_image_t *image =NULL ; static efi_status_t
Quoted 4 lines
I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put
image_offsetin the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".
Thank you both. I'll refresh the set and run it here tomorrow too, to make sure it boots here too.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteBorislav Petkov
maintainerQuoted 12 lines · Tom Lendacky
drivers/firmware/efi/libstub/x86-stub.c +1
-1@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -u32 image_offset; +u32 image_offset __section(".data");static efi_loaded_image_t *image =NULL ; static efi_status_t
Quoted 4 lines
I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put
image_offsetin the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".
Hmm, two points about this:
1. Can we do
u32 image_offset __bss_decrypted;
here instead? We have this special section just for that fun and it self-documents this way.
2. Also, why does my SEV-ES guest boot just fine without that change?
[ 0.000000] Linux version 6.1.0-rc6+ (root@ml) (gcc (Debian 11.3.0-1) 11.3.0, GNU ld (GNU Binutils for Debian) 2.38) #1 SMP PREEMPT_DYNAMIC Wed Nov 23 11:27:17 CET 2022 ... [ 0.336132] Memory Encryption Features active: AMD SEV SEV-ES
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerQuoted 21 lines · Borislav Petkov
Quoted 12 lines · Tom Lendacky
drivers/firmware/efi/libstub/x86-stub.c +1
-1@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -u32 image_offset; +u32 image_offset __section(".data");static efi_loaded_image_t *image =NULL ; static efi_status_tQuoted 4 lines
I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put
image_offsetin the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".Hmm, two points about this:
Quoted 7 lines
1. Can we do
u32
image_offset__bss_decrypted;here instead? We have this special section just for that fun and it self-documents this way.
The patch moves it from .data to .bss inadvertently, and I am not convinced Tom's analysis is entirely accurate: we may simply have garbage in image_offset if we access it before .bss gets cleared.
Quoted 2 lines
2. Also, why does my SEV-ES guest boot just fine without that change?
Indeed, so it needs to be in .data
Quoted 11 lines
[ 0.000000] Linux version 6.1.0-rc6+ (root@ml) (gcc (Debian 11.3.0-1) 11.3.0, GNU ld (GNU Binutils for Debian) 2.38) #1 SMP PREEMPT_DYNAMIC Wed Nov 23 11:27:17 CET 2022 ... [ 0.336132] Memory Encryption Features active: AMD SEV SEV-ESThx.
--
Regards/Gruss,
Boris.
Borislav Petkov
maintainerQuoted 3 lines · Ard Biesheuvel
The patch moves it from .data to .bss inadvertently, and I am not convinced Tom's analysis is entirely accurate: we may simply have garbage in
image_offsetif we access it before .bss gets cleared.
That should not be too hard to find out: add an endless loop in asm in the guest right after the first image_offset access:
1:
jmp 1b
and then dump its value.
Or Tom might have an even better solution.
But looking at the code, BSS clearing happens later, at .Lrelocated and the EFI stub comes before it. AFAICT.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerQuoted 12 lines · Borislav Petkov
Quoted 3 lines · Ard Biesheuvel
The patch moves it from .data to .bss inadvertently, and I am not convinced Tom's analysis is entirely accurate: we may simply have garbage in
image_offsetif we access it before .bss gets cleared.That should not be too hard to find out: add an endless loop in asm in the guest right after the first
image_offsetaccess:1:
jmp 1b
Quoted 7 lines
and then dump its value.
Or Tom might have an even better solution.
But looking at the code, BSS clearing happens later, at .Lrelocated and the EFI stub comes before it. AFAICT.
Indeed. And moving it back into .data makes the most sense in any case
- the point of the patch is to drop the duplicate definitions from asm code, not to move it into a different section.
The reason I hadn't spotted this is because my boot chain always sets the value of image_offset during the boot, and does not rely on the statically initialized value at all.
Quoted 26 lines · Ard Biesheuvel
Quoted 25 lines · Borislav Petkov
Quoted 12 lines · Tom Lendacky
drivers/firmware/efi/libstub/x86-stub.c +1
-1@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -u32 image_offset; +u32 image_offset __section(".data");static efi_loaded_image_t *image =NULL ; static efi_status_tQuoted 4 lines
I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put
image_offsetin the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".Hmm, two points about this:
1. Can we do
u32
image_offset__bss_decrypted;
Quoted 7 lines
Quoted 3 lines
here instead? We have this special section just for that fun and it self-documents this way.
The patch moves it from .data to .bss inadvertently, and I am not convinced Tom's analysis is entirely accurate: we may simply have garbage in
image_offsetif we access it before .bss gets cleared.
When running non-encrypted, I imagine all the memory is cleared to zero as part of Qemu allocating it. As soon as you put an SEV guest on top of that, host made zeroes will not appear as zeroes to the SEV guest, rather they will be decrypted and end up looking like ciphertext (hence the random values I kept seeing in image_offset). The SEV guest must explicitly clear it, which is why having it in .bss doesn't work for SEV.
Thanks,
Tom
Quoted 18 lines
Quoted 3 lines
2. Also, why does my SEV-ES guest boot just fine without that change?
Indeed, so it needs to be in .data
Quoted 11 lines
[ 0.000000] Linux version 6.1.0-rc6+ (root@ml) (gcc (Debian 11.3.0-1) 11.3.0, GNU ld (GNU Binutils for Debian) 2.38) #1 SMP PREEMPT_DYNAMIC Wed Nov 23 11:27:17 CET 2022 ... [ 0.336132] Memory Encryption Features active: AMD SEV SEV-ESThx.
--
Regards/Gruss,
Boris.
Ard Biesheuvel
maintainerQuoted 36 lines · Tom Lendacky
Quoted 26 lines · Ard Biesheuvel
Quoted 25 lines · Borislav Petkov
Quoted 12 lines · Tom Lendacky
drivers/firmware/efi/libstub/x86-stub.c +1
-1@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -u32 image_offset; +u32 image_offset __section(".data");static efi_loaded_image_t *image =NULL ; static efi_status_tQuoted 4 lines
I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put
image_offsetin the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".Hmm, two points about this:
1. Can we do
u32
image_offset__bss_decrypted;Quoted 7 lines
Quoted 3 lines
here instead? We have this special section just for that fun and it self-documents this way.
The patch moves it from .data to .bss inadvertently, and I am not convinced Tom's analysis is entirely accurate: we may simply have garbage in
image_offsetif we access it before .bss gets cleared.
Quoted 7 lines
When running non-encrypted, I imagine all the memory is cleared to zero as part of Qemu allocating it. As soon as you put an SEV guest on top of that, host made zeroes will not appear as zeroes to the SEV guest, rather they will be decrypted and end up looking like ciphertext (hence the random values I kept seeing in
image_offset). The SEV guest must explicitly clear it, which is why having it in .bss doesn't work for SEV.
Yes, QEMU will probably clear it, but GRUB, shim, etc do a terrible job at implementing image loading correctly, i.e., not bothering to parse the PE/COFF header at all, but just copying the image into memory and invoke it at a fixed offset of 0x290 bytes into the image (the famous EFI handover protocol, see the last patch in this series if you want to know more :-))
This also means that the balance of the file size vs the image size (where BSS lives) is completely ignored, and we actually have to relocate the image from the EFI stub in such cases, because we cannot be sure that the BSS does not overlap with memory that is already in use, given the top down allocation strategy the EFI usually employs.
In summary, I guess there are multiple way how we might end up in this situation, and simply putting the variable back into .data where it came from is probably the best approach here.
Quoted 20 lines · Borislav Petkov
Quoted 12 lines · Tom Lendacky
drivers/firmware/efi/libstub/x86-stub.c +1
-1@@ -23,7 +23,7 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -u32 image_offset; +u32 image_offset __section(".data");static efi_loaded_image_t *image =NULL ; static efi_status_tQuoted 4 lines
I assume it has to do with being in .data vs .bss and not being explicitly cleared with the encryption bit set. With the change to put
image_offsetin the .data section, it is read as zero, where as when it was in the .bss section it was reading "ciphertext".Hmm, two points about this:
Quoted 6 lines
1. Can we do
u32
image_offset__bss_decrypted;here instead? We have this special section just for that fun and it self-documents this way.
Yes, but __bss_decrypted is for the main kernel, not the decompression kernel.
The original value was in the .data section of the assembler (before the patch moved it), which gets initialized when loaded. Having it in the .bss section where you hope that memory was zeroed before hand is the issue.
Quoted 6 lines
2. Also, why does my SEV-ES guest boot just fine without that change?
[ 0.000000] Linux version 6.1.0-rc6+ (root@ml) (gcc (Debian 11.3.0-1) 11.3.0, GNU ld (GNU Binutils for Debian) 2.38) #1 SMP PREEMPT_DYNAMIC Wed Nov 23 11:27:17 CET 2022 ... [ 0.336132] Memory Encryption Features active: AMD SEV SEV-ES
Are you booting directly using the -kernel option on Qemu or going through the bootloader. It was only when using Grub that the problem appeared for me.
Thanks,
Tom
Quoted 3 lines
Thx.