Skip to contentSkip to conversation

x86: head_64.S spring cleaning

PATCH Superseded Ard Biesheuvel 2 participants 9 messages Aug 15, 2022 – Sep 20, 2022

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Ard Biesheuvel

author maintainer
#0 lore raw

cover x86: head_64.S spring cleaning

After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.

I have been sitting on these patches since November, waiting for the right time to post them, i.e., after the SEV/SNP review had finished. This has yet to happen, so I'm posting them now instead. Please feel free to disregard for the time being, and propose a suitable timeframe to repost them if this is likely to conflict with ongoing work.

Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Michael Roth <[email protected]>

Ard Biesheuvel (6):
x86/head_64: clean up mixed mode 32-bit entry code efi/x86: simplify IDT/GDT preserve/restore x86/compressed: move startup32_load_idt() out of startup code x86/compressed: move startup32_check_sev_cbit out of startup code x86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y

 arch/x86/boot/compressed/Makefile       |   8 +-
 arch/x86/boot/compressed/efi_mixed.S    | 352 ++++++++++++++++++++
 arch/x86/boot/compressed/efi_thunk_64.S | 195 -----------
 arch/x86/boot/compressed/head_32.S      |   4 -
 arch/x86/boot/compressed/head_64.S      | 309 +----------------
 arch/x86/boot/compressed/mem_encrypt.S  | 146 +++++++-
 drivers/firmware/efi/libstub/x86-stub.c |   3 +-
 7 files changed, 506 insertions(+), 511 deletions(-)
 create mode 100644 arch/x86/boot/compressed/efi_mixed.S
 delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S
Signature · 1 line
2.35.1
author maintainer
#3 lore raw

PATCH 2/6 efi/x86: simplify IDT/GDT preserve/restore

Tweak the asm and remove some redundant instructions. While at it, fix the associated comment for style and correctness.

---
 arch/x86/boot/compressed/efi_mixed.S | 20 +++++++-------------
 1 file changed, 7 insertions(+), 13 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +7 -13
index 6fd7ac517c53..a0f217c8ab38 100644
@@ -94,24 +94,20 @@ SYM_FUNC_START(__efi64_thunk)
 
 	leaq	0x20(%rsp), %rbx
 	sgdt	(%rbx)
-
-	addq	$16, %rbx
-	sidt	(%rbx)
+	sidt	16(%rbx)
 
 	leaq	1f(%rip), %rbp
 
 	/*
-	 * Switch to IDT and GDT with 32-bit segments. This is the firmware GDT
-	 * and IDT that was installed when the kernel started executing. The
-	 * pointers were saved at the efi32_stub_handover entry point below.
+	 * Switch to IDT and GDT with 32-bit segments. These are the firmware
+	 * GDT and IDT that were installed when the kernel started executing.
+	 * The pointers were saved at the efi32_entry entry point below.
 	 *
 	 * Pass the saved DS selector to the 32-bit code, and use far return to
 	 * restore the saved CS selector.
 	 */
-	leaq	efi32_boot_idt(%rip), %rax
-	lidt	(%rax)
-	leaq	efi32_boot_gdt(%rip), %rax
-	lgdt	(%rax)
+	lidt	efi32_boot_idt(%rip)
+	lgdt	efi32_boot_gdt(%rip)
 
 	movzwl	efi32_boot_ds(%rip), %edx
 	movzwq	efi32_boot_cs(%rip), %rax
@@ -185,9 +181,7 @@ SYM_FUNC_START_LOCAL(efi_enter32)
 	 */
 	cli
 
-	lidtl	(%ebx)
-	subl	$16, %ebx
-
+	lidtl	16(%ebx)
 	lgdtl	(%ebx)
 
 	movl	%cr4, %eax
Signature · 1 line
2.35.1
author maintainer
#1 lore raw

PATCH 1/6 x86/head_64: clean up mixed mode 32-bit entry code

In reply to Ard Biesheuvel

The x86_64 32-bit entry code is a jumble of EFI and SEV routines, which is not good for maintainability. Let's isolate the EFI mixed mode code and combine it with the boot service thunk that lives in another .S file, so that we can remove it from head_64.S

---
 arch/x86/boot/compressed/Makefile       |   6 +-
 arch/x86/boot/compressed/efi_mixed.S    | 358 ++++++++++++++++++++
 arch/x86/boot/compressed/efi_thunk_64.S | 195 -----------
 arch/x86/boot/compressed/head_32.S      |   4 -
 arch/x86/boot/compressed/head_64.S      | 149 +-------
 drivers/firmware/efi/libstub/x86-stub.c |   3 +-
 6 files changed, 370 insertions(+), 345 deletions(-)
arch/x86/boot/compressed/Makefile +3 -3
index 35ce1a64068b..d6dbb46696a2 100644
@@ -107,11 +107,11 @@ endif
 vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o
 vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o $(obj)/tdcall.o
 
-vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o
 vmlinux-objs-$(CONFIG_EFI) += $(obj)/efi.o
-efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.a
+vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_mixed.o
+vmlinux-objs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
 
-$(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE
+$(obj)/vmlinux: $(vmlinux-objs-y) FORCE
 	$(call if_changed,ld)
 
 OBJCOPYFLAGS_vmlinux.bin :=  -R .comment -S
arch/x86/boot/compressed/efi_mixed.S new file +358 -0

358 lines not on this page · raw message

arch/x86/boot/compressed/efi_thunk_64.S deleted +0 -195

195 lines not on this page · raw message

arch/x86/boot/compressed/head_32.S +0 -4
index 3b354eb9516d..6589ddd4cfaf 100644
@@ -208,10 +208,6 @@ SYM_DATA_START_LOCAL(gdt)
 	.quad	0x00cf92000000ffff	/* __KERNEL_DS */
 SYM_DATA_END_LABEL(gdt, SYM_L_LOCAL, gdt_end)
 
-#ifdef CONFIG_EFI_STUB
-SYM_DATA(image_offset, .long 0)
-#endif
-
 /*
  * Stack and heap for uncompression
  */
arch/x86/boot/compressed/head_64.S +7 -142

178 lines not on this page · raw message

drivers/firmware/efi/libstub/x86-stub.c +2 -1
index 05ae8bcc9d67..d7b53dec01be 100644
@@ -23,7 +23,8 @@
 
 const efi_system_table_t *efi_system_table;
 const efi_dxe_services_table_t *efi_dxe_table;
-extern u32 image_offset;
+
+u32 image_offset;
 static efi_loaded_image_t *image = NULL;
 
 static efi_status_t
Signature · 1 line
2.35.1
author maintainer
#4 lore raw

PATCH 3/6 x86/compressed: move startup32_load_idt() out of startup code

In reply to Ard Biesheuvel

Move the function startup32_load_idt() into mem_encrypt.S, and turn it into an ordinary function, instead of relying on hidden register arguments and non-standard calling conventions.

While at it, simplify the arithmetic involved in populating a IDT entry.

---
 arch/x86/boot/compressed/head_64.S     | 71 +------------------
 arch/x86/boot/compressed/mem_encrypt.S | 72 ++++++++++++++++++--
 2 files changed, 69 insertions(+), 74 deletions(-)
arch/x86/boot/compressed/head_64.S +2 -69
index c2cdbd8a3375..1ca2ed52f93c 100644
@@ -118,7 +118,9 @@ SYM_FUNC_START(startup_32)
 1:
 
 	/* Setup Exception handling for SEV-ES */
+#ifdef CONFIG_AMD_MEM_ENCRYPT
 	call	startup32_load_idt
+#endif
 
 	/* Make sure cpu supports long mode. */
 	call	verify_cpu
@@ -701,80 +703,11 @@ SYM_DATA_START(boot_idt)
 	.endr
 SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end)
 
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-SYM_DATA_START(boot32_idt_desc)
-	.word   boot32_idt_end - boot32_idt - 1
-	.long   0
-SYM_DATA_END(boot32_idt_desc)
-	.balign 8
-SYM_DATA_START(boot32_idt)
-	.rept 32
-	.quad 0
-	.endr
-SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
-#endif
-
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 	__HEAD
 	.code32
-/*
- * Write an IDT entry into boot32_idt
- *
- * Parameters:
- *
- * %eax:	Handler address
- * %edx:	Vector number
- *
- * Physical offset is expected in %ebp
- */
-SYM_FUNC_START(startup32_set_idt_entry)
-	push    %ebx
-	push    %ecx
-
-	/* IDT entry address to %ebx */
-	leal    rva(boot32_idt)(%ebp), %ebx
-	shl	$3, %edx
-	addl    %edx, %ebx
-
-	/* Build IDT entry, lower 4 bytes */
-	movl    %eax, %edx
-	andl    $0x0000ffff, %edx	# Target code segment offset [15:0]
-	movl    $__KERNEL32_CS, %ecx	# Target code segment selector
-	shl     $16, %ecx
-	orl     %ecx, %edx
-
-	/* Store lower 4 bytes to IDT */
-	movl    %edx, (%ebx)
-
-	/* Build IDT entry, upper 4 bytes */
-	movl    %eax, %edx
-	andl    $0xffff0000, %edx	# Target code segment offset [31:16]
-	orl     $0x00008e00, %edx	# Present, Type 32-bit Interrupt Gate
-
-	/* Store upper 4 bytes to IDT */
-	movl    %edx, 4(%ebx)
-
-	pop     %ecx
-	pop     %ebx
-	RET
-SYM_FUNC_END(startup32_set_idt_entry)
 #endif
 
-SYM_FUNC_START(startup32_load_idt)
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-	/* #VC handler */
-	leal    rva(startup32_vc_handler)(%ebp), %eax
-	movl    $X86_TRAP_VC, %edx
-	call    startup32_set_idt_entry
-
-	/* Load IDT */
-	leal	rva(boot32_idt)(%ebp), %eax
-	movl	%eax, rva(boot32_idt_desc+2)(%ebp)
-	lidt    rva(boot32_idt_desc)(%ebp)
-#endif
-	RET
-SYM_FUNC_END(startup32_load_idt)
-
 /*
  * Check for the correct C-bit position when the startup_32 boot-path is used.
  *
arch/x86/boot/compressed/mem_encrypt.S +67 -5

104 lines not on this page · raw message

Signature · 1 line
2.35.1
author maintainer
#5 lore raw

PATCH 4/6 x86/compressed: move startup32_check_sev_cbit out of startup code

In reply to Ard Biesheuvel

Move startup32_check_sev_cbit() out of head_64.S and turn it into an ordinary function using the ordinary calling conventions, rather than preserving and restoring the registers that are known to be live at the call site.

Also reorder the call with the EFI mixed mode check, so we are not omitting the call to startup32_check_sev_cbit() on mixed mode systems implicitly, even if the set of machines supporting both features is empty.

---
 arch/x86/boot/compressed/head_64.S     | 83 ++------------------
 arch/x86/boot/compressed/mem_encrypt.S | 65 +++++++++++++++
 2 files changed, 70 insertions(+), 78 deletions(-)
arch/x86/boot/compressed/head_64.S +5 -78
index 1ca2ed52f93c..382ed3d8b26a 100644
@@ -251,6 +251,11 @@ SYM_FUNC_START(startup_32)
 	movl    $__BOOT_TSS, %eax
 	ltr	%ax
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+	/* Check if the C-bit position is correct when SEV is active */
+	call	startup32_check_sev_cbit
+#endif
+
 	/*
 	 * Setup for the jump to 64bit mode
 	 *
@@ -266,9 +271,6 @@ SYM_FUNC_START(startup_32)
 	leal	rva(startup_64_mixedmode)(%ebp), %eax
 	jne	1f
 #endif
-	/* Check if the C-bit position is correct when SEV is active */
-	call	startup32_check_sev_cbit
-
 	leal	rva(startup_64)(%ebp), %eax
 1:
 	pushl	$__KERNEL_CS
@@ -703,81 +705,6 @@ SYM_DATA_START(boot_idt)
 	.endr
 SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end)
 
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-	__HEAD
-	.code32
-#endif
-
-/*
- * Check for the correct C-bit position when the startup_32 boot-path is used.
- *
- * The check makes use of the fact that all memory is encrypted when paging is
- * disabled. The function creates 64 bits of random data using the RDRAND
- * instruction. RDRAND is mandatory for SEV guests, so always available. If the
- * hypervisor violates that the kernel will crash right here.
- *
- * The 64 bits of random data are stored to a memory location and at the same
- * time kept in the %eax and %ebx registers. Since encryption is always active
- * when paging is off the random data will be stored encrypted in main memory.
- *
- * Then paging is enabled. When the C-bit position is correct all memory is
- * still mapped encrypted and comparing the register values with memory will
- * succeed. An incorrect C-bit position will map all memory unencrypted, so that
- * the compare will use the encrypted random data and fail.
- */
-SYM_FUNC_START(startup32_check_sev_cbit)
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-	pushl	%eax
-	pushl	%ebx
-	pushl	%ecx
-	pushl	%edx
-
-	/* Check for non-zero sev_status */
-	movl	rva(sev_status)(%ebp), %eax
-	testl	%eax, %eax
-	jz	4f
-
-	/*
-	 * Get two 32-bit random values - Don't bail out if RDRAND fails
-	 * because it is better to prevent forward progress if no random value
-	 * can be gathered.
-	 */
-1:	rdrand	%eax
-	jnc	1b
-2:	rdrand	%ebx
-	jnc	2b
-
-	/* Store to memory and keep it in the registers */
-	movl	%eax, rva(sev_check_data)(%ebp)
-	movl	%ebx, rva(sev_check_data+4)(%ebp)
-
-	/* Enable paging to see if encryption is active */
-	movl	%cr0, %edx			 /* Backup %cr0 in %edx */
-	movl	$(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */
-	movl	%ecx, %cr0
-
-	cmpl	%eax, rva(sev_check_data)(%ebp)
-	jne	3f
-	cmpl	%ebx, rva(sev_check_data+4)(%ebp)
-	jne	3f
-
-	movl	%edx, %cr0	/* Restore previous %cr0 */
-
-	jmp	4f
-
-3:	/* Check failed - hlt the machine */
-	hlt
-	jmp	3b
-
-4:
-	popl	%edx
-	popl	%ecx
-	popl	%ebx
-	popl	%eax
-#endif
-	RET
-SYM_FUNC_END(startup32_check_sev_cbit)
-
 /*
  * Stack and heap for uncompression
  */
arch/x86/boot/compressed/mem_encrypt.S +65 -0

71 lines not on this page · raw message

Signature · 1 line
2.35.1
author maintainer
#7 lore raw

PATCH 6/6 x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y

In reply to Ard Biesheuvel

Avoid building the mem_encrypt.o object if memory encryption support is not enabled to begin with.

---
 arch/x86/boot/compressed/Makefile      | 2 +-
 arch/x86/boot/compressed/mem_encrypt.S | 3 ---
 2 files changed, 1 insertion(+), 4 deletions(-)
arch/x86/boot/compressed/Makefile +1 -1
index d6dbb46696a2..9aad9ddcf3b4 100644
@@ -99,7 +99,7 @@ vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/kaslr.o
 ifdef CONFIG_X86_64
 	vmlinux-objs-y += $(obj)/ident_map_64.o
 	vmlinux-objs-y += $(obj)/idt_64.o $(obj)/idt_handlers_64.o
-	vmlinux-objs-y += $(obj)/mem_encrypt.o
+	vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/mem_encrypt.o
 	vmlinux-objs-y += $(obj)/pgtable_64.o
 	vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/sev.o
 endif
arch/x86/boot/compressed/mem_encrypt.S +0 -3
index b4a116283bd9..fad0978d6799 100644
@@ -56,8 +56,6 @@ SYM_FUNC_START(get_sev_encryption_bit)
 	RET
 SYM_FUNC_END(get_sev_encryption_bit)
 
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-
 /**
  * sev_es_req_cpuid - Request a CPUID value from the Hypervisor using
  *		      the GHCB MSR protocol
@@ -312,4 +310,3 @@ SYM_DATA_START_LOCAL(boot32_idt_desc)
 	.word   . - boot32_idt - 1
 	.long   0
 SYM_DATA_END(boot32_idt_desc)
-#endif
Signature · 1 line
2.35.1
author maintainer
#6 lore raw

PATCH 5/6 x86/compressed: adhere to calling convention in get_sev_encryption_bit()

In reply to Ard Biesheuvel

Make get_sev_encryption_bit() follow the ordinary x86 calling convention, and only call it if CONFIG_AMD_MEM_ENCRYPT is actually enabled. This clarifies the calling code, and makes it more maintainable.

---
 arch/x86/boot/compressed/head_64.S     |  6 ++++--
 arch/x86/boot/compressed/mem_encrypt.S | 10 ----------
 2 files changed, 4 insertions(+), 12 deletions(-)
arch/x86/boot/compressed/head_64.S +4 -2
index 382ed3d8b26a..4539e7c6d4c3 100644
@@ -180,12 +180,12 @@ SYM_FUNC_START(startup_32)
   */
 	/*
 	 * If SEV is active then set the encryption mask in the page tables.
-	 * This will insure that when the kernel is copied and decompressed
+	 * This will ensure that when the kernel is copied and decompressed
 	 * it will be done so encrypted.
 	 */
+#ifdef CONFIG_AMD_MEM_ENCRYPT
 	call	get_sev_encryption_bit
 	xorl	%edx, %edx
-#ifdef	CONFIG_AMD_MEM_ENCRYPT
 	testl	%eax, %eax
 	jz	1f
 	subl	$32, %eax	/* Encryption bit is always above bit 31 */
@@ -199,6 +199,8 @@ SYM_FUNC_START(startup_32)
 	 */
 	movl	$1, rva(sev_status)(%ebp)
 1:
+#else
+	xorl	%edx, %edx
 #endif
 
 	/* Initialize Page tables to 0 */
arch/x86/boot/compressed/mem_encrypt.S +0 -10
index 3cd3db0da49d..b4a116283bd9 100644
@@ -21,12 +21,7 @@
 
 	.code32
 SYM_FUNC_START(get_sev_encryption_bit)
-	xor	%eax, %eax
-
-#ifdef CONFIG_AMD_MEM_ENCRYPT
 	push	%ebx
-	push	%ecx
-	push	%edx
 
 	movl	$0x80000000, %eax	/* CPUID to check the highest leaf */
 	cpuid
@@ -57,12 +52,7 @@ SYM_FUNC_START(get_sev_encryption_bit)
 	xor	%eax, %eax
 
 .Lsev_exit:
-	pop	%edx
-	pop	%ecx
 	pop	%ebx
-
-#endif	/* CONFIG_AMD_MEM_ENCRYPT */
-
 	RET
 SYM_FUNC_END(get_sev_encryption_bit)
 
Signature · 1 line
2.35.1
author maintainer
#8 lore raw
In reply to Ard Biesheuvel
Quoted 11 lines · Ard Biesheuvel

After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.

I have been sitting on these patches since November, waiting for the right time to post them, i.e., after the SEV/SNP review had finished. This has yet to happen, so I'm posting them now instead. Please feel free to disregard for the time being, and propose a suitable timeframe to repost them if this is likely to conflict with ongoing work.

Ping?

Quoted 29 lines
Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Michael Roth <[email protected]>

Ard Biesheuvel (6):
x86/head_64: clean up mixed mode 32-bit entry code efi/x86: simplify IDT/GDT preserve/restore x86/compressed: move startup32_load_idt() out of startup code x86/compressed: move startup32_check_sev_cbit out of startup code x86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y

 arch/x86/boot/compressed/Makefile       |   8 +-
 arch/x86/boot/compressed/efi_mixed.S    | 352 ++++++++++++++++++++
 arch/x86/boot/compressed/efi_thunk_64.S | 195 -----------
 arch/x86/boot/compressed/head_32.S      |   4 -
 arch/x86/boot/compressed/head_64.S      | 309 +----------------
 arch/x86/boot/compressed/mem_encrypt.S  | 146 +++++++-
 drivers/firmware/efi/libstub/x86-stub.c |   3 +-
 7 files changed, 506 insertions(+), 511 deletions(-)
 create mode 100644 arch/x86/boot/compressed/efi_mixed.S
 delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S

--
2.35.1

maintainer
#2 lore raw

Re: [PATCH 1/6] x86/head_64: clean up mixed mode 32-bit entry code

In reply to Ard Biesheuvel
Quoted 4 lines · Ard Biesheuvel

The x86_64 32-bit entry code is a jumble of EFI and SEV routines, which is not good for maintainability. Let's isolate the EFI mixed mode code and combine it with the boot service thunk that lives in another .S file, so that we can remove it from head_64.S

Who is "we"?

Please use passive voice in all text.

Quoted 9 lines
---
 arch/x86/boot/compressed/Makefile       |   6 +-
 arch/x86/boot/compressed/efi_mixed.S    | 358 ++++++++++++++++++++
 arch/x86/boot/compressed/efi_thunk_64.S | 195 -----------
 arch/x86/boot/compressed/head_32.S      |   4 -
 arch/x86/boot/compressed/head_64.S      | 149 +-------
 drivers/firmware/efi/libstub/x86-stub.c |   3 +-
 6 files changed, 370 insertions(+), 345 deletions(-)

So I'm really nervous about patches touching early asm code where multiple things happen all at once instead of each logical change being split into a single patch: here I see code movement but then other functionality is being added too.

So I'd really appreciate it if you split this one into smaller, obvious, even boring patches - this will simplify review considerably. For example, do only a mechanical code movement in one patch and then add the new startup_64_mixedmode thing in another. And so on.

That would be greatly appreciated.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette