x86: head_64.S spring cleaning
Ard Biesheuvel
maintainercover x86: head_64.S spring cleaning
After doing some cleanup work on the EFI code in head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.
I have been sitting on these patches since November, waiting for the right time to post them, i.e., after the SEV/SNP review had finished. This has yet to happen, so I'm posting them now instead. Please feel free to disregard for the time being, and propose a suitable timeframe to repost them if this is likely to conflict with ongoing work.
Ard Biesheuvel (6):
x86/head_64: clean up mixed mode 32-bit entry code efi/x86: simplify IDT/GDT preserve/restore x86/compressed: move startup32_load_idt() out of startup code x86/compressed: move startup32_check_sev_cbit out of startup code x86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y
arch/x86/boot/compressed/Makefile | 8 +- arch/x86/boot/compressed/efi_mixed.S | 352 ++++++++++++++++++++ arch/x86/boot/compressed/efi_thunk_64.S | 195 ----------- arch/x86/boot/compressed/head_32.S | 4 - arch/x86/boot/compressed/head_64.S | 309 +---------------- arch/x86/boot/compressed/mem_encrypt.S | 146 +++++++- drivers/firmware/efi/libstub/x86-stub.c | 3 +- 7 files changed, 506 insertions(+), 511 deletions(-) create mode 100644 arch/x86/boot/compressed/efi_mixed.S delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S
Signature · 1 line
2.35.1
Ard Biesheuvel
maintainerPATCH 1/6 x86/head_64: clean up mixed mode 32-bit entry code
The x86_64 32-bit entry code is a jumble of EFI and SEV routines, which is not good for maintainability. Let's isolate the EFI mixed mode code and combine it with the boot service thunk that lives in another .S file, so that we can remove it from head_64.S
--- arch/x86/boot/compressed/Makefile | 6 +- arch/x86/boot/compressed/efi_mixed.S | 358 ++++++++++++++++++++ arch/x86/boot/compressed/efi_thunk_64.S | 195 ----------- arch/x86/boot/compressed/head_32.S | 4 - arch/x86/boot/compressed/head_64.S | 149 +------- drivers/firmware/efi/libstub/x86-stub.c | 3 +- 6 files changed, 370 insertions(+), 345 deletions(-)
arch/x86/boot/compressed/Makefile +3 -3
@@ -107,11 +107,11 @@ endif vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o $(obj)/tdcall.o -vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o vmlinux-objs-$(CONFIG_EFI) += $(obj)/efi.o -efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.a +vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_mixed.o +vmlinux-objs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a -$(obj)/vmlinux: $(vmlinux-objs-y) $(efi-obj-y) FORCE +$(obj)/vmlinux: $(vmlinux-objs-y) FORCE $(call if_changed,ld) OBJCOPYFLAGS_vmlinux.bin := -R .comment -S
arch/x86/boot/compressed/efi_mixed.S new file +358 -0
358 lines not on this page · raw message
arch/x86/boot/compressed/efi_thunk_64.S deleted +0 -195
195 lines not on this page · raw message
arch/x86/boot/compressed/head_32.S +0 -4
@@ -208,10 +208,6 @@ SYM_DATA_START_LOCAL(gdt) .quad 0x00cf92000000ffff /* __KERNEL_DS */ SYM_DATA_END_LABEL(gdt, SYM_L_LOCAL, gdt_end) -#ifdef CONFIG_EFI_STUB -SYM_DATA(image_offset, .long 0) -#endif - /* * Stack and heap for uncompression */
arch/x86/boot/compressed/head_64.S +7 -142
178 lines not on this page · raw message
drivers/firmware/efi/libstub/x86-stub.c +2 -1
@@ -23,7 +23,8 @@const efi_system_table_t *efi_system_table;const efi_dxe_services_table_t *efi_dxe_table; -extern u32 image_offset; + +u32 image_offset;static efi_loaded_image_t *image =NULL ; static efi_status_t
Signature · 1 line
2.35.1
Borislav Petkov
maintainerRe: [PATCH 1/6] x86/head_64: clean up mixed mode 32-bit entry code
Quoted 4 lines · Ard Biesheuvel
The
x86_6432-bit entry code is a jumble of EFI and SEV routines, which is not good for maintainability. Let's isolate the EFI mixed mode code and combine it with the boot service thunk that lives in another .S file, so that we can remove it fromhead_64.S
Who is "we"?
Please use passive voice in all text.
Quoted 9 lines
Signed-off-by Ard Biesheuvel <[email protected]>--- arch/x86/boot/compressed/Makefile | 6 +- arch/x86/boot/compressed/efi_mixed.S | 358 ++++++++++++++++++++ arch/x86/boot/compressed/efi_thunk_64.S | 195 ----------- arch/x86/boot/compressed/head_32.S | 4 - arch/x86/boot/compressed/head_64.S | 149 +------- drivers/firmware/efi/libstub/x86-stub.c | 3 +- 6 files changed, 370 insertions(+), 345 deletions(-)
So I'm really nervous about patches touching early asm code where multiple things happen all at once instead of each logical change being split into a single patch: here I see code movement but then other functionality is being added too.
So I'd really appreciate it if you split this one into smaller, obvious, even boring patches - this will simplify review considerably. For example, do only a mechanical code movement in one patch and then add the new startup_64_mixedmode thing in another. And so on.
That would be greatly appreciated.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerPATCH 2/6 efi/x86: simplify IDT/GDT preserve/restore
In reply to Ard BiesheuvelTweak the asm and remove some redundant instructions. While at it, fix the associated comment for style and correctness.
--- arch/x86/boot/compressed/efi_mixed.S | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-)
arch/x86/boot/compressed/efi_mixed.S +7 -13
@@ -94,24 +94,20 @@ SYM_FUNC_START(__efi64_thunk) leaq 0x20(%rsp), %rbx sgdt (%rbx) - - addq $16, %rbx - sidt (%rbx) + sidt 16(%rbx) leaq 1f(%rip), %rbp /* - * Switch to IDT and GDT with 32-bit segments. This is the firmware GDT - * and IDT that was installed when the kernel started executing. The - * pointers were saved at the efi32_stub_handover entry point below. + * Switch to IDT and GDT with 32-bit segments. These are the firmware + * GDT and IDT that were installed when the kernel started executing. + * The pointers were saved at the efi32_entry entry point below. * * Pass the saved DS selector to the 32-bit code, and use far return to * restore the saved CS selector. */ - leaq efi32_boot_idt(%rip), %rax - lidt (%rax) - leaq efi32_boot_gdt(%rip), %rax - lgdt (%rax) + lidt efi32_boot_idt(%rip) + lgdt efi32_boot_gdt(%rip) movzwl efi32_boot_ds(%rip), %edx movzwq efi32_boot_cs(%rip), %rax
@@ -185,9 +181,7 @@ SYM_FUNC_START_LOCAL(efi_enter32) */ cli - lidtl (%ebx) - subl $16, %ebx - + lidtl 16(%ebx) lgdtl (%ebx) movl %cr4, %eax
Signature · 1 line
2.35.1
Ard Biesheuvel
maintainerPATCH 3/6 x86/compressed: move startup32_load_idt() out of startup code
In reply to Ard BiesheuvelMove the function startup32_load_idt() into mem_encrypt.S, and turn it into an ordinary function, instead of relying on hidden register arguments and non-standard calling conventions.
While at it, simplify the arithmetic involved in populating a IDT entry.
--- arch/x86/boot/compressed/head_64.S | 71 +------------------ arch/x86/boot/compressed/mem_encrypt.S | 72 ++++++++++++++++++-- 2 files changed, 69 insertions(+), 74 deletions(-)
arch/x86/boot/compressed/head_64.S +2 -69
@@ -118,7 +118,9 @@ SYM_FUNC_START(startup_32) 1: /* Setup Exception handling for SEV-ES */ +#ifdef CONFIG_AMD_MEM_ENCRYPT call startup32_load_idt +#endif /* Make sure cpu supports long mode. */ call verify_cpu
@@ -701,80 +703,11 @@ SYM_DATA_START(boot_idt) .endr SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end) -#ifdef CONFIG_AMD_MEM_ENCRYPT -SYM_DATA_START(boot32_idt_desc) - .word boot32_idt_end - boot32_idt - 1 - .long 0 -SYM_DATA_END(boot32_idt_desc) - .balign 8 -SYM_DATA_START(boot32_idt) - .rept 32 - .quad 0 - .endr -SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end) -#endif - #ifdef CONFIG_AMD_MEM_ENCRYPT __HEAD .code32 -/* - * Write an IDT entry into boot32_idt - * - * Parameters: - * - * %eax: Handler address - * %edx: Vector number - * - * Physical offset is expected in %ebp - */ -SYM_FUNC_START(startup32_set_idt_entry) - push %ebx - push %ecx - - /* IDT entry address to %ebx */ - leal rva(boot32_idt)(%ebp), %ebx - shl $3, %edx - addl %edx, %ebx - - /* Build IDT entry, lower 4 bytes */ - movl %eax, %edx - andl $0x0000ffff, %edx # Target code segment offset [15:0] - movl $__KERNEL32_CS, %ecx # Target code segment selector - shl $16, %ecx - orl %ecx, %edx - - /* Store lower 4 bytes to IDT */ - movl %edx, (%ebx) - - /* Build IDT entry, upper 4 bytes */ - movl %eax, %edx - andl $0xffff0000, %edx # Target code segment offset [31:16] - orl $0x00008e00, %edx # Present, Type 32-bit Interrupt Gate - - /* Store upper 4 bytes to IDT */ - movl %edx, 4(%ebx) - - pop %ecx - pop %ebx - RET -SYM_FUNC_END(startup32_set_idt_entry) #endif -SYM_FUNC_START(startup32_load_idt) -#ifdef CONFIG_AMD_MEM_ENCRYPT - /* #VC handler */ - leal rva(startup32_vc_handler)(%ebp), %eax - movl $X86_TRAP_VC, %edx - call startup32_set_idt_entry - - /* Load IDT */ - leal rva(boot32_idt)(%ebp), %eax - movl %eax, rva(boot32_idt_desc+2)(%ebp) - lidt rva(boot32_idt_desc)(%ebp) -#endif - RET -SYM_FUNC_END(startup32_load_idt) - /* * Check for the correct C-bit position when the startup_32 boot-path is used. *
arch/x86/boot/compressed/mem_encrypt.S +67 -5
104 lines not on this page · raw message
Signature · 1 line
2.35.1
Ard Biesheuvel
maintainerPATCH 4/6 x86/compressed: move startup32_check_sev_cbit out of startup code
In reply to Ard BiesheuvelMove startup32_check_sev_cbit() out of head_64.S and turn it into an ordinary function using the ordinary calling conventions, rather than preserving and restoring the registers that are known to be live at the call site.
Also reorder the call with the EFI mixed mode check, so we are not omitting the call to startup32_check_sev_cbit() on mixed mode systems implicitly, even if the set of machines supporting both features is empty.
--- arch/x86/boot/compressed/head_64.S | 83 ++------------------ arch/x86/boot/compressed/mem_encrypt.S | 65 +++++++++++++++ 2 files changed, 70 insertions(+), 78 deletions(-)
arch/x86/boot/compressed/head_64.S +5 -78
@@ -251,6 +251,11 @@ SYM_FUNC_START(startup_32) movl $__BOOT_TSS, %eax ltr %ax +#ifdef CONFIG_AMD_MEM_ENCRYPT + /* Check if the C-bit position is correct when SEV is active */ + call startup32_check_sev_cbit +#endif + /* * Setup for the jump to 64bit mode *
@@ -266,9 +271,6 @@ SYM_FUNC_START(startup_32) leal rva(startup_64_mixedmode)(%ebp), %eax jne 1f #endif - /* Check if the C-bit position is correct when SEV is active */ - call startup32_check_sev_cbit - leal rva(startup_64)(%ebp), %eax 1: pushl $__KERNEL_CS
@@ -703,81 +705,6 @@ SYM_DATA_START(boot_idt) .endr SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end) -#ifdef CONFIG_AMD_MEM_ENCRYPT - __HEAD - .code32 -#endif - -/* - * Check for the correct C-bit position when the startup_32 boot-path is used. - * - * The check makes use of the fact that all memory is encrypted when paging is - * disabled. The function creates 64 bits of random data using the RDRAND - * instruction. RDRAND is mandatory for SEV guests, so always available. If the - * hypervisor violates that the kernel will crash right here. - * - * The 64 bits of random data are stored to a memory location and at the same - * time kept in the %eax and %ebx registers. Since encryption is always active - * when paging is off the random data will be stored encrypted in main memory. - * - * Then paging is enabled. When the C-bit position is correct all memory is - * still mapped encrypted and comparing the register values with memory will - * succeed. An incorrect C-bit position will map all memory unencrypted, so that - * the compare will use the encrypted random data and fail. - */ -SYM_FUNC_START(startup32_check_sev_cbit) -#ifdef CONFIG_AMD_MEM_ENCRYPT - pushl %eax - pushl %ebx - pushl %ecx - pushl %edx - - /* Check for non-zero sev_status */ - movl rva(sev_status)(%ebp), %eax - testl %eax, %eax - jz 4f - - /* - * Get two 32-bit random values - Don't bail out if RDRAND fails - * because it is better to prevent forward progress if no random value - * can be gathered. - */ -1: rdrand %eax - jnc 1b -2: rdrand %ebx - jnc 2b - - /* Store to memory and keep it in the registers */ - movl %eax, rva(sev_check_data)(%ebp) - movl %ebx, rva(sev_check_data+4)(%ebp) - - /* Enable paging to see if encryption is active */ - movl %cr0, %edx /* Backup %cr0 in %edx */ - movl $(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */ - movl %ecx, %cr0 - - cmpl %eax, rva(sev_check_data)(%ebp) - jne 3f - cmpl %ebx, rva(sev_check_data+4)(%ebp) - jne 3f - - movl %edx, %cr0 /* Restore previous %cr0 */ - - jmp 4f - -3: /* Check failed - hlt the machine */ - hlt - jmp 3b - -4: - popl %edx - popl %ecx - popl %ebx - popl %eax -#endif - RET -SYM_FUNC_END(startup32_check_sev_cbit) - /* * Stack and heap for uncompression */
arch/x86/boot/compressed/mem_encrypt.S +65 -0
71 lines not on this page · raw message
Signature · 1 line
2.35.1
Ard Biesheuvel
maintainerPATCH 5/6 x86/compressed: adhere to calling convention in get_sev_encryption_bit()
In reply to Ard BiesheuvelMake get_sev_encryption_bit() follow the ordinary x86 calling convention, and only call it if CONFIG_AMD_MEM_ENCRYPT is actually enabled. This clarifies the calling code, and makes it more maintainable.
--- arch/x86/boot/compressed/head_64.S | 6 ++++-- arch/x86/boot/compressed/mem_encrypt.S | 10 ---------- 2 files changed, 4 insertions(+), 12 deletions(-)
arch/x86/boot/compressed/head_64.S +4 -2
@@ -180,12 +180,12 @@ SYM_FUNC_START(startup_32) */ /* * If SEV is active then set the encryption mask in the page tables. - * This will insure that when the kernel is copied and decompressed + * This will ensure that when the kernel is copied and decompressed * it will be done so encrypted. */ +#ifdef CONFIG_AMD_MEM_ENCRYPT call get_sev_encryption_bit xorl %edx, %edx -#ifdef CONFIG_AMD_MEM_ENCRYPT testl %eax, %eax jz 1f subl $32, %eax /* Encryption bit is always above bit 31 */
@@ -199,6 +199,8 @@ SYM_FUNC_START(startup_32) */ movl $1, rva(sev_status)(%ebp) 1: +#else + xorl %edx, %edx #endif /* Initialize Page tables to 0 */
arch/x86/boot/compressed/mem_encrypt.S +0 -10
@@ -21,12 +21,7 @@ .code32 SYM_FUNC_START(get_sev_encryption_bit) - xor %eax, %eax - -#ifdef CONFIG_AMD_MEM_ENCRYPT push %ebx - push %ecx - push %edx movl $0x80000000, %eax /* CPUID to check the highest leaf */ cpuid
@@ -57,12 +52,7 @@ SYM_FUNC_START(get_sev_encryption_bit) xor %eax, %eax .Lsev_exit: - pop %edx - pop %ecx pop %ebx - -#endif /* CONFIG_AMD_MEM_ENCRYPT */ - RET SYM_FUNC_END(get_sev_encryption_bit)
Signature · 1 line
2.35.1
Ard Biesheuvel
maintainerPATCH 6/6 x86/compressed: only build mem_encrypt.S if AMD_MEM_ENCRYPT=y
In reply to Ard BiesheuvelAvoid building the mem_encrypt.o object if memory encryption support is not enabled to begin with.
--- arch/x86/boot/compressed/Makefile | 2 +- arch/x86/boot/compressed/mem_encrypt.S | 3 --- 2 files changed, 1 insertion(+), 4 deletions(-)
arch/x86/boot/compressed/Makefile +1 -1
@@ -99,7 +99,7 @@ vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/kaslr.o ifdef CONFIG_X86_64 vmlinux-objs-y += $(obj)/ident_map_64.o vmlinux-objs-y += $(obj)/idt_64.o $(obj)/idt_handlers_64.o - vmlinux-objs-y += $(obj)/mem_encrypt.o + vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/mem_encrypt.o vmlinux-objs-y += $(obj)/pgtable_64.o vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) += $(obj)/sev.o endif
arch/x86/boot/compressed/mem_encrypt.S +0 -3
@@ -56,8 +56,6 @@ SYM_FUNC_START(get_sev_encryption_bit) RET SYM_FUNC_END(get_sev_encryption_bit) -#ifdef CONFIG_AMD_MEM_ENCRYPT - /** * sev_es_req_cpuid - Request a CPUID value from the Hypervisor using * the GHCB MSR protocol
@@ -312,4 +310,3 @@ SYM_DATA_START_LOCAL(boot32_idt_desc) .word . - boot32_idt - 1 .long 0 SYM_DATA_END(boot32_idt_desc) -#endif
Signature · 1 line
2.35.1
Ard Biesheuvel
maintainerQuoted 11 lines · Ard Biesheuvel
After doing some cleanup work on the EFI code in
head_64.S, the mixed mode code in particular, I noticed that the memory encryption pieces could use some attention as well, so I cleaned that up too.I have been sitting on these patches since November, waiting for the right time to post them, i.e., after the SEV/SNP review had finished. This has yet to happen, so I'm posting them now instead. Please feel free to disregard for the time being, and propose a suitable timeframe to repost them if this is likely to conflict with ongoing work.
Ping?
Quoted 29 lines
Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc Dave Hansen <[email protected]>Cc Michael Roth <[email protected]>Ard Biesheuvel (6):
x86/head_64: clean up mixed mode 32-bit entry code efi/x86: simplify IDT/GDT preserve/restore x86/compressed: movestartup32_load_idt()out of startup code x86/compressed: movestartup32_check_sev_cbitout of startup code x86/compressed: adhere to calling convention in
get_sev_encryption_bit()
x86/compressed: only buildmem_encrypt.Sif AMD_MEM_ENCRYPT=yarch/x86/boot/compressed/Makefile | 8 +- arch/x86/boot/compressed/efi_mixed.S | 352 ++++++++++++++++++++ arch/x86/boot/compressed/efi_thunk_64.S | 195 ----------- arch/x86/boot/compressed/head_32.S | 4 - arch/x86/boot/compressed/head_64.S | 309 +---------------- arch/x86/boot/compressed/mem_encrypt.S | 146 +++++++- drivers/firmware/efi/libstub/x86-stub.c | 3 +- 7 files changed, 506 insertions(+), 511 deletions(-) create mode 100644 arch/x86/boot/compressed/efi_mixed.S delete mode 100644 arch/x86/boot/compressed/efi_thunk_64.S--
2.35.1