Add TDX Guest Support
v2 cover Add TDX Guest Support
Hi All,
NOTE: This series is not ready for wide public review. It is being specifically posted so that Peter Z and other experts on the entry code can look for problems with the new exception handler (#VE). That's also why x86@ is not being spammed.
Intel's Trust Domain Extensions (TDX) protect guest VMs from malicious hosts and some physical attacks. This series adds the bare-minimum support to run a TDX guest. The host-side support will be submitted separately. Also support for advanced TD guest features like attestation or debug-mode will be submitted separately. Also, at this point it is not secure with some known holes in drivers, and also hasn’t been fully audited and fuzzed yet.
TDX has a lot of similarities to SEV. It enhances confidentiality and of guest memory and state (like registers) and includes a new exception (#VE) for the same basic reasons as SEV-ES. Like SEV-SNP (not merged yet), TDX limits the host's ability to effect changes in the guest physical address space.
In contrast to the SEV code in the kernel, TDX guest memory is integrity protected and isolated; the host is prevented from accessing guest memory (even ciphertext).
The TDX architecture also includes a new CPU mode called Secure-Arbitration Mode (SEAM). The software (TDX module) running in this mode arbitrates interactions between host and guest and implements many of the guarantees of the TDX architecture.
Some of the key differences between TD and regular VM is,
1. Multi CPU bring-up is done using the ACPI MADT wake-up table.
2. A new #VE exception handler is added. The TDX module injects #VE exception to the guest TD in cases of instructions that need to be emulated, disallowed MSR accesses, subset of CPUID leaves, etc.
3. By default memory is marked as private, and TD will selectively share it with VMM based on need.
4. Remote attestation is supported to enable a third party (either the owner of the workload or a user of the services provided by the workload) to establish that the workload is running on an Intel-TDX-enabled platform located within a TD prior to providing that workload data.
You can find TDX related documents in the following link.
https://software.intel.com/content/www/br/pt/develop/articles/intel-trust-domain-extensions.html
Changes since v1:
* Implemented tdcall() and tdvmcall() helper functions in assembly and renamed them as __tdcall() and __tdvmcall().
* Added do_general_protection() helper function to re-use protection code between #GP exception and TDX #VE exception handlers.
* Addressed syscall gap issue in #VE handler support (for details check the commit log in "x86/traps: Add #VE support for TDX guest").
* Modified patch titled "x86/tdx: Handle port I/O" to re-use common tdvmcall() helper function.
* Added error handling support to MADT CPU wakeup code.
* Introduced enum tdx_map_type to identify SHARED vs PRIVATE memory type.
* Enabled shared memory in IOAPIC driver.
* Added BINUTILS version info for TDCALL.
* Changed the TDVMCALL vendor id from 0 to "TDX.KVM".
* Replaced WARN() with pr_warn_ratelimited() in __tdvmcall() wrappers.
* Fixed commit log and code comments related review comments.
* Renamed patch titled # "x86/topology: Disable CPU hotplug support for TDX platforms" to "x86/topology: Disable CPU online/offline control for TDX guest"
* Rebased on top of v5.12 kernel.
Erik Kaneda (1):
ACPICA: ACPI 6.4: MADT: add Multiprocessor Wakeup Structure
Isaku Yamahata (1):
x86/tdx: ioapic: Add shared bit for IOAPIC base address
Kirill A. Shutemov (16):
x86/paravirt: Introduce CONFIG_PARAVIRT_XL x86/tdx: Get TD execution environment information via TDINFO x86/traps: Add #VE support for TDX guest
x86/tdx: Add HLT support for TDX guest
x86/tdx: Wire up KVM hypercalls
x86/tdx: Add MSR support for TDX guest
x86/tdx: Handle CPUID via #VE
x86/io: Allow to override inX() and outX() implementation x86/tdx: Handle port I/O
x86/tdx: Handle in-kernel MMIO
x86/mm: Move force_dma_unencrypted() to common code x86/tdx: Exclude Shared bit from __PHYSICAL_MASK x86/tdx: Make pages shared in ioremap()
x86/tdx: Add helper to do MapGPA TDVMALL
x86/tdx: Make DMA pages shared
x86/kvm: Use bounce buffers for TD guest
Kuppuswamy Sathyanarayanan (10):
x86/tdx: Introduce INTEL_TDX_GUEST config option x86/cpufeatures: Add TDX Guest CPU feature
x86/x86: Add is_tdx_guest() interface
x86/tdx: Add __tdcall() and __tdvmcall() helper functions x86/traps: Add do_general_protection() helper function x86/tdx: Handle MWAIT, MONITOR and WBINVD
ACPICA: ACPI 6.4: MADT: add Multiprocessor Wakeup Mailbox Structure ACPI/table: Print MADT Wake table information
x86/acpi, x86/boot: Add multiprocessor wake-up support x86/topology: Disable CPU online/offline control for TDX guest
Sean Christopherson (4):
x86/boot: Add a trampoline for APs booting in 64-bit mode x86/boot: Avoid #VE during compressed boot for TDX platforms x86/boot: Avoid unnecessary #VE during boot process x86/tdx: Forcefully disable legacy PIC for TDX guests
arch/x86/Kconfig | 28 +- arch/x86/boot/compressed/Makefile | 2 + arch/x86/boot/compressed/head_64.S | 10 +- arch/x86/boot/compressed/misc.h | 1 + arch/x86/boot/compressed/pgtable.h | 2 +- arch/x86/boot/compressed/tdcall.S | 9 + arch/x86/boot/compressed/tdx.c | 32 ++ arch/x86/include/asm/apic.h | 3 + arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/idtentry.h | 4 + arch/x86/include/asm/io.h | 24 +- arch/x86/include/asm/irqflags.h | 38 +- arch/x86/include/asm/kvm_para.h | 21 + arch/x86/include/asm/paravirt.h | 22 +- arch/x86/include/asm/paravirt_types.h | 3 +- arch/x86/include/asm/pgtable.h | 3 + arch/x86/include/asm/realmode.h | 1 + arch/x86/include/asm/tdx.h | 176 +++++++++ arch/x86/kernel/Makefile | 1 + arch/x86/kernel/acpi/boot.c | 79 ++++ arch/x86/kernel/apic/apic.c | 8 + arch/x86/kernel/apic/io_apic.c | 12 +- arch/x86/kernel/asm-offsets.c | 22 ++ arch/x86/kernel/head64.c | 3 + arch/x86/kernel/head_64.S | 13 +- arch/x86/kernel/idt.c | 6 + arch/x86/kernel/paravirt.c | 4 +- arch/x86/kernel/pci-swiotlb.c | 2 +- arch/x86/kernel/smpboot.c | 5 + arch/x86/kernel/tdcall.S | 361 +++++++++++++++++ arch/x86/kernel/tdx-kvm.c | 45 +++ arch/x86/kernel/tdx.c | 480 +++++++++++++++++++++++ arch/x86/kernel/topology.c | 3 +- arch/x86/kernel/traps.c | 81 ++-- arch/x86/mm/Makefile | 2 + arch/x86/mm/ioremap.c | 8 +- arch/x86/mm/mem_encrypt.c | 75 ---- arch/x86/mm/mem_encrypt_common.c | 85 ++++ arch/x86/mm/mem_encrypt_identity.c | 1 + arch/x86/mm/pat/set_memory.c | 48 ++- arch/x86/realmode/rm/header.S | 1 + arch/x86/realmode/rm/trampoline_64.S | 49 ++- arch/x86/realmode/rm/trampoline_common.S | 5 +- drivers/acpi/tables.c | 11 + include/acpi/actbl2.h | 26 +- 45 files changed, 1654 insertions(+), 162 deletions(-) create mode 100644 arch/x86/boot/compressed/tdcall.S create mode 100644 arch/x86/boot/compressed/tdx.c create mode 100644 arch/x86/include/asm/tdx.h create mode 100644 arch/x86/kernel/tdcall.S create mode 100644 arch/x86/kernel/tdx-kvm.c create mode 100644 arch/x86/kernel/tdx.c create mode 100644 arch/x86/mm/mem_encrypt_common.c
Signature · 1 line
2.25.1
RFC v2 1/32 x86/paravirt: Introduce CONFIG_PARAVIRT_XL
From: "Kirill A. Shutemov" <[email protected]>
Split off halt paravirt calls from CONFIG_PARAVIRT_XXL into a separate config option. It provides a middle ground for not-so-deep paravirtulized environments.
CONFIG_PARAVIRT_XL will be used by TDX that needs couple of paravirt calls that were hidden under CONFIG_PARAVIRT_XXL, but the rest of the config would be a bloat for TDX.
--- arch/x86/Kconfig | 4 +++ arch/x86/boot/compressed/misc.h | 1 + arch/x86/include/asm/irqflags.h | 38 +++++++++++++++------------ arch/x86/include/asm/paravirt.h | 22 +++++++++------- arch/x86/include/asm/paravirt_types.h | 3 ++- arch/x86/kernel/paravirt.c | 4 ++- arch/x86/mm/mem_encrypt_identity.c | 1 + 7 files changed, 44 insertions(+), 29 deletions(-)
arch/x86/Kconfig +4 -0
@@ -783,8 +783,12 @@ config PARAVIRT over full virtualization. However, when run without a hypervisor the kernel is theoretically slower and slightly larger. +config PARAVIRT_XL + bool + config PARAVIRT_XXL bool + select PARAVIRT_XL config PARAVIRT_DEBUG bool "paravirt-ops debugging"
arch/x86/boot/compressed/misc.h +1 -0
@@ -9,6 +9,7 @@* paravirt and debugging variants are added.) */ #undef CONFIG_PARAVIRT +#undef CONFIG_PARAVIRT_XL#undef CONFIG_PARAVIRT_XXL#undef CONFIG_PARAVIRT_SPINLOCKS#undef CONFIG_KASAN
arch/x86/include/asm/irqflags.h +21 -17
@@ -59,27 +59,11 @@ static inline __cpuidle void native_halt(void)#endif -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_XL #include <asm/paravirt.h> #else #ifndef __ASSEMBLY__#include <linux/types.h> - -static __always_inline unsigned long arch_local_save_flags (void ) -{ -return native_save_fl (); -} - -static __always_inline void arch_local_irq_disable(void) -{ -native_irq_disable (); -} - -static __always_inline void arch_local_irq_enable(void) -{ -native_irq_enable (); -} -/* * Used in the idle loop; sti takes one instruction cycle * to complete:
@@ -97,6 +81,26 @@ static inline __cpuidle void halt(void) {native_halt (); } +#endif /* !__ASSEMBLY__ */ +#endif /* CONFIG_PARAVIRT_XL */ + +#ifndef CONFIG_PARAVIRT_XXL +#ifndef __ASSEMBLY__ + +static __always_inline unsigned long arch_local_save_flags (void ) +{ +return native_save_fl (); +} + +static __always_inline void arch_local_irq_disable(void) +{ +native_irq_disable (); +} + +static __always_inline void arch_local_irq_enable(void) +{ +native_irq_enable (); +}/* * For spinlocks, etc:
arch/x86/include/asm/paravirt.h +12 -10
@@ -84,6 +84,18 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct *mm)PVOP_VCALL1 (mmu.exit_mmap, mm); } +#ifdef CONFIG_PARAVIRT_XL +static inline void arch_safe_halt (void ) +{ +PVOP_VCALL0 (irq.safe_halt); +} + +static inline void halt (void ) +{ +PVOP_VCALL0 (irq.halt); +} +#endif +#ifdef CONFIG_PARAVIRT_XXL static inline void load_sp0 (unsigned long sp0) {
@@ -145,16 +157,6 @@ static inline void __write_cr4(unsigned long x)PVOP_VCALL1 (cpu.write_cr4, x); } -static inline void arch_safe_halt (void ) -{ -PVOP_VCALL0 (irq.safe_halt); -} - -static inline void halt (void ) -{ -PVOP_VCALL0 (irq.halt); -} -static inline void wbinvd (void ) {PVOP_VCALL0 (cpu.wbinvd);
arch/x86/include/asm/paravirt_types.h +2 -1
@@ -177,7 +177,8 @@ struct pv_irq_ops {struct paravirt_callee_save save_fl;struct paravirt_callee_save irq_disable;struct paravirt_callee_save irq_enable; - +#endif +#ifdef CONFIG_PARAVIRT_XL void (*safe_halt)(void );void (*halt)(void );#endif
arch/x86/kernel/paravirt.c +3 -1
@@ -322,9 +322,11 @@ struct paravirt_patch_template pv_ops = { .irq.save_fl = __PV_IS_CALLEE_SAVE(native_save_fl), .irq.irq_disable = __PV_IS_CALLEE_SAVE(native_irq_disable), .irq.irq_enable = __PV_IS_CALLEE_SAVE(native_irq_enable), +#endif /* CONFIG_PARAVIRT_XXL */ +#ifdef CONFIG_PARAVIRT_XL .irq.safe_halt = native_safe_halt, .irq.halt = native_halt, -#endif /* CONFIG_PARAVIRT_XXL */ +#endif /* CONFIG_PARAVIRT_XL */ /* Mmu ops. */ .mmu.flush_tlb_user = native_flush_tlb_local,
arch/x86/mm/mem_encrypt_identity.c +1 -0
@@ -24,6 +24,7 @@* be extended when new paravirt and debugging variants are added.) */ #undef CONFIG_PARAVIRT +#undef CONFIG_PARAVIRT_XL#undef CONFIG_PARAVIRT_XXL#undef CONFIG_PARAVIRT_SPINLOCKS
Signature · 1 line
2.25.1
RFC v2 2/32 x86/tdx: Introduce INTEL_TDX_GUEST config option
In reply to Kuppuswamy SathyanarayananAdd INTEL_TDX_GUEST config option to selectively compile TDX guest support.
--- arch/x86/Kconfig | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
arch/x86/Kconfig +15 -0
@@ -875,6 +875,21 @@ config ACRN_GUEST IOT with small footprint and real-time features. More details can be found in https://projectacrn.org/. +config INTEL_TDX_GUEST + bool "Intel Trusted Domain eXtensions Guest Support" + depends on X86_64 && CPU_SUP_INTEL && PARAVIRT + depends on SECURITY + select PARAVIRT_XL + select X86_X2APIC + select SECURITY_LOCKDOWN_LSM + help + Provide support for running in a trusted domain on Intel processors + equipped with Trusted Domain eXtenstions. TDX is an new Intel + technology that extends VMX and Memory Encryption with a new kind of + virtual machine guest called Trust Domain (TD). A TD is designed to + run in a CPU mode that protects the confidentiality of TD memory + contents and the TD’s CPU state from other software, including VMM. + endif #HYPERVISOR_GUEST source "arch/x86/Kconfig.cpu"
Signature · 1 line
2.25.1
RFC v2 3/32 x86/cpufeatures: Add TDX Guest CPU feature
In reply to Kuppuswamy SathyanarayananAdd CPU feature detection for Trusted Domain Extensions support. TDX feature adds capabilities to keep guest register state and memory isolated from hypervisor.
For TDX guest platforms, executing CPUID(0x21, 0) will return following values in EAX, EBX, ECX and EDX.
EAX: Maximum sub-leaf number: 0 EBX/EDX/ECX: Vendor string:
EBX = "Inte" EDX = "lTDX" ECX = " "
So when above condition is true, set X86_FEATURE_TDX_GUEST feature cap bit
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/tdx.h | 20 ++++++++++++++++++++ arch/x86/kernel/Makefile | 1 + arch/x86/kernel/head64.c | 3 +++ arch/x86/kernel/tdx.c | 30 ++++++++++++++++++++++++++++++ 5 files changed, 55 insertions(+) create mode 100644 arch/x86/include/asm/tdx.h create mode 100644 arch/x86/kernel/tdx.c
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -236,6 +236,7 @@#define X86_FEATURE_EPT_AD ( 8*32+17)/* Intel Extended Page Table access-dirty bit */ #define X86_FEATURE_VMCALL ( 8*32+18)/* "" Hypervisor supports the VMCALL instruction */ #define X86_FEATURE_VMW_VMMCALL ( 8*32+19)/* "" VMware prefers VMMCALL hypercall instruction */ +#define X86_FEATURE_TDX_GUEST ( 8*32+20)/* Trusted Domain Extensions Guest */ /* Intel-defined CPU features, CPUID level 0x00000007:0 (EBX), word 9 */ #define X86_FEATURE_FSGSBASE ( 9*32+ 0)/* RDFSBASE, WRFSBASE, RDGSBASE, WRGSBASE instructions*/
arch/x86/include/asm/tdx.h new file +20 -0
@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2020 Intel Corporation */ +#ifndef _ASM_X86_TDX_H +#define _ASM_X86_TDX_H + +#define TDX_CPUID_LEAF_ID 0x21 + +#ifdef CONFIG_INTEL_TDX_GUEST + +#include <asm/cpufeature.h> + +void __init tdx_early_init (void ); + +#else // !CONFIG_INTEL_TDX_GUEST + +static inline void tdx_early_init (void ) { }; + +#endif /* CONFIG_INTEL_TDX_GUEST */ + +#endif /* _ASM_X86_TDX_H */
arch/x86/kernel/Makefile +1 -0
@@ -127,6 +127,7 @@ obj-$(CONFIG_PARAVIRT_CLOCK) += pvclock.o obj-$(CONFIG_X86_PMEM_LEGACY_DEVICE) += pmem.o obj-$(CONFIG_JAILHOUSE_GUEST) += jailhouse.o +obj-$(CONFIG_INTEL_TDX_GUEST) += tdx.o obj-$(CONFIG_EISA) += eisa.o obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
arch/x86/kernel/head64.c +3 -0
@@ -40,6 +40,7 @@#include <asm/extable.h> #include <asm/trapnr.h> #include <asm/sev-es.h> +#include <asm/tdx.h> /* * Manage page tables very early on.
@@ -491,6 +492,8 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)kasan_early_init (); +tdx_early_init (); +idt_setup_early_handler ();copy_bootdata (__va (real_mode_data));
arch/x86/kernel/tdx.c new file +30 -0
@@ -0,0 +1,30 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2020 Intel Corporation */ + +#include <asm/tdx.h> + +static inline bool cpuid_has_tdx_guest (void ) +{ +u32 eax, signature[3 ]; + +if (cpuid_eax (0 ) <TDX_CPUID_LEAF_ID ) +return false; + +cpuid_count (TDX_CPUID_LEAF_ID ,0 , &eax, &signature[0 ], + &signature[1 ], &signature[2 ]); + +if (memcmp ("IntelTDX " , signature,12 )) +return false; + +return true; +} + +void __init tdx_early_init (void ) +{ +if (!cpuid_has_tdx_guest ()) +return ; + +setup_force_cpu_cap (X86_FEATURE_TDX_GUEST ); + +pr_info ("TDX guest is initialized\n" ); +}
Signature · 1 line
2.25.1
RFC v2 4/32 x86/x86: Add is_tdx_guest() interface
In reply to Kuppuswamy SathyanarayananAdd helper function to detect TDX feature support. It will be used to protect TDX specific code.
--- arch/x86/boot/compressed/Makefile | 1 + arch/x86/boot/compressed/tdx.c | 32 +++++++++++++++++++++++++++++++ arch/x86/include/asm/tdx.h | 8 ++++++++ arch/x86/kernel/tdx.c | 6 ++++++ 4 files changed, 47 insertions(+) create mode 100644 arch/x86/boot/compressed/tdx.c
arch/x86/boot/compressed/Makefile +1 -0
@@ -96,6 +96,7 @@ ifdef CONFIG_X86_64 endif vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o +vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.a
arch/x86/boot/compressed/tdx.c new file +32 -0
@@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* tdx.c - Early boot code for TDX +*/ + +#include <asm/tdx.h> + +static int __ro_after_init tdx_guest =-1 ; + +static inline bool native_cpuid_has_tdx_guest (void ) +{ +u32 eax =TDX_CPUID_LEAF_ID , signature[3 ] = {0 }; + +if (native_cpuid_eax (0 ) <TDX_CPUID_LEAF_ID ) +return false; + +native_cpuid (&eax, &signature[0 ], &signature[1 ], &signature[2 ]); + +if (memcmp ("IntelTDX " , signature,12 )) +return false; + +return true; +} + +bool is_tdx_guest (void ) +{ +if (tdx_guest <0 ) + tdx_guest =native_cpuid_has_tdx_guest (); + +return !!tdx_guest; +} +
arch/x86/include/asm/tdx.h +8 -0
@@ -9,10 +9,18 @@#include <asm/cpufeature.h> +/* Common API to check TDX support in decompression and common kernel code. */ +bool is_tdx_guest (void ); +void __init tdx_early_init (void );#else // !CONFIG_INTEL_TDX_GUEST +static inline bool is_tdx_guest (void ) +{ +return false; +} +static inline void tdx_early_init (void ) { };#endif /* CONFIG_INTEL_TDX_GUEST */
arch/x86/kernel/tdx.c +6 -0
@@ -19,6 +19,12 @@ static inline bool cpuid_has_tdx_guest(void)return true; } +bool is_tdx_guest (void ) +{ +return static_cpu_has (X86_FEATURE_TDX_GUEST ); +} +EXPORT_SYMBOL_GPL (is_tdx_guest); +void __init tdx_early_init (void ) {if (!cpuid_has_tdx_guest ())
Signature · 1 line
2.25.1
RFC v2 5/32 x86/tdx: Add __tdcall() and __tdvmcall() helper functions
In reply to Kuppuswamy SathyanarayananGuests communicate with VMMs with hypercalls. Historically, these are implemented using instructions that are known to cause VMEXITs like vmcall, vmlaunch, etc. However, with TDX, VMEXITs no longer expose guest state to the host. This prevents the old hypercall mechanisms from working. So to communicate with VMM, TDX specification defines a new instruction called "tdcall".
In TDX based VM, since VMM is an untrusted entity, a intermediary layer (TDX module) exists between host and guest to facilitate the secure communication. And "tdcall" instruction is used by the guest to request services from TDX module. And a variant of "tdcall" instruction (with specific arguments as defined by GHCI) is used by the guest to request services from VMM via the TDX module.
Implement common helper functions to communicate with the TDX Module and VMM (using TDCALL instruction).__tdvmcall() - function can be used to request services from the VMM.
__tdcall() - function can be used to communicate with the TDX Module.
Also define two additional wrappers, tdvmcall() and tdvmcall_out_r11() to cover common use cases of __tdvmcall() function. Since each use case of __tdcall() is different, we don't need such wrappers for it.
Implement __tdcall() and __tdvmcall() helper functions in assembly. Rationale behind choosing to use assembly over inline assembly are,
1. Since the number of lines of instructions (with comments) in __tdvmcall() implementation is over 70, using inline assembly to implement it will make it hard to read.
2. Also, since many registers (R8-R15, R[A-D]X)) will be used in TDVMCAL/TDCALL operation, if all these registers are included in in-line assembly constraints, some of the older compilers may not be able to meet this requirement.
Also, just like syscalls, not all TDVMCALL/TDCALLs use cases need to use the same set of argument registers. The implementation here picks the current worst-case scenario for TDCALL (4 registers). For TDCALLs with fewer than 4 arguments, there will end up being a few superfluous (cheap) instructions. But, this approach maximizes code reuse. The same argument applies to __tdvmcall() function as well.
Current implementation of __tdvmcall() includes error handling (ud2 on failure case) in assembly function instead of doing it in C wrapper function. The reason behind this choice is, when adding support for in/out instructions (refer to patch titled "x86/tdx: Handle port I/O" in this series), we use alternative_io() to substitute in/out instruction with __tdvmcall() calls. So use of C wrappers is not trivial in this case because the input parameters will be in the wrong registers and it's tricky to include proper buffer code to make this happen.
For registers used by TDCALL instruction, please check TDX GHCI specification, sec 2.4 and 3.
--- arch/x86/include/asm/tdx.h | 26 +++++ arch/x86/kernel/Makefile | 2 +- arch/x86/kernel/asm-offsets.c | 22 ++++ arch/x86/kernel/tdcall.S | 200 ++++++++++++++++++++++++++++++++++ arch/x86/kernel/tdx.c | 36 ++++++ 5 files changed, 285 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kernel/tdcall.S
arch/x86/include/asm/tdx.h +26 -0
@@ -8,12 +8,38 @@#ifdef CONFIG_INTEL_TDX_GUEST #include <asm/cpufeature.h> +#include <linux/types.h> + +struct tdcall_output { +u64 rcx; +u64 rdx; +u64 r8; +u64 r9; +u64 r10; +u64 r11; +}; + +struct tdvmcall_output { +u64 r11; +u64 r12; +u64 r13; +u64 r14; +u64 r15; +};/* Common API to check TDX support in decompression and common kernel code. */ bool is_tdx_guest (void );void __init tdx_early_init (void ); +/* Helper function used to communicate with the TDX module */ +u64 __tdcall (u64 fn,u64 rcx,u64 rdx,u64 r8,u64 r9, +struct tdcall_output *out); + +/* Helper function used to request services from VMM */ +u64 __tdvmcall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15, +struct tdvmcall_output *out); +#else // !CONFIG_INTEL_TDX_GUEST static inline bool is_tdx_guest (void )
arch/x86/kernel/Makefile +1 -1
@@ -127,7 +127,7 @@ obj-$(CONFIG_PARAVIRT_CLOCK) += pvclock.o obj-$(CONFIG_X86_PMEM_LEGACY_DEVICE) += pmem.o obj-$(CONFIG_JAILHOUSE_GUEST) += jailhouse.o -obj-$(CONFIG_INTEL_TDX_GUEST) += tdx.o +obj-$(CONFIG_INTEL_TDX_GUEST) += tdcall.o tdx.o obj-$(CONFIG_EISA) += eisa.o obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
arch/x86/kernel/asm-offsets.c +22 -0
@@ -23,6 +23,10 @@#include <xen/interface/xen.h> #endif +#ifdef CONFIG_INTEL_TDX_GUEST +#include <asm/tdx.h> +#endif +#ifdef CONFIG_X86_32 # include "asm-offsets_32.c" #else
@@ -75,6 +79,24 @@ static void __used common(void)OFFSET (XEN_vcpu_info_arch_cr2, vcpu_info, arch.cr2);#endif +#ifdef CONFIG_INTEL_TDX_GUEST +BLANK (); +/* Offset for fields in tdcall_output */ +OFFSET (TDCALL_rcx, tdcall_output, rcx); +OFFSET (TDCALL_rdx, tdcall_output, rdx); +OFFSET (TDCALL_r8, tdcall_output, r8); +OFFSET (TDCALL_r9, tdcall_output, r9); +OFFSET (TDCALL_r10, tdcall_output, r10); +OFFSET (TDCALL_r11, tdcall_output, r11); + +/* Offset for fields in tdvmcall_output */ +OFFSET (TDVMCALL_r11, tdvmcall_output, r11); +OFFSET (TDVMCALL_r12, tdvmcall_output, r12); +OFFSET (TDVMCALL_r13, tdvmcall_output, r13); +OFFSET (TDVMCALL_r14, tdvmcall_output, r14); +OFFSET (TDVMCALL_r15, tdvmcall_output, r15); +#endif +BLANK ();OFFSET (BP_scratch, boot_params, scratch);OFFSET (BP_secure_boot, boot_params, secure_boot);
arch/x86/kernel/tdcall.S new file +200 -0
200 lines not on this page · raw message
arch/x86/kernel/tdx.c +36 -0
@@ -1,8 +1,44 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (C) 2020 Intel Corporation */ +#define pr_fmt (fmt) "TDX: " fmt +#include <asm/tdx.h> +/* +* Wrapper for use case that checks for error code and print warning message. +*/ +static inline u64 tdvmcall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +{ +u64 err; + + err =__tdvmcall (fn, r12, r13, r14, r15,NULL ); + +if (err) +pr_warn_ratelimited ("TDVMCALL fn:%llx failed with err:%llx\n" , + fn, err); + +return err; +} + +/* +* Wrapper for the semi-common case where we need single output value (R11). +*/ +static inline u64 tdvmcall_out_r11 (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +{ + +struct tdvmcall_output out = {0 }; +u64 err; + + err =__tdvmcall (fn, r12, r13, r14, r15, &out); + +if (err) +pr_warn_ratelimited ("TDVMCALL fn:%llx failed with err:%llx\n" , + fn, err); + +return out.r11; +} +static inline bool cpuid_has_tdx_guest (void ) {u32 eax, signature[3 ];
Signature · 1 line
2.25.1
RFC v2 6/32 x86/tdx: Get TD execution environment information via TDINFO
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
Per Guest-Host-Communication Interface (GHCI) for Intel Trust Domain Extensions (Intel TDX) specification, sec 2.4.2, TDCALL[TDINFO] provides basic TD execution environment information, not provided by CPUID.
Call TDINFO during early boot to be used for following system initialization.
The call provides info on which bit in pfn is used to indicate that the page is shared with the host and attributes of the TD, such as debug.
We don't save information about the number of cpus as there's no users so far.
--- arch/x86/include/asm/tdx.h | 2 ++ arch/x86/kernel/tdx.c | 23 +++++++++++++++++++++++ 2 files changed, 25 insertions(+)
arch/x86/include/asm/tdx.h +2 -0
@@ -10,6 +10,8 @@#include <asm/cpufeature.h> #include <linux/types.h> +#define TDINFO 1 +struct tdcall_output {u64 rcx;u64 rdx;
arch/x86/kernel/tdx.c +23 -0
@@ -4,6 +4,14 @@#define pr_fmt (fmt) "TDX: " fmt#include <asm/tdx.h> +#include <asm/vmx.h> + +#include <linux/cpu.h> + +static struct { +unsigned int gpa_width; +unsigned long attributes; +} td_info__ro_after_init ;/* * Wrapper for use case that checks for error code and print warning message.
@@ -61,6 +69,19 @@ bool is_tdx_guest(void) }EXPORT_SYMBOL_GPL (is_tdx_guest); +static void tdg_get_info (void ) +{ +u64 ret; +struct tdcall_output out = {0 }; + + ret =__tdcall (TDINFO ,0 ,0 ,0 ,0 , &out); + +BUG_ON (ret); + + td_info.gpa_width = out.rcx &GENMASK (5 ,0 ); + td_info.attributes = out.rdx; +} +void __init tdx_early_init (void ) {if (!cpuid_has_tdx_guest ())
@@ -68,5 +89,7 @@ void __init tdx_early_init(void)setup_force_cpu_cap (X86_FEATURE_TDX_GUEST ); +tdg_get_info (); +pr_info ("TDX guest is initialized\n" ); }
Signature · 1 line
2.25.1
RFC v2 7/32 x86/traps: Add do_general_protection() helper function
In reply to Kuppuswamy SathyanarayananTDX guest #VE exception handler treats unsupported exceptions as #GP. So to handle the #GP, move the protection fault handler code to out of exc_general_protection() and create new helper function for it.
Also since exception handler is responsible to decide when to turn on/off IRQ, move cond_local_irq_{enable/disable)() calls out of do_general_protection().
This is a preparatory patch for adding #VE exception handler support for TDX guests.
--- arch/x86/kernel/traps.c | 51 ++++++++++++++++++++++------------------- 1 file changed, 27 insertions(+), 24 deletions(-)
arch/x86/kernel/traps.c +27 -24
@@ -527,44 +527,28 @@ static enum kernel_gp_hint get_kernel_gp_address(struct pt_regs *regs,#define GPFSTR "general protection fault" -DEFINE_IDTENTRY_ERRORCODE (exc_general_protection) +static void do_general_protection (struct pt_regs *regs,long error_code) {char desc[sizeof (GPFSTR ) +50 +2 *sizeof (unsigned long ) +1 ] =GPFSTR ;enum kernel_gp_hint hint =GP_NO_HINT ; -struct task_struct *tsk; +struct task_struct *tsk = current;unsigned long gp_addr;int ret; -cond_local_irq_enable (regs); - -if (static_cpu_has (X86_FEATURE_UMIP )) { -if (user_mode (regs) &&fixup_umip_exception (regs)) - goto exit; - } - -if (v8086_mode (regs)) { -local_irq_enable (); -handle_vm86_fault ((struct kernel_vm86_regs *) regs, error_code); -local_irq_disable (); -return ; - } - - tsk = current; -if (user_mode (regs)) { tsk->thread.error_code = error_code; tsk->thread.trap_nr =X86_TRAP_GP ;if (fixup_vdso_exception (regs,X86_TRAP_GP , error_code,0 )) - goto exit; +return ;show_signal (tsk,SIGSEGV ,"" , desc, regs, error_code);force_sig (SIGSEGV ); - goto exit; +return ; }if (fixup_exception (regs,X86_TRAP_GP , error_code,0 )) - goto exit; +return ; tsk->thread.error_code = error_code; tsk->thread.trap_nr =X86_TRAP_GP ;
@@ -576,11 +560,11 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)if (!preemptible () &&kprobe_running () &&kprobe_fault_handler (regs,X86_TRAP_GP )) - goto exit; +return ; ret =notify_die (DIE_GPF , desc, regs, error_code,X86_TRAP_GP ,SIGSEGV );if (ret ==NOTIFY_STOP ) - goto exit; +return ;if (error_code)snprintf (desc,sizeof (desc),"segment-related " GPFSTR );
@@ -601,8 +585,27 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) gp_addr =0 ;die_addr (desc, regs, error_code, gp_addr); +} -exit: +DEFINE_IDTENTRY_ERRORCODE (exc_general_protection) +{ +cond_local_irq_enable (regs); + +if (static_cpu_has (X86_FEATURE_UMIP )) { +if (user_mode (regs) &&fixup_umip_exception (regs)) { +cond_local_irq_disable (regs); +return ; + } + } + +if (v8086_mode (regs)) { +local_irq_enable (); +handle_vm86_fault ((struct kernel_vm86_regs *) regs, error_code); +local_irq_disable (); +return ; + } + +do_general_protection (regs, error_code);cond_local_irq_disable (regs); }
Signature · 1 line
2.25.1
RFC v2 8/32 x86/traps: Add #VE support for TDX guest
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
The TDX module injects #VE exception to the guest TD in cases of disallowed instructions, disallowed MSR accesses and subset of CPUID leaves. The TDX module guarantees that no #VE is injected on an EPT violation on guest physical addresses that are memory. We can still get #VE on MMIO mappings. This avoids any problems with the “system call gap”.
Add basic infrastructure to handle #VE. If there is no handler for a given #VE, since it is an unexpected event (fault case), treat it as a general protection fault and handle it using do_general_protection() call.
TDCALL[TDGETVEINFO] provides information about #VE such as exit reason.
The #VE cannot be nested before TDGETVEINFO is called, if there is any reason for it to nest the TD would shut down. The TDX module guarantees that no NMIs (or #MC or similar) can happen in this window. After TDGETVEINFO the #VE handler can nest if needed, although we don’t expect it to happen normally.
--- arch/x86/include/asm/idtentry.h | 4 ++++ arch/x86/include/asm/tdx.h | 15 +++++++++++++ arch/x86/kernel/idt.c | 6 ++++++ arch/x86/kernel/tdx.c | 38 +++++++++++++++++++++++++++++++++ arch/x86/kernel/traps.c | 30 ++++++++++++++++++++++++++ 5 files changed, 93 insertions(+)
arch/x86/include/asm/idtentry.h +4 -0
@@ -619,6 +619,10 @@ DECLARE_IDTENTRY_XENCB(X86_TRAP_OTHER, exc_xen_hypervisor_callback);DECLARE_IDTENTRY_RAW (X86_TRAP_OTHER , exc_xen_unknown_trap);#endif +#ifdef CONFIG_INTEL_TDX_GUEST +DECLARE_IDTENTRY (X86_TRAP_VE , exc_virtualization_exception); +#endif +/* Device interrupts common/spurious */ DECLARE_IDTENTRY_IRQ (X86_TRAP_OTHER , common_interrupt);#ifdef CONFIG_X86_LOCAL_APIC
arch/x86/include/asm/tdx.h +15 -0
@@ -11,6 +11,7 @@#include <linux/types.h> #define TDINFO 1 +#define TDGETVEINFO 3struct tdcall_output {u64 rcx;
@@ -29,6 +30,20 @@ struct tdvmcall_output {u64 r15; }; +struct ve_info { +u64 exit_reason; +u64 exit_qual; +u64 gla; +u64 gpa; +u32 instr_len; +u32 instr_info; +}; + +unsigned long tdg_get_ve_info (struct ve_info *ve); + +int tdg_handle_virtualization_exception (struct pt_regs *regs, +struct ve_info *ve); +/* Common API to check TDX support in decompression and common kernel code. */ bool is_tdx_guest (void );
arch/x86/kernel/idt.c +6 -0
@@ -64,6 +64,9 @@ static const __initconst struct idt_data early_idts[] = {*/ INTG(X86_TRAP_PF, asm_exc_page_fault),#endif +#ifdef CONFIG_INTEL_TDX_GUEST + INTG(X86_TRAP_VE, asm_exc_virtualization_exception), +#endif };/*
@@ -87,6 +90,9 @@ static const __initconst struct idt_data def_idts[] = {INTG (X86_TRAP_MF , asm_exc_coprocessor_error),INTG (X86_TRAP_AC , asm_exc_alignment_check),INTG (X86_TRAP_XF , asm_exc_simd_coprocessor_error), +#ifdef CONFIG_INTEL_TDX_GUEST +INTG (X86_TRAP_VE , asm_exc_virtualization_exception), +#endif #ifdef CONFIG_X86_32 TSKG(X86_TRAP_DF, GDT_ENTRY_DOUBLEFAULT_TSS),
arch/x86/kernel/tdx.c +38 -0
@@ -82,6 +82,44 @@ static void tdg_get_info(void) td_info.attributes = out.rdx; } +unsigned long tdg_get_ve_info (struct ve_info *ve) +{ +u64 ret; +struct tdcall_output out = {0 }; + +/* +* The #VE cannot be nested before TDGETVEINFO is called, +* if there is any reason for it to nest the TD would shut +* down. The TDX module guarantees that no NMIs (or #MC or +* similar) can happen in this window. After TDGETVEINFO +* the #VE handler can nest if needed, although we don’t +* expect it to happen normally. +*/ + + ret =__tdcall (TDGETVEINFO ,0 ,0 ,0 ,0 , &out); + + ve->exit_reason = out.rcx; + ve->exit_qual = out.rdx; + ve->gla = out.r8; + ve->gpa = out.r9; + ve->instr_len = out.r10 &UINT_MAX ; + ve->instr_info = out.r10 >>32 ; + +return ret; +} + +int tdg_handle_virtualization_exception (struct pt_regs *regs, +struct ve_info *ve) +{ +/* +* TODO: Add handler support for various #VE exit +* reasons. It will be added by other patches in +* the series. +*/ +pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason); +return -EFAULT ; +} +void __init tdx_early_init (void ) {if (!cpuid_has_tdx_guest ())
arch/x86/kernel/traps.c +30 -0
@@ -61,6 +61,7 @@#include <asm/insn.h> #include <asm/insn-eval.h> #include <asm/vdso.h> +#include <asm/tdx.h> #ifdef CONFIG_X86_64 #include <asm/x86_init.h>
@@ -1140,6 +1141,35 @@ DEFINE_IDTENTRY(exc_device_not_available) } } +#ifdef CONFIG_INTEL_TDX_GUEST +DEFINE_IDTENTRY (exc_virtualization_exception) +{ +struct ve_info ve; +int ret; + +RCU_LOCKDEP_WARN (!rcu_is_watching (),"entry code didn't wake RCU" ); + +/* +* Consume #VE info before re-enabling interrupts. It will be +* re-enabled after executing the TDGETVEINFO TDCALL. +*/ + ret =tdg_get_ve_info (&ve); + +cond_local_irq_enable (regs); + +if (!ret) + ret =tdg_handle_virtualization_exception (regs, &ve); +/* +* If tdg_handle_virtualization_exception() could not process +* it successfully, treat it as #GP(0) and handle it. +*/ +if (ret) +do_general_protection (regs,0 ); + +cond_local_irq_disable (regs); +} +#endif +#ifdef CONFIG_X86_32 DEFINE_IDTENTRY_SW (iret_error) {
Signature · 1 line
2.25.1
RFC v2 10/32 x86/tdx: Wire up KVM hypercalls
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
KVM hypercalls have to be wrapped into vendor-specific TDVMCALLs.
--- arch/x86/include/asm/kvm_para.h | 21 +++++++++++++++ arch/x86/include/asm/tdx.h | 39 ++++++++++++++++++++++++++++ arch/x86/kernel/tdcall.S | 7 +++++ arch/x86/kernel/tdx-kvm.c | 45 +++++++++++++++++++++++++++++++++ arch/x86/kernel/tdx.c | 4 +++ 5 files changed, 116 insertions(+) create mode 100644 arch/x86/kernel/tdx-kvm.c
arch/x86/include/asm/kvm_para.h +21 -0
@@ -6,6 +6,7 @@#include <asm/alternative.h> #include <linux/interrupt.h> #include <uapi/asm/kvm_para.h> +#include <asm/tdx.h> extern void kvmclock_init (void );
@@ -34,6 +35,10 @@ static inline bool kvm_check_and_clear_guest_paused(void)static inline long kvm_hypercall0 (unsigned int nr) {long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall0 (nr); + asm volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr)
@@ -44,6 +49,10 @@ static inline long kvm_hypercall0(unsigned int nr)static inline long kvm_hypercall1 (unsigned int nr,unsigned long p1) {long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall1 (nr, p1); + asm volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1)
@@ -55,6 +64,10 @@ static inline long kvm_hypercall2(unsigned int nr, unsigned long p1, unsigned long p2) {long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall2 (nr, p1, p2); + asm volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2)
@@ -66,6 +79,10 @@ static inline long kvm_hypercall3(unsigned int nr, unsigned long p1, unsigned long p2, unsigned long p3) { long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall3 (nr, p1, p2, p3); + asm volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2), "d"(p3)
@@ -78,6 +95,10 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned long p1, unsigned long p4) {long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall4 (nr, p1, p2, p3, p4); + asm volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2), "d"(p3), "S"(p4)
arch/x86/include/asm/tdx.h +39 -0
@@ -56,6 +56,16 @@ u64 __tdcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,/* Helper function used to request services from VMM */ u64 __tdvmcall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15,struct tdvmcall_output *out); +u64 __tdvmcall_vendor_kvm (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15, +struct tdvmcall_output *out); + +long tdx_kvm_hypercall0 (unsigned int nr); +long tdx_kvm_hypercall1 (unsigned int nr,unsigned long p1); +long tdx_kvm_hypercall2 (unsigned int nr,unsigned long p1,unsigned long p2); +long tdx_kvm_hypercall3 (unsigned int nr,unsigned long p1,unsigned long p2, +unsigned long p3); +long tdx_kvm_hypercall4 (unsigned int nr,unsigned long p1,unsigned long p2, +unsigned long p3,unsigned long p4);#else // !CONFIG_INTEL_TDX_GUEST
@@ -66,6 +76,35 @@ static inline bool is_tdx_guest(void)static inline void tdx_early_init (void ) { }; +static inline long tdx_kvm_hypercall0 (unsigned int nr) +{ +return -ENODEV ; +} + +static inline long tdx_kvm_hypercall1 (unsigned int nr,unsigned long p1) +{ +return -ENODEV ; +} + +static inline long tdx_kvm_hypercall2 (unsigned int nr,unsigned long p1, +unsigned long p2) +{ +return -ENODEV ; +} + +static inline long tdx_kvm_hypercall3 (unsigned int nr,unsigned long p1, +unsigned long p2,unsigned long p3) +{ +return -ENODEV ; +} + +static inline long tdx_kvm_hypercall4 (unsigned int nr,unsigned long p1, +unsigned long p2,unsigned long p3, +unsigned long p4) +{ +return -ENODEV ; +} +#endif /* CONFIG_INTEL_TDX_GUEST */ #endif /* _ASM_X86_TDX_H */
arch/x86/kernel/tdcall.S +7 -0
@@ -11,6 +11,7 @@ * refer to TDX GHCI specification). */ #define TDVMCALL_EXPOSE_REGS_MASK 0xfc00 +#define TDVMCALL_VENDOR_KVM 0x4d564b2e584454 /* "TDX.KVM" */ /* * TDX guests use the TDCALL instruction to make
@@ -198,3 +199,9 @@ SYM_FUNC_START(__tdvmcall) call do_tdvmcall retq SYM_FUNC_END(__tdvmcall) + +SYM_FUNC_START(__tdvmcall_vendor_kvm) + movq $TDVMCALL_VENDOR_KVM, %r10 + call do_tdvmcall + retq +SYM_FUNC_END(__tdvmcall_vendor_kvm)
arch/x86/kernel/tdx-kvm.c new file +45 -0
45 lines not on this page · raw message
arch/x86/kernel/tdx.c +4 -0
@@ -8,6 +8,10 @@#include <linux/cpu.h> +#ifdef CONFIG_KVM_GUEST +#include "tdx-kvm.c" +#endif + static struct { unsigned int gpa_width; unsigned long attributes;
Signature · 1 line
2.25.1
RFC v2 9/32 x86/tdx: Add HLT support for TDX guest
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
Per Guest-Host-Communication Interface (GHCI) for Intel Trust Domain Extensions (Intel TDX) specification, sec 3.8, TDVMCALL[Instruction.HLT] provides HLT operation. Use it to implement halt() and safe_halt() paravirtualization calls.
The same TDVMCALL is used to handle #VE exception due to EXIT_REASON_HLT.
--- arch/x86/kernel/tdx.c | 44 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 37 insertions(+), 7 deletions(-)
arch/x86/kernel/tdx.c +37 -7
@@ -82,6 +82,27 @@ static void tdg_get_info(void) td_info.attributes = out.rdx; } +static __cpuidle void tdg_halt(void) +{ +u64 ret; + + ret =__tdvmcall (EXIT_REASON_HLT ,0 ,0 ,0 ,0 ,NULL ); + +/* It should never fail */ +BUG_ON (ret); +} + +static __cpuidle void tdg_safe_halt(void) +{ +/* +* Enable interrupts next to the TDVMCALL to avoid +* performance degradation. +*/ + asmvolatile ("sti\n\t" ); + +tdg_halt (); +} +unsigned long tdg_get_ve_info (struct ve_info *ve) {u64 ret;
@@ -111,13 +132,19 @@ unsigned long tdg_get_ve_info(struct ve_info *ve)int tdg_handle_virtualization_exception (struct pt_regs *regs,struct ve_info *ve) { -/* -* TODO: Add handler support for various #VE exit -* reasons. It will be added by other patches in -* the series. -*/ -pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason); -return -EFAULT ; +switch (ve->exit_reason) { +case EXIT_REASON_HLT : +tdg_halt (); +break ; +default : +pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason); +return -EFAULT ; + } + +/* After successful #VE handling, move the IP */ + regs->ip += ve->instr_len; + +return 0 ; }void __init tdx_early_init (void )
@@ -129,5 +156,8 @@ void __init tdx_early_init(void)tdg_get_info (); + pv_ops.irq.safe_halt = tdg_safe_halt; + pv_ops.irq.halt = tdg_halt; +pr_info ("TDX guest is initialized\n" ); }
Signature · 1 line
2.25.1
RFC v2 13/32 x86/io: Allow to override inX() and outX() implementation
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
The patch allows to override the implementation of the port IO helpers. TDX code will provide an implementation that redirect the helpers to paravirt calls.
--- arch/x86/include/asm/io.h | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
arch/x86/include/asm/io.h +12 -4
@@ -271,18 +271,26 @@ static inline bool sev_key_active(void) { return false; }#endif /* CONFIG_AMD_MEM_ENCRYPT */ +#ifndef __out +#define __out (bwl, bw) \ + asm volatile("out" #bwl " %" #bw "0, %w1" : : "a"(value), "Nd"(port)) +#endif + +#ifndef __in +#define __in (bwl, bw) \ + asm volatile("in" #bwl " %w1, %" #bw "0" : "=a"(value) : "Nd"(port)) +#endif +#define BUILDIO (bwl, bw, type) \ static inline void out##bwl(unsigned type value, int port) \ { \ - asm volatile("out" #bwl " %" #bw "0, %w1" \ - : : "a"(value), "Nd"(port)); \ + __out(bwl, bw); \ } \ \ static inline unsigned type in##bwl(int port) \ { \ unsigned type value; \ - asm volatile("in" #bwl " %w1, %" #bw "0" \ - : "=a"(value) : "Nd"(port)); \ + __in(bwl, bw); \ return value; \ } \ \
Signature · 1 line
2.25.1
RFC v2 12/32 x86/tdx: Handle CPUID via #VE
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
TDX has three classes of CPUID leaves: some CPUID leaves are always handled by the CPU, others are handled by the TDX module, and some others are handled by the VMM. Since the VMM cannot directly intercept the instruction these are reflected with a #VE exception to the guest, which then converts it into a TDCALL to the VMM, or handled directly.
The TDX module EAS has a full list of CPUID leaves which are handled natively or by the TDX module in 16.2. Only unknown CPUIDs are handled by the #VE method. In practice this typically only applies to the hypervisor specific CPUIDs unknown to the native CPU.
Therefore there is no risk of causing this in early CPUID code which runs before the #VE handler is set up because it will never access those exotic CPUID leaves.
--- arch/x86/kernel/tdx.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
arch/x86/kernel/tdx.c +18 -0
@@ -174,6 +174,21 @@ static int tdg_write_msr_safe(unsigned int msr, unsigned int low,return ret ? -EIO :0 ; } +static void tdg_handle_cpuid (struct pt_regs *regs) +{ +u64 ret; +struct tdvmcall_output out = {0 }; + + ret =__tdvmcall (EXIT_REASON_CPUID , regs->ax, regs->cx,0 ,0 , &out); + +WARN_ON (ret); + + regs->ax = out.r12; + regs->bx = out.r13; + regs->cx = out.r14; + regs->dx = out.r15; +} +unsigned long tdg_get_ve_info (struct ve_info *ve) {u64 ret;
@@ -220,6 +235,9 @@ int tdg_handle_virtualization_exception(struct pt_regs *regs,case EXIT_REASON_MSR_WRITE : ret =tdg_write_msr_safe (regs->cx, regs->ax, regs->dx);break ; +case EXIT_REASON_CPUID : +tdg_handle_cpuid (regs); +break ;default :pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason);return -EFAULT ;
Signature · 1 line
2.25.1
RFC v2 11/32 x86/tdx: Add MSR support for TDX guest
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
Operations on context-switched MSRs can be run natively. The rest of MSRs should be handled through TDVMCALLs.
TDVMCALL[Instruction.RDMSR] and TDVMCALL[Instruction.WRMSR] provide MSR oprations.
You can find RDMSR and WRMSR details in Guest-Host-Communication Interface (GHCI) for Intel Trust Domain Extensions (Intel TDX) specification, sec 3.10, 3.11.
Also, since CSTAR MSR is not used on Intel CPUs as SYSCALL instruction, ignore accesses to CSTAR MSR.
--- arch/x86/kernel/tdx.c | 85 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 83 insertions(+), 2 deletions(-)
arch/x86/kernel/tdx.c +83 -2
@@ -107,6 +107,73 @@ static __cpuidle void tdg_safe_halt(void)tdg_halt (); } +static bool tdg_is_context_switched_msr (unsigned int msr) +{ +/* XXX: Update the list of context-switched MSRs */ + +switch (msr) { +case MSR_EFER : +case MSR_IA32_CR_PAT : +case MSR_FS_BASE : +case MSR_GS_BASE : +case MSR_KERNEL_GS_BASE : +case MSR_IA32_SYSENTER_CS : +case MSR_IA32_SYSENTER_EIP : +case MSR_IA32_SYSENTER_ESP : +case MSR_STAR : +case MSR_LSTAR : +case MSR_SYSCALL_MASK : +case MSR_IA32_XSS : +case MSR_TSC_AUX : +case MSR_IA32_BNDCFGS : +return true; + } +return false; +} + +static u64 tdg_read_msr_safe (unsigned int msr,int *err) +{ +u64 ret; +struct tdvmcall_output out = {0 }; + +WARN_ON_ONCE (tdg_is_context_switched_msr (msr)); + +/* +* Since CSTAR MSR is not used by Intel CPUs as SYSCALL +* instruction, just ignore it. Even raising TDVMCALL +* will lead to same result. +*/ +if (msr ==MSR_CSTAR ) +return 0 ; + + ret =__tdvmcall (EXIT_REASON_MSR_READ , msr,0 ,0 ,0 , &out); + + *err = (ret) ? -EIO :0 ; + +return out.r11; +} + +static int tdg_write_msr_safe (unsigned int msr,unsigned int low, +unsigned int high) +{ +u64 ret; + +WARN_ON_ONCE (tdg_is_context_switched_msr (msr)); + +/* +* Since CSTAR MSR is not used by Intel CPUs as SYSCALL +* instruction, just ignore it. Even raising TDVMCALL +* will lead to same result. +*/ +if (msr ==MSR_CSTAR ) +return 0 ; + + ret =__tdvmcall (EXIT_REASON_MSR_WRITE , msr, (u64 )high <<32 | low, +0 ,0 ,NULL ); + +return ret ? -EIO :0 ; +} +unsigned long tdg_get_ve_info (struct ve_info *ve) {u64 ret;
@@ -136,19 +203,33 @@ unsigned long tdg_get_ve_info(struct ve_info *ve)int tdg_handle_virtualization_exception (struct pt_regs *regs,struct ve_info *ve) { +unsigned long val; +int ret =0 ; +switch (ve->exit_reason) {case EXIT_REASON_HLT :tdg_halt ();break ; +case EXIT_REASON_MSR_READ : + val =tdg_read_msr_safe (regs->cx, (unsigned int *)&ret); +if (!ret) { + regs->ax = val &UINT_MAX ; + regs->dx = val >>32 ; + } +break ; +case EXIT_REASON_MSR_WRITE : + ret =tdg_write_msr_safe (regs->cx, regs->ax, regs->dx); +break ;default :pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason);return -EFAULT ; }/* After successful #VE handling, move the IP */ - regs->ip += ve->instr_len; +if (!ret) + regs->ip += ve->instr_len; -return 0 ; +return ret; }void __init tdx_early_init (void )
Signature · 1 line
2.25.1
RFC v2 14/32 x86/tdx: Handle port I/O
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
Unroll string operations and handle port I/O through TDVMCALLs. Also handle #VE due to I/O operations with the same TDVMCALLs.
Decompression code uses port IO for earlyprintk. We must use paravirt calls there too if we want to allow earlyprintk.
Decompresion code cannot deal with alternatives: use branches instead to implement inX() and outX() helpers.
Since we use call instruction in place of in/out instruction, the argument passed to call instruction has to be in a register, it cannot be an immediate value like in/out instruction. So change constraint flag from "Nd" to "d"
--- arch/x86/boot/compressed/Makefile | 1 + arch/x86/boot/compressed/tdcall.S | 9 ++ arch/x86/include/asm/io.h | 5 +- arch/x86/include/asm/tdx.h | 46 ++++++++- arch/x86/kernel/tdcall.S | 154 ++++++++++++++++++++++++++++++ arch/x86/kernel/tdx.c | 33 +++++++ 6 files changed, 245 insertions(+), 3 deletions(-) create mode 100644 arch/x86/boot/compressed/tdcall.S
arch/x86/boot/compressed/Makefile +1 -0
@@ -97,6 +97,7 @@ endif vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o +vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdcall.o vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.a
arch/x86/boot/compressed/tdcall.S new file +9 -0
@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include <asm/export.h> + +/* Do not export symbols in decompression code */ +#undef EXPORT_SYMBOL +#define EXPORT_SYMBOL(sym) + +#include "../../kernel/tdcall.S"
arch/x86/include/asm/io.h +3 -2
@@ -43,6 +43,7 @@#include <asm/page.h> #include <asm/early_ioremap.h> #include <asm/pgtable_types.h> +#include <asm/tdx.h> #define build_mmio_read (name, size, type, reg, barrier) \ static inline type name(const volatile void __iomem *addr) \
@@ -309,7 +310,7 @@ static inline unsigned type in##bwl##_p(int port) \ \ static inline void outs##bwl(int port, const void *addr, unsigned long count) \ { \ - if (sev_key_active()) { \ + if (sev_key_active() || is_tdx_guest()) { \ unsigned type *value = (unsigned type *)addr; \ while (count) { \ out##bwl(*value, port); \
@@ -325,7 +326,7 @@ static inline void outs##bwl(int port, const void *addr, unsigned long count) \ \ static inline void ins##bwl(int port, void *addr, unsigned long count) \ { \ - if (sev_key_active()) { \ + if (sev_key_active() || is_tdx_guest()) { \ unsigned type *value = (unsigned type *)addr; \ while (count) { \ *value = in##bwl(port); \
arch/x86/include/asm/tdx.h +45 -1
62 lines not on this page · raw message
arch/x86/kernel/tdcall.S +154 -0
169 lines not on this page · raw message
arch/x86/kernel/tdx.c +33 -0
@@ -189,6 +189,36 @@ static void tdg_handle_cpuid(struct pt_regs *regs) regs->dx = out.r15; } +static void tdg_out (int size,int port,unsigned int value) +{ +tdvmcall (EXIT_REASON_IO_INSTRUCTION , size,1 , port, value); +} + +static unsigned int tdg_in (int size,int port) +{ +return tdvmcall_out_r11 (EXIT_REASON_IO_INSTRUCTION , size,0 , port,0 ); +} + +static void tdg_handle_io (struct pt_regs *regs,u32 exit_qual) +{ +bool string = exit_qual &16 ; +int out, size, port; + +/* I/O strings ops are unrolled at build time. */ +BUG_ON (string); + + out = (exit_qual &8 ) ?0 :1 ; + size = (exit_qual &7 ) +1 ; + port = exit_qual >>16 ; + +if (out) { +tdg_out (size, port, regs->ax); + }else { + regs->ax &= ~GENMASK (8 * size,0 ); + regs->ax |=tdg_in (size, port) &GENMASK (8 * size,0 ); + } +} +unsigned long tdg_get_ve_info (struct ve_info *ve) {u64 ret;
@@ -238,6 +268,9 @@ int tdg_handle_virtualization_exception(struct pt_regs *regs,case EXIT_REASON_CPUID :tdg_handle_cpuid (regs);break ; +case EXIT_REASON_IO_INSTRUCTION : +tdg_handle_io (regs, ve->exit_qual); +break ;default :pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason);return -EFAULT ;
Signature · 1 line
2.25.1
RFC v2 15/32 x86/tdx: Handle in-kernel MMIO
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
Handle #VE due to MMIO operations. MMIO triggers #VE with EPT_VIOLATION exit reason.
For now we only handle subset of instruction that kernel uses for MMIO oerations. User-space access triggers SIGBUS.
--- arch/x86/kernel/tdx.c | 100 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+)
arch/x86/kernel/tdx.c +100 -0
@@ -5,6 +5,8 @@#include <asm/tdx.h> #include <asm/vmx.h> +#include <asm/insn.h> +#include <linux/sched/signal.h> /* force_sig_fault() */ #include <linux/cpu.h>
@@ -219,6 +221,101 @@ static void tdg_handle_io(struct pt_regs *regs, u32 exit_qual) } } +static unsigned long tdg_mmio (int size,bool write,unsigned long addr, +unsigned long val) +{ +return tdvmcall_out_r11 (EXIT_REASON_EPT_VIOLATION , size, + write, addr, val); +} + +static inline void *get_reg_ptr (struct pt_regs *regs,struct insn *insn) +{ +static const int regoff[] = { + offsetof(struct pt_regs , ax), + offsetof(struct pt_regs , cx), + offsetof(struct pt_regs , dx), + offsetof(struct pt_regs , bx), + offsetof(struct pt_regs , sp), + offsetof(struct pt_regs , bp), + offsetof(struct pt_regs , si), + offsetof(struct pt_regs , di), + offsetof(struct pt_regs , r8), + offsetof(struct pt_regs , r9), + offsetof(struct pt_regs , r10), + offsetof(struct pt_regs , r11), + offsetof(struct pt_regs , r12), + offsetof(struct pt_regs , r13), + offsetof(struct pt_regs , r14), + offsetof(struct pt_regs , r15), + }; +int regno; + + regno =X86_MODRM_REG (insn->modrm.value); +if (X86_REX_R (insn->rex_prefix.value)) + regno +=8 ; + +return (void *)regs + regoff[regno]; +} + +static int tdg_handle_mmio (struct pt_regs *regs,struct ve_info *ve) +{ +int size; +bool write; +unsigned long *reg; +struct insn insn; +unsigned long val =0 ; + +/* +* User mode would mean the kernel exposed a device directly +* to ring3, which shouldn't happen except for things like +* DPDK. +*/ +if (user_mode (regs)) { +pr_err ("Unexpected user-mode MMIO access.\n" ); +force_sig_fault (SIGBUS ,BUS_ADRERR , (void __user *) ve->gla); +return 0 ; + } + +kernel_insn_init (&insn, (void *) regs->ip,MAX_INSN_SIZE ); +insn_get_length (&insn); +insn_get_opcode (&insn); + + write = ve->exit_qual &0x2 ; + + size = insn.opnd_bytes; +switch (insn.opcode.bytes[0 ]) { +/* MOV r/m8 r8 */ +case 0x88 : +/* MOV r8 r/m8 */ +case 0x8A : +/* MOV r/m8 imm8 */ +case 0xC6 : + size =1 ; +break ; + } + +if (inat_has_immediate (insn.attr)) { +BUG_ON (!write); + val = insn.immediate.value; +tdg_mmio (size, write, ve->gpa, val); +return insn.length; + } + +BUG_ON (!inat_has_modrm (insn.attr)); + + reg =get_reg_ptr (regs, &insn); + +if (write) { +memcpy (&val, reg, size); +tdg_mmio (size, write, ve->gpa, val); + }else { + val =tdg_mmio (size, write, ve->gpa, val); +memset (reg,0 , size); +memcpy (reg, &val, size); + } +return insn.length; +} +unsigned long tdg_get_ve_info (struct ve_info *ve) {u64 ret;
@@ -271,6 +368,9 @@ int tdg_handle_virtualization_exception(struct pt_regs *regs,case EXIT_REASON_IO_INSTRUCTION :tdg_handle_io (regs, ve->exit_qual);break ; +case EXIT_REASON_EPT_VIOLATION : + ve->instr_len =tdg_handle_mmio (regs, ve); +break ;default :pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason);return -EFAULT ;
Signature · 1 line
2.25.1
RFC v2 16/32 x86/tdx: Handle MWAIT, MONITOR and WBINVD
In reply to Kuppuswamy SathyanarayananWhen running as a TDX guest, there are a number of existing, privileged instructions that do not work. If the guest kernel uses these instructions, the hardware generates a #VE.
You can find the list of unsupported instructions in Intel Trust Domain Extensions (Intel® TDX) Module specification, sec 9.2.2 and in Guest-Host Communication Interface (GHCI) Specification for Intel TDX, sec 2.4.1.
To prevent TD guest from using MWAIT/MONITOR instructions, support for these instructions are already disabled by TDX module (SEAM). So CPUID flags for these instructions should be in disabled state.
After the above mentioned preventive measures, if TD guests still execute these instructions, add appropriate warning messages in #VE handler. For WBIND instruction, since it's related to memory writeback and cache flushes, it's mainly used in context of IO devices. Since TDX 1.0 does not support non-virtual I/O devices, skipping it should not cause any fatal issues. But to let users know about its usage, use WARN() to report about it.. For MWAIT/MONITOR instruction, since its unsupported use WARN() to report unsupported usage.
--- arch/x86/kernel/tdx.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
arch/x86/kernel/tdx.c +15 -0
@@ -371,6 +371,21 @@ int tdg_handle_virtualization_exception(struct pt_regs *regs,case EXIT_REASON_EPT_VIOLATION : ve->instr_len =tdg_handle_mmio (regs, ve);break ; +case EXIT_REASON_WBINVD : +/* +* WBINVD is not supported inside TDX guests. All in- +* kernel uses should have been disabled. +*/ +WARN_ONCE (1 ,"TD Guest used unsupported WBINVD instruction\n" ); +break ; +case EXIT_REASON_MONITOR_INSTRUCTION : +case EXIT_REASON_MWAIT_INSTRUCTION : +/* +* Something in the kernel used MONITOR or MWAIT despite +* X86_FEATURE_MWAIT being cleared for TDX guests. +*/ +WARN_ONCE (1 ,"TD Guest used unsupported MWAIT/MONITOR instruction\n" ); +break ;default :pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason);return -EFAULT ;
Signature · 1 line
2.25.1
RFC v2 24/32 x86/topology: Disable CPU online/offline control for TDX guest
In reply to Kuppuswamy SathyanarayananAs per Intel TDX Virtual Firmware Design Guide, sec 4.3.5 and sec 9.4, all unused CPUs are put in spinning state by TDVF until OS requests for CPU bring-up via mailbox address passed by ACPI MADT table. Since by default all unused CPUs are always in spinning state, there is no point in supporting dynamic CPU online/offline feature. So current generation of TDVF does not support CPU hotplug feature. It may be supported in next generation.
--- arch/x86/kernel/tdx.c | 14 ++++++++++++++ arch/x86/kernel/topology.c | 3 ++- 2 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/kernel/tdx.c +14 -0
@@ -316,6 +316,17 @@ static int tdg_handle_mmio(struct pt_regs *regs, struct ve_info *ve)return insn.length; } +static int tdg_cpu_offline_prepare (unsigned int cpu) +{ +/* +* Per Intel TDX Virtual Firmware Design Guide, +* sec 4.3.5 and sec 9.4, Hotplug is not supported +* in TDX platforms. So don't support CPU +* offline feature once its turned on. +*/ +return -EOPNOTSUPP ; +} +unsigned long tdg_get_ve_info (struct ve_info *ve) {u64 ret;
@@ -410,5 +421,8 @@ void __init tdx_early_init(void) pv_ops.irq.safe_halt = tdg_safe_halt; pv_ops.irq.halt = tdg_halt; +cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"tdg:cpu_hotplug" , +NULL , tdg_cpu_offline_prepare); +pr_info ("TDX guest is initialized\n" ); }
arch/x86/kernel/topology.c +2 -1
@@ -34,6 +34,7 @@#include <linux/irq.h> #include <asm/io_apic.h> #include <asm/cpu.h> +#include <asm/tdx.h> static DEFINE_PER_CPU(struct x86_cpu, cpu_devices);
@@ -130,7 +131,7 @@ int arch_register_cpu(int num) } } } -if (num || cpu0_hotpluggable) +if ((num || cpu0_hotpluggable) && !is_tdx_guest ())per_cpu (cpu_devices, num).cpu.hotpluggable =1 ;return register_cpu (&per_cpu (cpu_devices, num).cpu, num);
Signature · 1 line
2.25.1
RFC v2 17/32 ACPICA: ACPI 6.4: MADT: add Multiprocessor Wakeup Structure
In reply to Kuppuswamy SathyanarayananFrom: Erik Kaneda <[email protected]>
ACPICA commit b9eb6f3a19b816824d6f47a6bc86fd8ce690e04b
--- include/acpi/actbl2.h | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
include/acpi/actbl2.h +11 -1
@@ -516,7 +516,8 @@ enum acpi_madt_type {ACPI_MADT_TYPE_GENERIC_MSI_FRAME =13 ,ACPI_MADT_TYPE_GENERIC_REDISTRIBUTOR =14 ,ACPI_MADT_TYPE_GENERIC_TRANSLATOR =15 , - ACPI_MADT_TYPE_RESERVED = 16 /* 16 and greater are reserved */ +ACPI_MADT_TYPE_MULTIPROC_WAKEUP =16 , + ACPI_MADT_TYPE_RESERVED = 17 /* 17 and greater are reserved */ };/*
@@ -723,6 +724,15 @@ struct acpi_madt_generic_translator {u32 reserved2; }; +/* 16: Multiprocessor wakeup (ACPI 6.4) */ + +struct acpi_madt_multiproc_wakeup { +struct acpi_subtable_header header; +u16 mailbox_version; +u32 reserved;/* reserved - must be zero */ +u64 base_address; +}; +/* * Common flags fields for MADT subtables */
Signature · 1 line
2.25.1
RFC v2 18/32 ACPICA: ACPI 6.4: MADT: add Multiprocessor Wakeup Mailbox Structure
In reply to Kuppuswamy SathyanarayananACPICA commit f1ee04207a212f6c519441e7e25397649ebc4cea
Add Multiprocessor Wakeup Mailbox Structure definition. It is useful in parsing MADT Wake table.
--- include/acpi/actbl2.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
include/acpi/actbl2.h +14 -0
@@ -733,6 +733,20 @@ struct acpi_madt_multiproc_wakeup {u64 base_address; }; +#define ACPI_MULTIPROC_WAKEUP_MB_OS_SIZE 2032 +#define ACPI_MULTIPROC_WAKEUP_MB_FIRMWARE_SIZE 2048 + +struct acpi_madt_multiproc_wakeup_mailbox { +u16 command; +u16 reserved;/* reserved - must be zero */ +u32 apic_id; +u64 wakeup_vector; +u8 reserved_os[ACPI_MULTIPROC_WAKEUP_MB_OS_SIZE ];/* reserved for OS use */ +u8 reserved_firmware[ACPI_MULTIPROC_WAKEUP_MB_FIRMWARE_SIZE ];/* reserved for firmware use */ +}; + +#define ACPI_MP_WAKE_COMMAND_WAKEUP 1 +/* * Common flags fields for MADT subtables */
Signature · 1 line
2.25.1
RFC v2 19/32 ACPI/table: Print MADT Wake table information
In reply to Kuppuswamy SathyanarayananWhen MADT is parsed, print MADT Wake table information as debug message. It will be useful to debug CPU boot issues related to MADT wake table.
--- drivers/acpi/tables.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
drivers/acpi/tables.c +11 -0
@@ -207,6 +207,17 @@ void acpi_table_print_madt_entry(struct acpi_subtable_header *header) }break ; +case ACPI_MADT_TYPE_MULTIPROC_WAKEUP : + { +struct acpi_madt_multiproc_wakeup *p; + + p = (struct acpi_madt_multiproc_wakeup *) header; + +pr_debug ("MP Wake (Mailbox version[%d] base_address[%llx])\n" , + p->mailbox_version, p->base_address); + } +break ; +default :pr_warn ("Found unsupported MADT entry (type = 0x%x)\n" , header->type);
Signature · 1 line
2.25.1
RFC v2 26/32 x86/mm: Move force_dma_unencrypted() to common code
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
Intel TDX doesn't allow VMM to access guest memory. Any memory that is required for communication with VMM must be shared explicitly by setting the bit in page table entry. And, after setting the shared bit, the conversion must be completed with MapGPA TDVMALL. The call informs VMM about the conversion and makes it remove the GPA from the S-EPT mapping. The shared memory is similar to unencrypted memory in AMD SME/SEV terminology but the underlying process of sharing/un-sharing the memory is different for Intel TDX guest platform.
SEV assumes that I/O devices can only do DMA to "decrypted" physical addresses without the C-bit set. In order for the CPU to interact with this memory, the CPU needs a decrypted mapping. To add this support, AMD SME code forces force_dma_unencrypted() to return true for platforms that support AMD SEV feature. It will be used for DMA memory allocation API to trigger set_memory_decrypted() for platforms that support AMD SEV feature.
TDX is similar. TDX architecturally prevents access to private guest memory by anything other than the guest itself. This means that any DMA buffers must be shared.
So move force_dma_unencrypted() out of AMD specific code.
It will be modified to return true for Intel TDX guest platform, similar to AMD SEV feature.
Introduce new config option X86_MEM_ENCRYPT_COMMON that has to be selected by all x86 memory encryption features. This will be selected by both AMD SEV and Intel TDX guest config options.
This is preparation for TDX changes in DMA code and it has not functional change.
--- arch/x86/Kconfig | 8 +++++-- arch/x86/mm/Makefile | 2 ++ arch/x86/mm/mem_encrypt.c | 30 ------------------------- arch/x86/mm/mem_encrypt_common.c | 38 ++++++++++++++++++++++++++++++++ 4 files changed, 46 insertions(+), 32 deletions(-) create mode 100644 arch/x86/mm/mem_encrypt_common.c
arch/x86/Kconfig +6 -2
@@ -1529,14 +1529,18 @@ config X86_CPA_STATISTICS helps to determine the effectiveness of preserving large and huge page mappings when mapping protections are changed. +config X86_MEM_ENCRYPT_COMMON + select ARCH_HAS_FORCE_DMA_UNENCRYPTED + select DYNAMIC_PHYSICAL_MASK + def_bool n + config AMD_MEM_ENCRYPT bool "AMD Secure Memory Encryption (SME) support" depends on X86_64 && CPU_SUP_AMD select DMA_COHERENT_POOL - select DYNAMIC_PHYSICAL_MASK select ARCH_USE_MEMREMAP_PROT - select ARCH_HAS_FORCE_DMA_UNENCRYPTED select INSTRUCTION_DECODER + select X86_MEM_ENCRYPT_COMMON help Say yes to enable support for the encryption of system memory. This requires an AMD processor that supports Secure Memory
arch/x86/mm/Makefile +2 -0
@@ -52,6 +52,8 @@ obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o obj-$(CONFIG_PAGE_TABLE_ISOLATION) += pti.o +obj-$(CONFIG_X86_MEM_ENCRYPT_COMMON) += mem_encrypt_common.o + obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_identity.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_boot.o
arch/x86/mm/mem_encrypt.c +0 -30
@@ -15,10 +15,6 @@#include <linux/dma-direct.h> #include <linux/swiotlb.h> #include <linux/mem_encrypt.h> -#include <linux/device.h> -#include <linux/kernel.h> -#include <linux/bitops.h> -#include <linux/dma-mapping.h> #include <asm/tlbflush.h> #include <asm/fixmap.h>
@@ -390,32 +386,6 @@ bool noinstr sev_es_active(void)return sev_status &MSR_AMD64_SEV_ES_ENABLED ; } -/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ -bool force_dma_unencrypted (struct device *dev) -{ -/* -* For SEV, all DMA must be to unencrypted addresses. -*/ -if (sev_active ()) -return true; - -/* -* For SME, all DMA must be to unencrypted addresses if the -* device does not support DMA to addresses that include the -* encryption mask. -*/ -if (sme_active ()) { -u64 dma_enc_mask =DMA_BIT_MASK (__ffs64 (sme_me_mask)); -u64 dma_dev_mask =min_not_zero (dev->coherent_dma_mask, - dev->bus_dma_limit); - -if (dma_dev_mask <= dma_enc_mask) -return true; - } - -return false; -} -void __init mem_encrypt_free_decrypted_mem (void ) {unsigned long vaddr, vaddr_end, npages;
arch/x86/mm/mem_encrypt_common.c new file +38 -0
@@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* AMD Memory Encryption Support +* +* Copyright (C) 2016 Advanced Micro Devices, Inc. +* +* Author: Tom Lendacky <[email protected]> +*/ + +#include <linux/mm.h> +#include <linux/mem_encrypt.h> +#include <linux/dma-mapping.h> + +/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ +bool force_dma_unencrypted (struct device *dev) +{ +/* +* For SEV, all DMA must be to unencrypted/shared addresses. +*/ +if (sev_active ()) +return true; + +/* +* For SME, all DMA must be to unencrypted addresses if the +* device does not support DMA to addresses that include the +* encryption mask. +*/ +if (sme_active ()) { +u64 dma_enc_mask =DMA_BIT_MASK (__ffs64 (sme_me_mask)); +u64 dma_dev_mask =min_not_zero (dev->coherent_dma_mask, + dev->bus_dma_limit); + +if (dma_dev_mask <= dma_enc_mask) +return true; + } + +return false; +}
Signature · 1 line
2.25.1
RFC v2 25/32 x86/tdx: Forcefully disable legacy PIC for TDX guests
In reply to Kuppuswamy SathyanarayananFrom: Sean Christopherson <[email protected]>
Disable the legacy PIC (8259) for TDX guests as the PIC cannot be supported by the VMM. TDX Module does not allow direct IRQ injection, and using posted interrupt style delivery requires the guest to EOI the IRQ, which diverges from the legacy PIC behavior.
--- arch/x86/kernel/tdx.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kernel/tdx.c +3 -0
@@ -4,6 +4,7 @@#define pr_fmt (fmt) "TDX: " fmt#include <asm/tdx.h> +#include <asm/i8259.h> #include <asm/vmx.h> #include <asm/insn.h> #include <linux/sched/signal.h> /* force_sig_fault() */
@@ -421,6 +422,8 @@ void __init tdx_early_init(void) pv_ops.irq.safe_halt = tdg_safe_halt; pv_ops.irq.halt = tdg_halt; + legacy_pic = &null_legacy_pic; +cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"tdg:cpu_hotplug" ,NULL , tdg_cpu_offline_prepare);
Signature · 1 line
2.25.1
RFC v2 20/32 x86/acpi, x86/boot: Add multiprocessor wake-up support
In reply to Kuppuswamy SathyanarayananAs per ACPI specification r6.4, sec 5.2.12.19, a new sub structure – multiprocessor wake-up structure - is added to the ACPI Multiple APIC Description Table (MADT) to describe the information of the mailbox. If a platform firmware produces the multiprocessor wake-up structure, then OS may use this new mailbox-based mechanism to wake up the APs.
Add ACPI MADT wake table parsing support for x86 platform and if MADT wake table is present, update apic->wakeup_secondary_cpu with new API which uses MADT wake mailbox to wake-up CPU.
--- arch/x86/include/asm/apic.h | 3 ++ arch/x86/kernel/acpi/boot.c | 79 +++++++++++++++++++++++++++++++++++++ arch/x86/kernel/apic/apic.c | 8 ++++ 3 files changed, 90 insertions(+)
arch/x86/include/asm/apic.h +3 -0
@@ -487,6 +487,9 @@ static inline unsigned int read_apic_id(void)return apic->get_apic_id (reg); } +typedef int (*wakeup_cpu_handler )(int apicid,unsigned long start_eip); +extern void acpi_wake_cpu_handler_update (wakeup_cpu_handler handler); +extern int default_apic_id_valid (u32 apicid);extern int default_acpi_madt_oem_check (char *,char *);extern void default_setup_apic_routing (void );
arch/x86/kernel/acpi/boot.c +79 -0
103 lines not on this page · raw message
arch/x86/kernel/apic/apic.c +8 -0
@@ -2554,6 +2554,14 @@ u32 x86_msi_msg_get_destid(struct msi_msg *msg, bool extid) }EXPORT_SYMBOL_GPL (x86_msi_msg_get_destid); +void __init acpi_wake_cpu_handler_update (wakeup_cpu_handler handler) +{ +struct apic **drv; + +for (drv = __apicdrivers; drv < __apicdrivers_end; drv++) + (*drv)->wakeup_secondary_cpu = handler; +} +/* * Override the generic EOI implementation with an optimized version. * Only called during early boot when only one CPU is active and with
Signature · 1 line
2.25.1
RFC v2 27/32 x86/tdx: Exclude Shared bit from __PHYSICAL_MASK
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
tdx_shared_mask() returns the mask that has to be set in a page table entry to make page shared with VMM.
Also, note that we cannot club shared mapping configuration between AMD SME and Intel TDX Guest platforms in common function. SME has to do it very early in __startup_64() as it sets the bit on all memory, except what is used for communication. TDX can postpone as we don't need any shared mapping in very early boot.
--- arch/x86/Kconfig | 1 + arch/x86/include/asm/tdx.h | 6 ++++++ arch/x86/kernel/tdx.c | 9 +++++++++ 3 files changed, 16 insertions(+)
arch/x86/Kconfig +1 -0
@@ -882,6 +882,7 @@ config INTEL_TDX_GUEST select PARAVIRT_XL select X86_X2APIC select SECURITY_LOCKDOWN_LSM + select X86_MEM_ENCRYPT_COMMON help Provide support for running in a trusted domain on Intel processors equipped with Trusted Domain eXtenstions. TDX is an new Intel
arch/x86/include/asm/tdx.h +6 -0
@@ -111,6 +111,8 @@ unsigned char tdg_inb(unsigned short port);unsigned short tdg_inw (unsigned short port);unsigned int tdg_inl (unsigned short port); +extern phys_addr_t tdg_shared_mask (void ); +#else // !CONFIG_INTEL_TDX_GUEST static inline bool is_tdx_guest (void )
@@ -149,6 +151,10 @@ static inline long tdx_kvm_hypercall4(unsigned int nr, unsigned long p1,return -ENODEV ; } +static inline phys_addr_t tdg_shared_mask (void ) +{ +return 0 ; +}#endif /* CONFIG_INTEL_TDX_GUEST */ #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_TDX_H */
arch/x86/kernel/tdx.c +9 -0
@@ -76,6 +76,12 @@ bool is_tdx_guest(void) }EXPORT_SYMBOL_GPL (is_tdx_guest); +/* The highest bit of a guest physical address is the "sharing" bit */ +phys_addr_t tdg_shared_mask (void ) +{ +return 1ULL << (td_info.gpa_width -1 ); +} +static void tdg_get_info (void ) {u64 ret;
@@ -87,6 +93,9 @@ static void tdg_get_info(void) td_info.gpa_width = out.rcx &GENMASK (5 ,0 ); td_info.attributes = out.rdx; + +/* Exclude Shared bit from the __PHYSICAL_MASK */ + physical_mask &= ~tdg_shared_mask (); } static __cpuidle void tdg_halt(void)
Signature · 1 line
2.25.1
RFC v2 28/32 x86/tdx: Make pages shared in ioremap()
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
All ioremap()ed pages that are not backed by normal memory (NONE or RESERVED) have to be mapped as shared.
Reuse the infrastructure we have for AMD SEV.
Note that DMA code doesn't use ioremap() to convert memory to shared as DMA buffers backed by normal memory. DMA code make buffer shared with set_memory_decrypted().
--- arch/x86/include/asm/pgtable.h | 3 +++ arch/x86/mm/ioremap.c | 8 +++++--- 2 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/include/asm/pgtable.h +3 -0
@@ -21,6 +21,9 @@#define pgprot_encrypted (prot) __pgprot(__sme_set(pgprot_val(prot)))#define pgprot_decrypted (prot) __pgprot(__sme_clr(pgprot_val(prot))) +/* Make the page accesable by VMM */ +#define pgprot_tdg_shared (prot) __pgprot(pgprot_val(prot) | tdg_shared_mask()) +#ifndef __ASSEMBLY__#include <asm/x86_init.h> #include <asm/fpu/xstate.h>
arch/x86/mm/ioremap.c +5 -3
@@ -87,12 +87,12 @@ static unsigned int __ioremap_check_ram(struct resource *res) }/* -* In a SEV guest, NONE and RESERVED should not be mapped encrypted because -* there the whole memory is already encrypted. +* In a SEV or TDX guest, NONE and RESERVED should not be mapped encrypted (or +* private in TDX case) because there the whole memory is already encrypted. */ static unsigned int __ioremap_check_encrypted (struct resource *res) { -if (!sev_active ()) +if (!sev_active () && !is_tdx_guest ())return 0 ;switch (res->desc) {
@@ -244,6 +244,8 @@ __ioremap_caller(resource_size_t phys_addr, unsigned long size, prot =PAGE_KERNEL_IO ;if ((io_desc.flags &IORES_MAP_ENCRYPTED ) || encrypted) prot =pgprot_encrypted (prot); +else if (is_tdx_guest ()) + prot =pgprot_tdg_shared (prot);switch (pcm) {case _PAGE_CACHE_MODE_UC:
Signature · 1 line
2.25.1
RFC v2 21/32 x86/boot: Add a trampoline for APs booting in 64-bit mode
In reply to Kuppuswamy SathyanarayananFrom: Sean Christopherson <[email protected]>
Add a trampoline for booting APs in 64-bit mode via a software handoff with BIOS, and use the new trampoline for the ACPI MP wake protocol used by TDX.
Extend the real mode IDT pointer by four bytes to support LIDT in 64-bit mode. For the GDT pointer, create a new entry as the existing storage for the pointer occupies the zero entry in the GDT itself.
--- arch/x86/include/asm/realmode.h | 1 + arch/x86/kernel/smpboot.c | 5 +++ arch/x86/realmode/rm/header.S | 1 + arch/x86/realmode/rm/trampoline_64.S | 49 +++++++++++++++++++++++- arch/x86/realmode/rm/trampoline_common.S | 5 ++- 5 files changed, 58 insertions(+), 3 deletions(-)
arch/x86/include/asm/realmode.h +1 -0
@@ -25,6 +25,7 @@ struct real_mode_header {u32 sev_es_trampoline_start;#endif #ifdef CONFIG_X86_64 +u32 trampoline_start64;u32 trampoline_pgd;#endif /* ACPI S3 wakeup */
arch/x86/kernel/smpboot.c +5 -0
@@ -1036,6 +1036,11 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,unsigned long boot_error =0 ;unsigned long timeout; +#ifdef CONFIG_X86_64 +if (is_tdx_guest ()) + start_ip = real_mode_header->trampoline_start64; +#endif + idle->thread.sp = (unsigned long )task_pt_regs (idle); early_gdt_descr.address = (unsigned long )get_cpu_gdt_rw (cpu); initial_code = (unsigned long )start_secondary;
arch/x86/realmode/rm/header.S +1 -0
@@ -24,6 +24,7 @@ SYM_DATA_START(real_mode_header) .long pa_sev_es_trampoline_start #endif #ifdef CONFIG_X86_64 + .long pa_trampoline_start64 .long pa_trampoline_pgd; #endif /* ACPI S3 wakeup */
arch/x86/realmode/rm/trampoline_64.S +47 -2
78 lines not on this page · raw message
arch/x86/realmode/rm/trampoline_common.S +4 -1
@@ -1,4 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ .section ".rodata","a" .balign 16 -SYM_DATA_LOCAL(tr_idt, .fill 1, 6, 0) +SYM_DATA_START_LOCAL(tr_idt) + .short 0 + .quad 0 +SYM_DATA_END(tr_idt)
Signature · 1 line
2.25.1
RFC v2 22/32 x86/boot: Avoid #VE during compressed boot for TDX platforms
In reply to Kuppuswamy SathyanarayananFrom: Sean Christopherson <[email protected]>
Avoid operations which will inject #VE during compressed boot, which is obviously fatal for TDX platforms.
Details are,
1. TDX module injects #VE if a TDX guest attempts to write EFER. So skip the WRMSR to set EFER.LME=1 if it's already set. TDX also forces EFER.LME=1, i.e. the branch will always be taken and thus the #VE avoided.
2. TDX module also injects a #VE if the guest attempts to clear CR0.NE. Ensure CR0.NE is set when loading CR0 during compressed boot. The Setting CR0.NE should be a nop on all CPUs that support 64-bit mode.
--- arch/x86/boot/compressed/head_64.S | 5 +++-- arch/x86/boot/compressed/pgtable.h | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-)
arch/x86/boot/compressed/head_64.S +3 -2
@@ -616,8 +616,9 @@ SYM_CODE_START(trampoline_32bit_src) movl $MSR_EFER, %ecx rdmsr btsl $_EFER_LME, %eax + jc 1f wrmsr - popl %edx +1: popl %edx popl %ecx /* Enable PAE and LA57 (if required) paging modes */
@@ -636,7 +637,7 @@ SYM_CODE_START(trampoline_32bit_src) pushl %eax /* Enable paging again */ - movl $(X86_CR0_PG | X86_CR0_PE), %eax + movl $(X86_CR0_PG | X86_CR0_NE | X86_CR0_PE), %eax movl %eax, %cr0 lret
arch/x86/boot/compressed/pgtable.h +1 -1
@@ -6,7 +6,7 @@#define TRAMPOLINE_32BIT_PGTABLE_OFFSET 0#define TRAMPOLINE_32BIT_CODE_OFFSET PAGE_SIZE -#define TRAMPOLINE_32BIT_CODE_SIZE 0x70 +#define TRAMPOLINE_32BIT_CODE_SIZE 0x80#define TRAMPOLINE_32BIT_STACK_END TRAMPOLINE_32BIT_SIZE
Signature · 1 line
2.25.1
RFC v2 29/32 x86/tdx: Add helper to do MapGPA TDVMALL
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
MapGPA TDVMCALL requests the host VMM to map a GPA range as private or shared memory mappings. Shared GPA mappings can be used for communication beteen TD guest and host VMM, for example for paravirtualized IO.
The new helper tdx_map_gpa() provides access to the operation.
--- arch/x86/include/asm/tdx.h | 13 +++++++++++++ arch/x86/kernel/tdx.c | 13 +++++++++++++ 2 files changed, 26 insertions(+)
arch/x86/include/asm/tdx.h +13 -0
@@ -7,6 +7,11 @@#ifndef __ASSEMBLY__ +enum tdx_map_type { +TDX_MAP_PRIVATE , +TDX_MAP_SHARED , +}; +#ifdef CONFIG_INTEL_TDX_GUEST #include <asm/cpufeature.h>
@@ -112,6 +117,8 @@ unsigned short tdg_inw(unsigned short port);unsigned int tdg_inl (unsigned short port);extern phys_addr_t tdg_shared_mask (void ); +extern int tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type);#else // !CONFIG_INTEL_TDX_GUEST
@@ -155,6 +162,12 @@ static inline phys_addr_t tdg_shared_mask(void) {return 0 ; } + +static inline int tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type) +{ +return -ENODEV ; +}#endif /* CONFIG_INTEL_TDX_GUEST */ #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_TDX_H */
arch/x86/kernel/tdx.c +13 -0
@@ -15,6 +15,8 @@#include "tdx-kvm.c" #endif +#define TDVMCALL_MAP_GPA 0x10001 + static struct { unsigned int gpa_width; unsigned long attributes;
@@ -98,6 +100,17 @@ static void tdg_get_info(void) physical_mask &= ~tdg_shared_mask (); } +int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +{ +u64 ret; + +if (map_type ==TDX_MAP_SHARED ) + gpa |=tdg_shared_mask (); + + ret =tdvmcall (TDVMCALL_MAP_GPA , gpa,PAGE_SIZE * numpages,0 ,0 ); +return ret ? -EIO :0 ; +} + static __cpuidle void tdg_halt(void) {u64 ret;
Signature · 1 line
2.25.1
RFC v2 23/32 x86/boot: Avoid unnecessary #VE during boot process
In reply to Kuppuswamy SathyanarayananFrom: Sean Christopherson <[email protected]>
Skip writing EFER during secondary_startup_64() if the current value is also the desired value. This avoids a #VE when running as a TDX guest, as the TDX-Module does not allow writes to EFER (even when writing the current, fixed value).
Also, preserve CR4.MCE instead of clearing it during boot to avoid a #VE when running as a TDX guest. The TDX-Module (effectively part of the hypervisor) requires CR4.MCE to be set at all times and injects a #VE if the guest attempts to clear CR4.MCE.
--- arch/x86/boot/compressed/head_64.S | 5 ++++- arch/x86/kernel/head_64.S | 13 +++++++++++-- 2 files changed, 15 insertions(+), 3 deletions(-)
arch/x86/boot/compressed/head_64.S +4 -1
@@ -622,7 +622,10 @@ SYM_CODE_START(trampoline_32bit_src) popl %ecx /* Enable PAE and LA57 (if required) paging modes */ - movl $X86_CR4_PAE, %eax + movl %cr4, %eax + /* Clearing CR4.MCE will #VE on TDX guests. Leave it alone. */ + andl $X86_CR4_MCE, %eax + orl $X86_CR4_PAE, %eax testl %edx, %edx jz 1f orl $X86_CR4_LA57, %eax
arch/x86/kernel/head_64.S +11 -2
@@ -141,7 +141,10 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) 1: /* Enable PAE mode, PGE and LA57 */ - movl $(X86_CR4_PAE | X86_CR4_PGE), %ecx + movq %cr4, %rcx + /* Clearing CR4.MCE will #VE on TDX guests. Leave it alone. */ + andl $X86_CR4_MCE, %ecx + orl $(X86_CR4_PAE | X86_CR4_PGE), %ecx #ifdef CONFIG_X86_5LEVEL testl $1, __pgtable_l5_enabled(%rip) jz 1f
@@ -229,13 +232,19 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) /* Setup EFER (Extended Feature Enable Register) */ movl $MSR_EFER, %ecx rdmsr + movl %eax, %edx btsl $_EFER_SCE, %eax /* Enable System Call */ btl $20,%edi /* No Execute supported? */ jnc 1f btsl $_EFER_NX, %eax btsq $_PAGE_BIT_NX,early_pmd_flags(%rip) -1: wrmsr /* Make changes effective */ + /* Skip the WRMSR if the current value matches the desired value. */ +1: cmpl %edx, %eax + je 1f + xor %edx, %edx + wrmsr /* Make changes effective */ +1: /* Setup cr0 */ movl $CR0_STATE, %eax /* Make changes effective */
Signature · 1 line
2.25.1
RFC v2 30/32 x86/tdx: Make DMA pages shared
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
Make force_dma_unencrypted() return true for TDX to get DMA pages mapped as shared.
__set_memory_enc_dec() is now aware about TDX and sets Shared bit accordingly following with relevant TDVMCALL.
Also, Do TDACCEPTPAGE on every 4k page after mapping the GPA range when converting memory to private. If the VMM uses a common pool for private and shared memory, it will likely do TDAUGPAGE in response to MAP_GPA (or on the first access to the private GPA), in which case TDX-Module will hold the page in a non-present "pending" state until it is explicitly accepted.
BUG() if TDACCEPTPAGE fails (except the above case), as the guest is completely hosed if it can't access memory.
--- arch/x86/include/asm/tdx.h | 3 ++ arch/x86/kernel/tdx.c | 26 ++++++++++++++++- arch/x86/mm/mem_encrypt_common.c | 4 +-- arch/x86/mm/pat/set_memory.c | 48 ++++++++++++++++++++++++++------ 4 files changed, 70 insertions(+), 11 deletions(-)
arch/x86/include/asm/tdx.h +3 -0
@@ -19,6 +19,9 @@ enum tdx_map_type {#define TDINFO 1#define TDGETVEINFO 3 +#define TDACCEPTPAGE 6 + +#define TDX_PAGE_ALREADY_ACCEPTED 0x8000000000000001struct tdcall_output {u64 rcx;
arch/x86/kernel/tdx.c +25 -1
@@ -100,7 +100,8 @@ static void tdg_get_info(void) physical_mask &= ~tdg_shared_mask (); } -int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +static int __tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type) {u64 ret;
@@ -111,6 +112,29 @@ int tdg_map_gpa(phys_addr_t gpa, int numpages, enum tdx_map_type map_type)return ret ? -EIO :0 ; } +static void tdg_accept_page (phys_addr_t gpa) +{ +u64 ret; + + ret =__tdcall (TDACCEPTPAGE , gpa,0 ,0 ,0 ,NULL ); + +BUG_ON (ret && ret !=TDX_PAGE_ALREADY_ACCEPTED ); +} + +int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +{ +int ret, i; + + ret =__tdg_map_gpa (gpa, numpages, map_type); +if (ret || map_type ==TDX_MAP_SHARED ) +return ret; + +for (i =0 ; i < numpages; i++) +tdg_accept_page (gpa + i*PAGE_SIZE ); + +return 0 ; +} + static __cpuidle void tdg_halt(void) {u64 ret;
arch/x86/mm/mem_encrypt_common.c +2 -2
@@ -15,9 +15,9 @@bool force_dma_unencrypted (struct device *dev) {/* -* For SEV, all DMA must be to unencrypted/shared addresses. +* For SEV and TDX, all DMA must be to unencrypted/shared addresses. */ -if (sev_active ()) +if (sev_active () ||is_tdx_guest ())return true;/*
arch/x86/mm/pat/set_memory.c +40 -8
93 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC v2 31/32 x86/kvm: Use bounce buffers for TD guest
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
TDX doesn't allow to perform DMA access to guest private memory. In order for DMA to work properly in TD guest, user SWIOTLB bounce buffers.
Move AMD SEV initialization into common code and adopt for TDX.
--- arch/x86/include/asm/io.h | 3 +- arch/x86/kernel/pci-swiotlb.c | 2 +- arch/x86/kernel/tdx.c | 3 ++ arch/x86/mm/mem_encrypt.c | 45 ------------------------------ arch/x86/mm/mem_encrypt_common.c | 47 ++++++++++++++++++++++++++++++++ 5 files changed, 53 insertions(+), 47 deletions(-)
arch/x86/include/asm/io.h +2 -1
@@ -257,10 +257,11 @@ static inline void slow_down_io(void)#endif +extern struct static_key_false sev_enable_key; +#ifdef CONFIG_AMD_MEM_ENCRYPT #include <linux/jump_label.h> -extern struct static_key_false sev_enable_key;static inline bool sev_key_active (void ) {return static_branch_unlikely (&sev_enable_key);
arch/x86/kernel/pci-swiotlb.c +1 -1
@@ -49,7 +49,7 @@ int __init pci_swiotlb_detect_4gb(void)* buffers are allocated and used for devices that do not support * the addressing range required for the encryption mask. */ -if (sme_active ()) +if (sme_active () ||is_tdx_guest ()) swiotlb =1 ;return swiotlb;
arch/x86/kernel/tdx.c +3 -0
@@ -8,6 +8,7 @@#include <asm/vmx.h> #include <asm/insn.h> #include <linux/sched/signal.h> /* force_sig_fault() */ +#include <linux/swiotlb.h> #include <linux/cpu.h>
@@ -470,6 +471,8 @@ void __init tdx_early_init(void) legacy_pic = &null_legacy_pic; + swiotlb_force =SWIOTLB_FORCE ; +cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"tdg:cpu_hotplug" ,NULL , tdg_cpu_offline_prepare);
arch/x86/mm/mem_encrypt.c +0 -45
48 lines not on this page · raw message
arch/x86/mm/mem_encrypt_common.c +47 -0
56 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC v2 32/32 x86/tdx: ioapic: Add shared bit for IOAPIC base address
In reply to Kuppuswamy SathyanarayananFrom: Isaku Yamahata <[email protected]>
IOAPIC is emulated by KVM which means its MMIO address is shared by host. Add shared bit for base address of IOAPIC. Most MMIO region is handled by ioremap which is already marked as shared for TDX guest platform, but IOAPIC is an exception which uses fixed map.
--- arch/x86/kernel/apic/io_apic.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
arch/x86/kernel/apic/io_apic.c +10 -2
@@ -2675,6 +2675,14 @@ static struct resource * __init ioapic_setup_resources(void)return res; } +static void io_apic_set_fixmap_nocache (enum fixed_addresses idx,phys_addr_t phys) +{ +pgprot_t flags =FIXMAP_PAGE_NOCACHE ; +if (is_tdx_guest ()) + flags =pgprot_tdg_shared (flags); +__set_fixmap (idx, phys, flags); +} +void __init io_apic_init_mappings (void ) {unsigned long ioapic_phys, idx =FIX_IO_APIC_BASE_0 ;
@@ -2707,7 +2715,7 @@ void __init io_apic_init_mappings(void) __func__, PAGE_SIZE, PAGE_SIZE); ioapic_phys =__pa (ioapic_phys); } -set_fixmap_nocache (idx, ioapic_phys); +io_apic_set_fixmap_nocache (idx, ioapic_phys);apic_printk (APIC_VERBOSE ,"mapped IOAPIC to %08lx (%08lx)\n" ,__fix_to_virt (idx) + (ioapic_phys & ~PAGE_MASK ), ioapic_phys);
@@ -2836,7 +2844,7 @@ int mp_register_ioapic(int id, u32 address, u32 gsi_base, ioapics[idx].mp_config.flags =MPC_APIC_USABLE ; ioapics[idx].mp_config.apicaddr = address; -set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address); +io_apic_set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address);if (bad_ioapic_register (idx)) {clear_fixmap (FIX_IO_APIC_BASE_0 + idx);return -ENODEV ;
Signature · 1 line
2.25.1
Re: [RFC v2 05/32] x86/tdx: Add __tdcall() and __tdvmcall() helper functions
In reply to Kuppuswamy Sathyanarayananarch/x86/kernel/tdcall.S line 13
Quoted 5 lines
arch/x86/kernel/tdcall.S new file +5
-0+/* + * Expose registers R10-R15 to VMM (for bitfield info + * refer to TDX GHCI specification). + */ +#define TDVMCALL_EXPOSE_REGS_MASK 0xfc00
Why can't we do:
#define TDC_R10 BIT(18)
#define TDC_R11 BIT(19)
and:
#define TDVMCALL_EXPOSE_REGS_MASK (TDX_R10 | TDX_R11 | TDX_R12 ...
or at least:
#define TDVMCALL_EXPOSE_REGS_MASK BIT(18) | BIT(19) ...
?
arch/x86/kernel/tdcall.S line 23
Quoted 9 lines
arch/x86/kernel/tdcall.S new file +9
-0+/* + * TDX guests use the TDCALL instruction to make + * hypercalls to the VMM. It is supported in + * Binutils >= 2.36. + */ +#define tdcall .byte 0x66,0x0f,0x01,0xcc + +/* + * __tdcall() - Used to communicate with the TDX module
Why is this function here? What does it do? Why do we need it?
I'd like this to actually talk about doing impedance matching between the function call and TDCALL ABIs.
Quoted 9 lines
+ * @arg1 (RDI) - TDCALL Leaf ID
+ * @arg2 (RSI) - Input parameter 1 passed to TDX module
+ * via register RCX
+ * @arg2 (RDX) - Input parameter 2 passed to TDX module
+ * via register RDX
+ * @arg3 (RCX) - Input parameter 3 passed to TDX module
+ * via register R8
+ * @arg4 (R8) - Input parameter 4 passed to TDX module
+ * via register R9
The unnecessary repitition and verbosity actually make this harder to read. This looks like it was easy to write, but not much effort is being made to make it easy to consume. Could you please apply some consideration to making it more readable?
Quoted 3 lines
+ * @arg5 (R9) - struct
tdcall_outputpointer
+ *
+ * @out - Return status of tdcall via RAX.
Don't comments usually just say "returns ... foo"? Also, the @params usually refer to *REAL* variable names. Where the heck does "out" come from? Why are you even putting argX? Shouldn't these be @'s be their literal function argument names?
@rdi - Input parameter, moved to RCX
arch/x86/kernel/tdcall.S line 48
Quoted 11 lines
+ * NOTE: This function should only used for non TDVMCALL
+ * use cases
+ */arch/x86/kernel/tdcall.S new file +8
-0+SYM_FUNC_START(__tdcall) + FRAME_BEGIN + + /* Save non-volatile GPRs that are exposed to the VMM. */ + push %r15 + push %r14 + push %r13 + push %r12
Why do we have to save these? Because they might be clobbered? If so, let's say *THAT* instead of just "exposed". "Exposed" could mean "VMM can read".
Also, this just told me that this function can't be used to talk to the VMM. Why is this talking about exposure to the VMM?
arch/x86/kernel/tdcall.S line 53
Quoted 4 lines
arch/x86/kernel/tdcall.S new file +4
-0+ /* Move TDCALL Leaf ID to RAX */ + mov %rdi, %rax + /* Move output pointer to R12 */ + mov %r9, %r12
I thought 'struct tdcall_output' was a purely software construct. Why are we passing a pointer to it into TDCALL?
arch/x86/kernel/tdcall.S line 60
Quoted 7 lines
arch/x86/kernel/tdcall.S new file +7
-0+ /* Move input param 4 to R9 */ + mov %r8, %r9 + /* Move input param 3 to R8 */ + mov %rcx, %r8 + /* Leave input param 2 in RDX */ + /* Move input param 1 to RCX */ + mov %rsi, %rcx
With a little work, this can be made a *LOT* more readable:
/* Mangle function call ABI into TDCALL ABI: */ mov %rdi, %rax /* Move TDCALL Leaf ID to RAX */ mov %r9, %r12 /* Move output pointer to R12 */ mov %r8, %r9 /* Move input 4 to R9 */ mov %rcx, %r8 /* Move input 3 to R8 */ mov %rsi, %rcx /* Move input 1 to RCX */ /* Leave input param 2 in RDX */
arch/x86/kernel/tdcall.S line 70
Quoted 9 lines
arch/x86/kernel/tdcall.S new file +9
-0+ tdcall + + /* Check for TDCALL success: 0 - Successful, otherwise failed */ + test %rax, %rax + jnz 1f + + /* Check for a TDCALL output struct */ + test %r12, %r12 + jz 1f
Does some universal status come back in r12? Aren't we dealing with a VMM/SEAM-controlled register here? Isn't this dangerous?
Quoted 5 lines
arch/x86/kernel/tdcall.S new file +5
-0+ /* Copy TDCALL result registers to output struct: */ + movq %rcx, TDCALL_rcx(%r12) + movq %rdx, TDCALL_rdx(%r12) + movq %r8, TDCALL_r8(%r12) + movq %r9, TDCALL_r9(%r12)
arch/x86/kernel/tdcall.S line 86
Quoted 10 lines
arch/x86/kernel/tdcall.S new file +10
-0+ movq %r10, TDCALL_r10(%r12) + movq %r11, TDCALL_r11(%r12) +1: + /* Zero out registers exposed to the TDX Module. */ + xor %rcx, %rcx + xor %rdx, %rdx + xor %r8d, %r8d + xor %r9d, %r9d + xor %r10d, %r10d + xor %r11d, %r11d
... why?
Quoted 17 lines
arch/x86/kernel/tdcall.S new file +17
-0+ /* Restore non-volatile GPRs that are exposed to the VMM. */ + pop %r12 + pop %r13 + pop %r14 + pop %r15 + + FRAME_END + ret +SYM_FUNC_END(__tdcall) + +/* + * do_tdvmcall() - Used to communicate with the VMM. + * + * @arg1 (RDI) - TDVMCALL function, e.g. exit reason + * @arg2 (RSI) - Input parameter 1 passed to VMM + * via register R12 + * @arg3 (RDX) - Input parameter 2 passed to VMM
arch/x86/kernel/tdcall.S line 114
Quoted 10 lines
arch/x86/kernel/tdcall.S new file +10
-0+ * via register R13 + * @arg4 (RCX) - Input parameter 3 passed to VMM + * via register R14 + * @arg5 (R8) - Input parameter 4 passed to VMM + * via register R15 + * @arg6 (R9) - struct tdvmcall_output pointer + * + * @out - Return status of tdvmcall(R10) via RAX. + * + */
Same comments on the sparse comment style.
arch/x86/kernel/tdcall.S line 124
Quoted 10 lines
arch/x86/kernel/tdcall.S new file +10
-0+SYM_CODE_START_LOCAL(do_tdvmcall) + FRAME_BEGIN + + /* Save non-volatile GPRs that are exposed to the VMM. */ + push %r15 + push %r14 + push %r13 + push %r12 + + /* Set TDCALL leaf ID to TDVMCALL (0) in RAX */
I think there needs to be some discussion of what TDCALL and TDVMCALL are. They are named too similarly not to do so.
Quoted 5 lines
arch/x86/kernel/tdcall.S new file +5
-0+ xor %eax, %eax + /* Move TDVMCALL function id (1st argument) to R11 */ + mov %rdi, %r11> + /* Move Input parameter 1-4 to R12-R15 */ + mov %rsi, %r12 + mov %rdx, %r13
arch/x86/kernel/tdcall.S line 140
Quoted 10 lines
arch/x86/kernel/tdcall.S new file +10
-0+ mov %rcx, %r14 + mov %r8, %r15 + /* Leave tdvmcall output pointer in R9 */ + + /* + * Value of RCX is used by the TDX Module to determine which + * registers are exposed to VMM. Each bit in RCX represents a + * register id. You can find the bitmap details from TDX GHCI + * spec. + */
This doesn't belong here. Put it along with theTDVMCALL_EXPOSE_REGS_MASK, please.
arch/x86/kernel/tdcall.S line 151
Quoted 11 lines
arch/x86/kernel/tdcall.S new file +11
-0+ movl $TDVMCALL_EXPOSE_REGS_MASK, %ecx + + tdcall + + /* + * Check for TDCALL success: 0 - Successful, otherwise failed. + * If failed, there is an issue with TDX Module which is fatal + * for the guest. So panic. + */ + test %rax, %rax + jnz 2f
So, just to be clear: %RAX is under the control of the SEAM module. The VMM has no control over it. Right?
Shouldn't we say that explicitly?
arch/x86/kernel/tdcall.S line 162
Quoted 10 lines
arch/x86/kernel/tdcall.S new file +10
-0+ /* Move TDVMCALL success/failure to RAX to return to user */ + mov %r10, %rax + + /* Check for TDVMCALL success: 0 - Successful, otherwise failed */ + test %rax, %rax + jnz 1f + + /* Check for a TDVMCALL output struct */ + test %r9, %r9 + jz 1f
I'd also include a note that %r9 was neither writable nor its value exposed to the VMM.
Quoted 27 lines
arch/x86/kernel/tdcall.S new file +27
-0+ /* Copy TDVMCALL result registers to output struct: */ + movq %r11, TDVMCALL_r11(%r9) + movq %r12, TDVMCALL_r12(%r9) + movq %r13, TDVMCALL_r13(%r9) + movq %r14, TDVMCALL_r14(%r9) + movq %r15, TDVMCALL_r15(%r9) +1: + /* + * Zero out registers exposed to the VMM to avoid + * speculative execution with VMM-controlled values. + */ + xor %r10d, %r10d + xor %r11d, %r11d + xor %r12d, %r12d + xor %r13d, %r13d + xor %r14d, %r14d + xor %r15d, %r15d + + /* Restore non-volatile GPRs that are exposed to the VMM. */ + pop %r12 + pop %r13 + pop %r14 + pop %r15 + + FRAME_END + ret +2:
arch/x86/kernel/tdcall.S line 200
Quoted 10 lines
arch/x86/kernel/tdcall.S new file +10
-0+ ud2 +SYM_CODE_END(do_tdvmcall) + +/* Helper function for standard type of TDVMCALL */ +SYM_FUNC_START(__tdvmcall) + /* Set TDVMCALL type info (0 - Standard, > 0 - vendor) in R10 */ + xor %r10, %r10 + call do_tdvmcall + retq +SYM_FUNC_END(__tdvmcall)
Why do we need this helper? Why does it need to be in assembly?
Quoted 5 lines
arch/x86/kernel/tdx.c +0
-0
arch/x86/kernel/tdx.c line 10
Quoted 10 lines
arch/x86/kernel/tdx.c +5
-0@@ -1,8 +1,44 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (C) 2020 Intel Corporation */ +#define pr_fmt (fmt) "TDX: " fmt +#include <asm/tdx.h> +/* +* Wrapper for use case that checks for error code and print warning message. +*/
This comment isn't very useful. I can see the error check and warning by reading the code.
Quoted 20 lines
arch/x86/kernel/tdx.c +20
-0+static inline u64 tdvmcall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +{ +u64 err; + + err =__tdvmcall (fn, r12, r13, r14, r15,NULL ); + +if (err) +pr_warn_ratelimited ("TDVMCALL fn:%llx failed with err:%llx\n" , + fn, err); + +return err; +} + +/* +* Wrapper for the semi-common case where we need single output value (R11). +*/ +static inline u64 tdvmcall_out_r11 (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +{ + +struct tdvmcall_output out = {0 };
arch/x86/kernel/tdx.c line 40
Quoted 10 lines
arch/x86/kernel/tdx.c +10
-0+u64 err; + + err =__tdvmcall (fn, r12, r13, r14, r15, &out); + +if (err) +pr_warn_ratelimited ("TDVMCALL fn:%llx failed with err:%llx\n" , + fn, err); + +return out.r11; +}
How do callers check for errors? Is the error value superfluously returned in r11 and another output register?
Re: [RFC v2 02/32] x86/tdx: Introduce INTEL_TDX_GUEST config option
In reply to Kuppuswamy SathyanarayananQuoted 18 lines · Kuppuswamy Sathyanarayanan
Add
INTEL_TDX_GUESTconfig option to selectively compile TDX guest support.Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>Reviewed-by Andi Kleen <[email protected]>Reviewed-by Tony Luck <[email protected]>--- arch/x86/Kconfig | 15 +++++++++++++++ 1 file changed, 15 insertions(+)arch/x86/Kconfig +0
-0@@ -875,6 +875,21 @@ config ACRN_GUEST IOT with small footprint and real-time features. More details can be found in https://projectacrn.org/.
arch/x86/Kconfig config ACRN_GUEST · line 887
Quoted 10 lines
arch/x86/Kconfig +10
-0+config INTEL_TDX_GUEST + bool "Intel Trusted Domain eXtensions Guest Support" + depends on X86_64 && CPU_SUP_INTEL && PARAVIRT + depends on SECURITY + select PARAVIRT_XL + select X86_X2APIC + select SECURITY_LOCKDOWN_LSM + help + Provide support for running in a trusted domain on Intel processors + equipped with Trusted Domain eXtenstions. TDX is an new Intel
a new Intel
Quoted 9 lines
arch/x86/Kconfig +5
-0+ technology that extends VMX and Memory Encryption with a new kind of + virtual machine guest called Trust Domain (TD). A TD is designed to + run in a CPU mode that protects the confidentiality of TD memory + contents and the TD’s CPU state from other software, including VMM. + endif #HYPERVISOR_GUEST source "arch/x86/Kconfig.cpu"
Signature · 1 line
~Randy
Re: [RFC v2 05/32] x86/tdx: Add __tdcall() and __tdvmcall() helper functions
In reply to Dave HansenQuoted 11 lines · Dave Hansen
arch/x86/kernel/tdcall.S line 13
Quoted 5 lines
arch/x86/kernel/tdcall.S new file +5
-0+/* + * Expose registers R10-R15 to VMM (for bitfield info + * refer to TDX GHCI specification). + */ +#define TDVMCALL_EXPOSE_REGS_MASK 0xfc00Why can't we do:
#define
TDC_R10BIT(18)
#defineTDC_R11BIT(19)
Quoted 7 lines
and:
#define
TDVMCALL_EXPOSE_REGS_MASK(TDX_R10|TDX_R11|TDX_R12...or at least:
#define
TDVMCALL_EXPOSE_REGS_MASKBIT(18) | BIT(19) ...
If this is the preferred way, I will change it use macros (TDX_Rxx).
Quoted 14 lines
?
arch/x86/kernel/tdcall.S line 23
Quoted 9 lines
arch/x86/kernel/tdcall.S new file +9
-0+/* + * TDX guests use the TDCALL instruction to make + * hypercalls to the VMM. It is supported in + * Binutils >= 2.36. + */ +#define tdcall .byte 0x66,0x0f,0x01,0xcc + +/* + * __tdcall() - Used to communicate with the TDX moduleWhy is this function here? What does it do? Why do we need it?
__tdcall() function is used to request services from the TDX Module. Example use cases are, TDREPORT, VEINFO, TDINFO, etc.
Quoted 20 lines
I'd like this to actually talk about doing impedance matching between the function call and TDCALL ABIs.
Quoted 9 lines
+ * @arg1 (RDI) - TDCALL Leaf ID
+ * @arg2 (RSI) - Input parameter 1 passed to TDX module
+ * via register RCX
+ * @arg2 (RDX) - Input parameter 2 passed to TDX module
+ * via register RDX
+ * @arg3 (RCX) - Input parameter 3 passed to TDX module
+ * via register R8
+ * @arg4 (R8) - Input parameter 4 passed to TDX module
+ * via register R9The unnecessary repitition and verbosity actually make this harder to read. This looks like it was easy to write, but not much effort is being made to make it easy to consume. Could you please apply some consideration to making it more readable?
Quoted 8 lines
Quoted 3 lines
+ * @arg5 (R9) - struct
tdcall_outputpointer
+ *
+ * @out - Return status of tdcall via RAX.Don't comments usually just say "returns ... foo"? Also, the @params usually refer to *REAL* variable names. Where the heck does "out" come from? Why are you even putting argX? Shouldn't these be @'s be their literal function argument names?
I have added this comment block to make it easier for us to understand the register mapping between function arguments and TDCALL ABI. But I got your point. Usage of @arg1 or @out does not comply the function comment standards. I will fix this in next version.
Quoted 2 lines
@rdi - Input parameter, moved to RCX
I will use the above format to document function arguments.
Quoted 17 lines
arch/x86/kernel/tdcall.S line 48
Quoted 12 lines
+ * NOTE: This function should only used for non TDVMCALL
+ * use cases
+ */arch/x86/kernel/tdcall.S new file +8
-0+SYM_FUNC_START(__tdcall) + FRAME_BEGIN + + /* Save non-volatile GPRs that are exposed to the VMM. */ + push %r15 + push %r14 + push %r13 + push %r12Why do we have to save these? Because they might be clobbered? If so, let's say *THAT* instead of just "exposed". "Exposed" could mean "VMM can read".
Quoted 2 lines
Also, this just told me that this function can't be used to talk to the VMM. Why is this talking about exposure to the VMM?
Although __tdcall() is only used to communicate with the TDX module and the TDX module is not supposed to touch these registers, just to be on the safe side, I have tried to save the context of registers R12-R15. Anyway cycles used by instructions are less compared to tdcall.
Quoted 8 lines
arch/x86/kernel/tdcall.S line 53
Quoted 5 lines
arch/x86/kernel/tdcall.S new file +4
-0+ /* Move TDCALL Leaf ID to RAX */ + mov %rdi, %rax + /* Move output pointer to R12 */ + mov %r9, %r12I thought 'struct
tdcall_output' was a purely software construct. Why are we passing a pointer to it into TDCALL?
Its used to store the TDCALL result (RCX, RDX, R8-R11). As far as this function is concerned, its just a block of memory (accessed using base address + TDCALL_r* offsets).
Quoted 11 lines
arch/x86/kernel/tdcall.S line 60
Quoted 8 lines
arch/x86/kernel/tdcall.S new file +7
-0+ /* Move input param 4 to R9 */ + mov %r8, %r9 + /* Move input param 3 to R8 */ + mov %rcx, %r8 + /* Leave input param 2 in RDX */ + /* Move input param 1 to RCX */ + mov %rsi, %rcxWith a little work, this can be made a *LOT* more readable:
Quoted 7 lines
/* Mangle function call ABI into TDCALL ABI: */ mov %rdi, %rax /* Move TDCALL Leaf ID to RAX */ mov %r9, %r12 /* Move output pointer to R12 */ mov %r8, %r9 /* Move input 4 to R9 */ mov %rcx, %r8 /* Move input 3 to R8 */ mov %rsi, %rcx /* Move input 1 to RCX */ /* Leave input param 2 in RDX */
Ok. I will use your version.
Quoted 14 lines
arch/x86/kernel/tdcall.S line 70
Quoted 11 lines
arch/x86/kernel/tdcall.S new file +9
-0+ tdcall + + /* Check for TDCALL success: 0 - Successful, otherwise failed */ + test %rax, %rax + jnz 1f + + /* Check for a TDCALL output struct */ + test %r12, %r12 + jz 1fDoes some universal status come back in r12? Aren't we dealing with a VMM/SEAM-controlled register here? Isn't this dangerous?
R12 is the temporary register we have used to store the address of user passed output pointer. We just check for NULL condition here. R12 will not be used by the TDX module.
If you prefer, we can just push the output pointer to stack and get it after we make the tdcall.
Quoted 10 lines
Quoted 10 lines
arch/x86/kernel/tdcall.S new file +9
-0+ /* Copy TDCALL result registers to output struct: */ + movq %rcx, TDCALL_rcx(%r12) + movq %rdx, TDCALL_rdx(%r12) + movq %r8, TDCALL_r8(%r12) + movq %r9, TDCALL_r9(%r12) + movq %r10, TDCALL_r10(%r12) + movq %r11, TDCALL_r11(%r12) +1: + /* Zero out registers exposed to the TDX Module. */
Quoted 8 lines
arch/x86/kernel/tdcall.S line 86
Quoted 6 lines
arch/x86/kernel/tdcall.S new file +6
-0+ xor %rcx, %rcx + xor %rdx, %rdx + xor %r8d, %r8d + xor %r9d, %r9d + xor %r10d, %r10d + xor %r11d, %r11d... why?
These registers are used by the TDX Module. Why pass the stale values back to the user? So we clear them here.
Quoted 22 lines
Quoted 22 lines
arch/x86/kernel/tdcall.S new file +21
-0+ /* Restore non-volatile GPRs that are exposed to the VMM. */ + pop %r12 + pop %r13 + pop %r14 + pop %r15 + + FRAME_END + ret +SYM_FUNC_END(__tdcall) + +/* + * do_tdvmcall() - Used to communicate with the VMM. + * + * @arg1 (RDI) - TDVMCALL function, e.g. exit reason + * @arg2 (RSI) - Input parameter 1 passed to VMM + * via register R12 + * @arg3 (RDX) - Input parameter 2 passed to VMM + * via register R13 + * @arg4 (RCX) - Input parameter 3 passed to VMM + * via register R14 + * @arg5 (R8) - Input parameter 4 passed to VMM
Quoted 8 lines
Quoted 6 lines
+ * via register R15
+ * @arg6 (R9) - structtdvmcall_outputpointer
+ *
+ * @out - Return status of tdvmcall(R10) via RAX.
+ *
+ */Same comments on the sparse comment style.
will fix it similar to __tdcall().
Quoted 14 lines
arch/x86/kernel/tdcall.S line 124
Quoted 11 lines
arch/x86/kernel/tdcall.S new file +10
-0+SYM_CODE_START_LOCAL(do_tdvmcall) + FRAME_BEGIN + + /* Save non-volatile GPRs that are exposed to the VMM. */ + push %r15 + push %r14 + push %r13 + push %r12 + + /* Set TDCALL leaf ID to TDVMCALL (0) in RAX */I think there needs to be some discussion of what TDCALL and TDVMCALL are. They are named too similarly not to do so.
TDVMCALL is the sub function of TDCALL (selected by setting RAX register to 0). TDVMCALL is used to request services from VMM.
Quoted 11 lines
Quoted 11 lines
arch/x86/kernel/tdcall.S new file +10
-0+ xor %eax, %eax + /* Move TDVMCALL function id (1st argument) to R11 */ + mov %rdi, %r11> + /* Move Input parameter 1-4 to R12-R15 */ + mov %rsi, %r12 + mov %rdx, %r13 + mov %rcx, %r14 + mov %r8, %r15 + /* Leave tdvmcall output pointer in R9 */ + + /*
Quoted 8 lines
arch/x86/kernel/tdcall.S line 140
Quoted 5 lines
arch/x86/kernel/tdcall.S new file +5
-0+ * Value of RCX is used by the TDX Module to determine which + * registers are exposed to VMM. Each bit in RCX represents a + * register id. You can find the bitmap details from TDX GHCI + * spec. + */This doesn't belong here. Put it along with the
TDVMCALL_EXPOSE_REGS_MASK, please.
Ok. I will do it.
Quoted 15 lines
arch/x86/kernel/tdcall.S line 151
Quoted 12 lines
arch/x86/kernel/tdcall.S new file +11
-0+ movl $TDVMCALL_EXPOSE_REGS_MASK, %ecx + + tdcall + + /* + * Check for TDCALL success: 0 - Successful, otherwise failed. + * If failed, there is an issue with TDX Module which is fatal + * for the guest. So panic. + */ + test %rax, %rax + jnz 2fSo, just to be clear: %RAX is under the control of the SEAM module. The VMM has no control over it. Right?
AFAIK, VMM will not touch it.
Sean, please confirm it.
Quoted 2 lines
Shouldn't we say that explicitly?
I can add it to above comment.
Quoted 14 lines
arch/x86/kernel/tdcall.S line 162
Quoted 11 lines
arch/x86/kernel/tdcall.S new file +10
-0+ /* Move TDVMCALL success/failure to RAX to return to user */ + mov %r10, %rax + + /* Check for TDVMCALL success: 0 - Successful, otherwise failed */ + test %rax, %rax + jnz 1f + + /* Check for a TDVMCALL output struct */ + test %r9, %r9 + jz 1fI'd also include a note that %r9 was neither writable nor its value exposed to the VMM.
will do it.
Quoted 32 lines
Quoted 32 lines
arch/x86/kernel/tdcall.S new file +31
-0+ /* Copy TDVMCALL result registers to output struct: */ + movq %r11, TDVMCALL_r11(%r9) + movq %r12, TDVMCALL_r12(%r9) + movq %r13, TDVMCALL_r13(%r9) + movq %r14, TDVMCALL_r14(%r9) + movq %r15, TDVMCALL_r15(%r9) +1: + /* + * Zero out registers exposed to the VMM to avoid + * speculative execution with VMM-controlled values. + */ + xor %r10d, %r10d + xor %r11d, %r11d + xor %r12d, %r12d + xor %r13d, %r13d + xor %r14d, %r14d + xor %r15d, %r15d + + /* Restore non-volatile GPRs that are exposed to the VMM. */ + pop %r12 + pop %r13 + pop %r14 + pop %r15 + + FRAME_END + ret +2: + ud2 +SYM_CODE_END(do_tdvmcall) + +/* Helper function for standard type of TDVMCALL */
Quoted 8 lines
arch/x86/kernel/tdcall.S line 200
Quoted 6 lines
arch/x86/kernel/tdcall.S new file +6
-0+SYM_FUNC_START(__tdvmcall) + /* Set TDVMCALL type info (0 - Standard, > 0 - vendor) in R10 */ + xor %r10, %r10 + call do_tdvmcall + retq +SYM_FUNC_END(__tdvmcall)Why do we need this helper? Why does it need to be in assembly?
Its simpler to do it in assembly. Also, grouping all register updates in the same file will make it easier for us to read or debug issues. Another reason is, we also call do_tdvmcall() from in/out instruction use case.
Quoted 13 lines
Quoted 13 lines
arch/x86/kernel/tdx.c +2
-0@@ -1,8 +1,44 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (C) 2020 Intel Corporation */ +#define pr_fmt (fmt) "TDX: " fmt +#include <asm/tdx.h>
Quoted 6 lines
arch/x86/kernel/tdx.c line 10
Quoted 3 lines
arch/x86/kernel/tdx.c +3
-0+/* +* Wrapper for use case that checks for error code and print warning message. +*/ This comment isn't very useful. I can see the error check and warning by reading the code.
Its just a helper function that covers common case of checking for error and print the warning message. If this comment is superfluous, I can remove it.
Quoted 26 lines
Quoted 26 lines
arch/x86/kernel/tdx.c +25
-0+static inline u64 tdvmcall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +{ +u64 err; + + err =__tdvmcall (fn, r12, r13, r14, r15,NULL ); + +if (err) +pr_warn_ratelimited ("TDVMCALL fn:%llx failed with err:%llx\n" , + fn, err); + +return err; +} + +/* +* Wrapper for the semi-common case where we need single output value (R11). +*/ +static inline u64 tdvmcall_out_r11 (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +{ + +struct tdvmcall_output out = {0 }; +u64 err; + + err =__tdvmcall (fn, r12, r13, r14, r15, &out); + +if (err)
Quoted 8 lines
arch/x86/kernel/tdx.c line 40
Quoted 5 lines
arch/x86/kernel/tdx.c +5
-0+pr_warn_ratelimited ("TDVMCALL fn:%llx failed with err:%llx\n" , + fn, err); + +return out.r11; +}How do callers check for errors? Is the error value superfluously returned in r11 and another output register?
We already check for error in this helper function. User of this function only cares about output value (R11). Mainly for in/out use case.
Quoted 1 line
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 02/32] x86/tdx: Introduce INTEL_TDX_GUEST config option
In reply to Randy DunlapQuoted 19 lines · Randy Dunlap
Quoted 18 lines · Kuppuswamy Sathyanarayanan
Add
INTEL_TDX_GUESTconfig option to selectively compile TDX guest support.Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>Reviewed-by Andi Kleen <[email protected]>Reviewed-by Tony Luck <[email protected]>--- arch/x86/Kconfig | 15 +++++++++++++++ 1 file changed, 15 insertions(+)arch/x86/Kconfig +0
-0@@ -875,6 +875,21 @@ config ACRN_GUEST IOT with small footprint and real-time features. More details can be found in https://projectacrn.org/.
Quoted 13 lines
arch/x86/Kconfig config ACRN_GUEST · line 887
Quoted 10 lines
arch/x86/Kconfig +10
-0+config INTEL_TDX_GUEST + bool "Intel Trusted Domain eXtensions Guest Support" + depends on X86_64 && CPU_SUP_INTEL && PARAVIRT + depends on SECURITY + select PARAVIRT_XL + select X86_X2APIC + select SECURITY_LOCKDOWN_LSM + help + Provide support for running in a trusted domain on Intel processors + equipped with Trusted Domain eXtenstions. TDX is an new Intela new Intel
Good catch. I will fix it in next version.
Quoted 11 lines
Quoted 9 lines
arch/x86/Kconfig +5
-0+ technology that extends VMX and Memory Encryption with a new kind of + virtual machine guest called Trust Domain (TD). A TD is designed to + run in a CPU mode that protects the confidentiality of TD memory + contents and the TD’s CPU state from other software, including VMM. + endif #HYPERVISOR_GUEST source "arch/x86/Kconfig.cpu"
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 05/32] x86/tdx: Add __tdcall() and __tdvmcall() helper functions
In reply to Kuppuswamy, SathyanarayananQuoted 9 lines · Sathyanarayanan
Quoted 6 lines
arch/x86/kernel/tdcall.S line 23
Quoted 4 lines
arch/x86/kernel/tdcall.S new file +4
-0+#define tdcall .byte 0x66,0x0f,0x01,0xcc + +/* + * __tdcall() - Used to communicate with the TDX moduleWhy is this function here? What does it do? Why do we need it?
__tdcall()function is used to request services from the TDX Module. Example use cases are, TDREPORT, VEINFO, TDINFO, etc.
I think there might be some misinterpretation of my question. What you are describing is what *TDCALL* does. Why do we need a wrapper function? What purpose does this wrapper function serve? Why do we need this wrapper function?
Quoted 16 lines
Quoted 16 lines
arch/x86/kernel/tdcall.S line 48
Quoted 11 lines
+ * NOTE: This function should only used for non TDVMCALL
+ * use cases
+ */arch/x86/kernel/tdcall.S new file +8
-0+SYM_FUNC_START(__tdcall) + FRAME_BEGIN + + /* Save non-volatile GPRs that are exposed to the VMM. */ + push %r15 + push %r14 + push %r13 + push %r12Why do we have to save these? Because they might be clobbered? If so, let's say *THAT* instead of just "exposed". "Exposed" could mean "VMM can read".
Quoted 7 lines
Quoted 2 lines
Also, this just told me that this function can't be used to talk to the VMM. Why is this talking about exposure to the VMM?
Although
__tdcall()is only used to communicate with the TDX module and the TDX module is not supposed to touch these registers, just to be on the safe side, I have tried to save the context of registers R12-R15. Anyway cycles used by instructions are less compared to tdcall.
Why are you talking about the VMM if this is a call to the SEAM module?
Let's say someone is reading the TDCALL architecture spec. It will say something like, "blah blah, in this case TDCALL will not modify %r12->%r15". Then someone goes and looks at this code that basically says (or implies) "save these before the SEAM module modifies them". What is a coder to do?
Please remove the ambiguity, either by removing this superfluous (according to the spec) code, or documenting why it is not superfluous.
Quoted 11 lines
Quoted 7 lines
Quoted 4 lines
+ /* Move TDCALL Leaf ID to RAX */
+ mov %rdi, %rax
+ /* Move output pointer to R12 */
+ mov %r9, %r12I thought 'struct
tdcall_output' was a purely software construct. Why are we passing a pointer to it into TDCALL?Its used to store the TDCALL result (RCX, RDX, R8-R11). As far as this function is concerned, its just a block of memory (accessed using base address + TDCALL_r* offsets).
Is 'struct tdcall_output' a hardware architectural structure or a software structure?
If it's a software structure, then why are we passing a pointer to a software structure into a hardware ABI?
If it's a hardware architecture structure, where is the documentation for it?
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
+ tdcall
+
+ /* Check for TDCALL success: 0 - Successful, otherwise failed */ + test %rax, %raxQuoted 8 lines
Quoted 5 lines
+ jnz 1f
+
+ /* Check for a TDCALL output struct */
+ test %r12, %r12
+ jz 1fDoes some universal status come back in r12? Aren't we dealing with a VMM/SEAM-controlled register here? Isn't this dangerous?
Quoted 3 lines
R12 is the temporary register we have used to store the address of user passed output pointer. We just check for NULL condition here. R12 will not be used by the TDX module.
OK, so how do you know this? Could you share your logic, please?
Quoted 2 lines
If you prefer, we can just push the output pointer to stack and get it after we make the tdcall.
I prefer that the code be understandable and be written for a clear purpose. If you're using r12 for temporary storage, I expect to see at least one reference *SOMEWHERE* to its use as temporary storage. Right now.... nothing.
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
+ /* Copy TDCALL result registers to output struct: */ + movq %rcx,
TDCALL_rcx(%r12)
+ movq %rdx,TDCALL_rdx(%r12)
+ movq %r8,TDCALL_r8(%r12)
+ movq %r9,TDCALL_r9(%r12)
+ movq %r10,TDCALL_r10(%r12)
+ movq %r11,TDCALL_r11(%r12)
+1:
+ /* Zero out registers exposed to the TDX Module. */ + xor %rcx, %rcx
+ xor %rdx, %rdx
+ xor %r8d, %r8d
+ xor %r9d, %r9d
+ xor %r10d, %r10d
+ xor %r11d, %r11d
Quoted 4 lines
Quoted 1 line
... why?
These registers are used by the TDX Module. Why pass the stale values back to the user? So we clear them here.
Please go look at some other assembly code in the kernel called from C.
Do those functions do this? Why? Why not? Do they care about
"passing stale values back up"?
Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
arch/x86/kernel/tdcall.S new file +5
-0+SYM_CODE_START_LOCAL(do_tdvmcall) + FRAME_BEGIN + + /* Save non-volatile GPRs that are exposed to the VMM. */ + push %r15Quoted 8 lines
Quoted 5 lines
+ push %r14
+ push %r13
+ push %r12
+
+ /* Set TDCALL leaf ID to TDVMCALL (0) in RAX */I think there needs to be some discussion of what TDCALL and TDVMCALL are. They are named too similarly not to do so.
Quoted 2 lines
TDVMCALL is the sub function of TDCALL (selected by setting RAX register to 0). TDVMCALL is used to request services from VMM.
Actually, I think these functions are horribly misnamed.
I think we should make them
__tdx_seam_call()
or __tdx_module_call()
and
__tdx_hypercall() __tdcall() and __tdvmcall()
are really nonsensical in this context, especially since TDVMCALL is implemented with the TDCALL instruction, but not the __tdcall() function.
Quoted 10 lines
Quoted 9 lines
arch/x86/kernel/tdcall.S line 200
Quoted 7 lines
arch/x86/kernel/tdcall.S new file +7
-0+/* Helper function for standard type of TDVMCALL */ +SYM_FUNC_START(__tdvmcall) + /* Set TDVMCALL type info (0 - Standard, > 0 - vendor) in R10 */ + xor %r10, %r10 + call do_tdvmcall + retq +SYM_FUNC_END(__tdvmcall)Why do we need this helper? Why does it need to be in assembly?
Quoted 4 lines
Its simpler to do it in assembly. Also, grouping all register updates in the same file will make it easier for us to read or debug issues. Another
reason is, we also calldo_tdvmcall()from in/out instruction use case.
Sathya, I seem to have to reverse-engineer what you are doing for all this stuff. Your answers to my questions are almost entirely orthogonal to the things I really want to know. I guess I need to be more precise with the questions I'm asking. But, this is yet another case where I think the burden for this series continues to fall on the reviewer rather than the submitter. Not the way I think it is best.
So, trying to reverse-engineer what you are doing here... it seems that you can't *practically* call do_tdvmcall() directly because %r10 would be garbage. That makes this (or a wrapper like it) required for every practical call to do_tdvmcall().
But, even if that's the case, you need to *DOCUMENT* that up in do_tdvmcall(): Hey, this function is worthless without something that sets up %r10 before calling it.
I'm also not *SURE* this is simpler to do in assembly.
Quoted 18 lines
Quoted 9 lines
Quoted 9 lines
arch/x86/kernel/tdx.c +0
-0@@ -1,8 +1,44 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (C) 2020 Intel Corporation */+#define
pr_fmt(fmt)"TDX: " fmt
+Quoted 8 lines
Quoted 5 lines
#include <
asm/tdx.h>
+/*
+ * Wrapper for use case that checks for error code and print warning message.
+ */This comment isn't very useful. I can see the error check and warning by reading the code.
Quoted 3 lines
Its just a helper function that covers common case of checking for error and print the warning message. If this comment is superfluous, I can remove it.
I'd prefer that you actually write a comment about what the function is doing, maybe:
/*
* Wrapper for simple hypercalls that only return a success/error code. */
... or *SOMETHING* that tells what its purpose in life is.
Quoted 36 lines
Quoted 27 lines
Quoted 27 lines
arch/x86/kernel/tdx.c +15
-0+static inline u64 tdvmcall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +{ + u64 err; + + err = __tdvmcall(fn, r12, r13, r14, r15, NULL); + + if (err) + pr_warn_ratelimited("TDVMCALL fn:%llx failed with err:%llx\n", + fn, err); + + return err; +} + +/* +* Wrapper for the semi-common case where we need single output value (R11).
+ */
+static inline u64tdvmcall_out_r11(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15)arch/x86/kernel/tdx.c +8
-0+{ + + struct tdvmcall_output out = {0}; + u64 err; + + err = __tdvmcall(fn, r12, r13, r14, r15, &out); + + if (err)Quoted 8 lines
arch/x86/kernel/tdx.c line 40
Quoted 5 lines
+
pr_warn_ratelimited("TDVMCALL fn:%llx failed with err:%llx\n", + fn, err);arch/x86/kernel/tdx.c +3
-0+ + return out.r11; +}How do callers check for errors? Is the error value superfluously returned in r11 and another output register?
Quoted 2 lines
We already check for error in this helper function. User of this function only cares about output value (R11). Mainly for in/out use case.
That's pretty valuable information.
Re: [RFC v2 05/32] x86/tdx: Add __tdcall() and __tdvmcall() helper functions
Quoted 8 lines · Dave Hansen
Quoted 7 lines · Sathyanarayanan
Quoted 6 lines
arch/x86/kernel/tdcall.S line 23
Quoted 4 lines
arch/x86/kernel/tdcall.S new file +4
-0+#define tdcall .byte 0x66,0x0f,0x01,0xcc + +/* + * __tdcall() - Used to communicate with the TDX moduleWhy is this function here? What does it do? Why do we need it?
Quoted 8 lines
Quoted 2 lines
__tdcall()function is used to request services from the TDX Module. Example use cases are, TDREPORT, VEINFO, TDINFO, etc.I think there might be some misinterpretation of my question. What you are describing is what *TDCALL* does. Why do we need a wrapper function? What purpose does this wrapper function serve? Why do we need this wrapper function?
How about following explanation?
Helper function for "tdcall" instruction, which can be used to request services from the TDX module (does not include VMM). Few examples of valid TDX module services are, "TDREPORT", "MEM PAGE ACCEPT", "VEINFO", etc.
This function serves as a wrapper to move user call arguments to the correct registers as specified by "tdcall" ABI and shares it with the TDX module. If the "tdcall" operation is successful and a valid "struct tdcall_out" pointer is available (in "out" argument), output from the TDX module (RCX, RDX, R8-R11) is saved to the memory specified in the "out" pointer. Also the status of the "tdcall" operation is returned back to the user as a function return value.
Quoted 14 lines
Quoted 11 lines
Quoted 6 lines
Why do we have to save these? Because they might be clobbered? If so, let's say *THAT* instead of just "exposed". "Exposed" could mean "VMM can read".
Also, this just told me that this function can't be used to talk to the VMM. Why is this talking about exposure to the VMM?
Although
__tdcall()is only used to communicate with the TDX module and the TDX module is not supposed to touch these registers, just to be on the safe side, I have tried to save the context of registers R12-R15. Anyway cycles used by instructions are less compared to tdcall.Why are you talking about the VMM if this is a call to the SEAM module?
Quoted 8 lines
Let's say someone is reading the TDCALL architecture spec. It will say something like, "blah blah, in this case TDCALL will not modify %r12->%r15". Then someone goes and looks at this code that basically says (or implies) "save these before the SEAM module modifies them". What is a coder to do?
Please remove the ambiguity, either by removing this superfluous (according to the spec) code, or documenting why it is not superfluous.
Agree. I will remove the save/restore context code.
Quoted 13 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
+ /* Move TDCALL Leaf ID to RAX */
+ mov %rdi, %rax
+ /* Move output pointer to R12 */
+ mov %r9, %r12Quoted 6 lines
Quoted 2 lines
I thought 'struct
tdcall_output' was a purely software construct. Why are we passing a pointer to it into TDCALL?Its used to store the TDCALL result (RCX, RDX, R8-R11). As far as this function is concerned, its just a block of memory (accessed using base address + TDCALL_r* offsets).
Quoted 9 lines
Is 'struct
tdcall_output' a hardware architectural structure or a software structure?If it's a software structure, then why are we passing a pointer to a software structure into a hardware ABI?
If it's a hardware architecture structure, where is the documentation for it?
I think there is a misunderstanding here. We don't share the tdcall_output pointer with the TDX module. Current use cases of TDCALL (other than TDVMCALL) do not use registers from R12-R15. Since the registers R12-R15 are free and available, we are using R12 as temporary storage to hold the tdcall_output pointer.
I will include some comment about using it as temporary storage.
Quoted 6 lines
I prefer that the code be understandable and be written for a clear purpose. If you're using r12 for temporary storage, I expect to see at least one reference *SOMEWHERE* to its use as temporary storage. Right now.... nothing.
I will include some reference to it.
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
+ /* Copy TDCALL result registers to output struct: */ + movq %rcx,
TDCALL_rcx(%r12)
+ movq %rdx,TDCALL_rdx(%r12)
+ movq %r8,TDCALL_r8(%r12)
+ movq %r9,TDCALL_r9(%r12)
+ movq %r10,TDCALL_r10(%r12)
+ movq %r11,TDCALL_r11(%r12)
+1:
+ /* Zero out registers exposed to the TDX Module. */ + xor %rcx, %rcx
+ xor %rdx, %rdx
+ xor %r8d, %r8d
+ xor %r9d, %r9d
+ xor %r10d, %r10d
+ xor %r11d, %r11d
Quoted 9 lines
Quoted 4 lines
Quoted 1 line
... why?
These registers are used by the TDX Module. Why pass the stale values back to the user? So we clear them here.
Please go look at some other assembly code in the kernel called from C.
Do those functions do this? Why? Why not? Do they care about
"passing stale values back up"?
Maybe I am being overly cautious here. Since TDX module is the trusted code, speculation attack is not a consideration here. I will remove this block of code.
Quoted 33 lines
Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
arch/x86/kernel/tdcall.S new file +5
-0+SYM_CODE_START_LOCAL(do_tdvmcall) + FRAME_BEGIN + + /* Save non-volatile GPRs that are exposed to the VMM. */ + push %r15Quoted 8 lines
Quoted 5 lines
+ push %r14
+ push %r13
+ push %r12
+
+ /* Set TDCALL leaf ID to TDVMCALL (0) in RAX */I think there needs to be some discussion of what TDCALL and TDVMCALL are. They are named too similarly not to do so.
Quoted 2 lines
TDVMCALL is the sub function of TDCALL (selected by setting RAX register to 0). TDVMCALL is used to request services from VMM.
Actually, I think these functions are horribly misnamed.
I think we should make them
__tdx_seam_call()or
__tdx_module_call()and
__tdx_hypercall() __tdcall() and __tdvmcall()
Quoted 3 lines
are really nonsensical in this context, especially since TDVMCALL is implemented with the TDCALL instruction, but not the
__tdcall()function.
TDVMCALL is a short form of "TDG.VP.VMCALL". This term usage came from GHCI document. We can read it as "Trusted Domain VMCALL". Maybe because we are used to GHCI spec, we don't find it confusing. I agree that if you consider the "tdcall" instruction usage, it is confusing.
But if it's confusing for new readers and rename is preferred,
Do we need to rename the helper functions ?
tdvmcall(), tdvmcall_out_r11()
Also what about output structs?
struct tdcall_output
struct tdvmcall_output
Quoted 15 lines
Quoted 10 lines
Quoted 9 lines
arch/x86/kernel/tdcall.S line 200
Quoted 7 lines
arch/x86/kernel/tdcall.S new file +7
-0+/* Helper function for standard type of TDVMCALL */ +SYM_FUNC_START(__tdvmcall) + /* Set TDVMCALL type info (0 - Standard, > 0 - vendor) in R10 */ + xor %r10, %r10 + call do_tdvmcall + retq +SYM_FUNC_END(__tdvmcall)Why do we need this helper? Why does it need to be in assembly?
Quoted 4 lines
Its simpler to do it in assembly. Also, grouping all register updates in the same file will make it easier for us to read or debug issues. Another
reason is, we also calldo_tdvmcall()from in/out instruction use case.
Quoted 6 lines
Sathya, I seem to have to reverse-engineer what you are doing for all this stuff. Your answers to my questions are almost entirely orthogonal to the things I really want to know. I guess I need to be more precise with the questions I'm asking. But, this is yet another case where I think the burden for this series continues to fall on the reviewer rather than the submitter. Not the way I think it is best.
I have assumed that you are aware of reason for the existence of do_tdvmcall() helper function. It is mainly created to hold common code between vendor specific and standard type of tdvmcall's.
But it is a mistake from my end. I will try to be elaborate in my future replies.
Quoted 9 lines
So, trying to reverse-engineer what you are doing here... it seems that you can't *practically* call
do_tdvmcall()directly because %r10 would be garbage. That makes this (or a wrapper like it) required for every practical call todo_tdvmcall().But, even if that's the case, you need to *DOCUMENT* that up in
do_tdvmcall(): Hey, this function is worthless without something that sets up %r10 before calling it.
Agree. This needs to be documented. I will add it in next version.
Quoted 25 lines
I'm also not *SURE* this is simpler to do in assembly.
Quoted 18 lines
Quoted 9 lines
Quoted 9 lines
arch/x86/kernel/tdx.c +1
-0@@ -1,8 +1,44 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (C) 2020 Intel Corporation */ +#define pr_fmt(fmt) "TDX: " fmt +Quoted 8 lines
Quoted 5 lines
arch/x86/kernel/tdx.c +1
-0#include <asm/tdx.h> +/* +* Wrapper for use case that checks for error code and print warning message.
+ */This comment isn't very useful. I can see the error check and warning by reading the code.
Quoted 3 lines
Its just a helper function that covers common case of checking for error and print the warning message. If this comment is superfluous, I can remove it.
Quoted 8 lines
I'd prefer that you actually write a comment about what the function is doing, maybe:
/*
* Wrapper for simple hypercalls that only return a success/error code. */... or *SOMETHING* that tells what its purpose in life is.
I will fix it in next version.
Quoted 37 lines
Quoted 37 lines
Quoted 36 lines
Quoted 23 lines
arch/x86/kernel/tdx.c +15
-0+static inline u64 tdvmcall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +{ + u64 err; + + err = __tdvmcall(fn, r12, r13, r14, r15, NULL); + + if (err) + pr_warn_ratelimited("TDVMCALL fn:%llx failed with err:%llx\n", + fn, err); + + return err; +} + +/* +* Wrapper for the semi-common case where we need single output value (R11).
+ */
+static inline u64tdvmcall_out_r11(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15)arch/x86/kernel/tdx.c +3
-0+{ + + struct tdvmcall_output out = {0};arch/x86/kernel/tdx.c line 40
Quoted 10 lines
arch/x86/kernel/tdx.c +10
-0+ u64 err; + + err = __tdvmcall(fn, r12, r13, r14, r15, &out); + + if (err) + pr_warn_ratelimited("TDVMCALL fn:%llx failed with err:%llx\n", + fn, err); + + return out.r11; +}How do callers check for errors? Is the error value superfluously returned in r11 and another output register?
Quoted 4 lines
Quoted 2 lines
We already check for error in this helper function. User of this function only cares about output value (R11). Mainly for in/out use case.
That's pretty valuable information.
I will include this note in the function comment.
Quoted 1 line
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 05/32] x86/tdx: Add __tdcall() and __tdvmcall() helper functions
Quoted 17 lines · Sathyanarayanan
Quoted 8 lines · Dave Hansen
Quoted 7 lines · Sathyanarayanan
Quoted 6 lines
arch/x86/kernel/tdcall.S line 23
Quoted 4 lines
arch/x86/kernel/tdcall.S new file +4
-0+#define tdcall .byte 0x66,0x0f,0x01,0xcc + +/* + * __tdcall() - Used to communicate with the TDX moduleWhy is this function here? What does it do? Why do we need it?
Quoted 8 lines
Quoted 2 lines
__tdcall()function is used to request services from the TDX Module. Example use cases are, TDREPORT, VEINFO, TDINFO, etc.I think there might be some misinterpretation of my question. What you are describing is what *TDCALL* does. Why do we need a wrapper function? What purpose does this wrapper function serve? Why do we need this wrapper function?
Quoted 6 lines
How about following explanation?
Helper function for "tdcall" instruction, which can be used to request services from the TDX module (does not include VMM). Few examples of valid TDX module services are, "TDREPORT", "MEM PAGE ACCEPT", "VEINFO", etc.
Naming the services here is not useful. If I want to know who calls this, I'll just literally do that: look up the callers of this function.
Quoted 7 lines
This function serves as a wrapper to move user call arguments to the correct registers as specified by "tdcall" ABI and shares it with the TDX module. If the "tdcall" operation is successful and a valid "struct
tdcall_out" pointer is available (in "out" argument), output from the TDX module (RCX, RDX, R8-R11) is saved to the memory specified in the "out" pointer. Also the status of the "tdcall" operation is returned back to the user as a function return value.
I tend to prefer function comments that talk high-level about what the function does rather than waste space on the exact registers used in the ABI. I also tend not to talk about things that can be trivially grepped for, like the callers of this function.
I'd trim the fat out of there, but it's generally OK, although too rotund for my taste.
Quoted 22 lines
Quoted 12 lines
Quoted 11 lines
Quoted 7 lines
Quoted 4 lines
+ /* Move TDCALL Leaf ID to RAX */
+ mov %rdi, %rax
+ /* Move output pointer to R12 */
+ mov %r9, %r12I thought 'struct
tdcall_output' was a purely software construct. Why are we passing a pointer to it into TDCALL?Its used to store the TDCALL result (RCX, RDX, R8-R11). As far as this function is concerned, its just a block of memory (accessed using base address + TDCALL_r* offsets).
Quoted 9 lines
Is 'struct
tdcall_output' a hardware architectural structure or a software structure?If it's a software structure, then why are we passing a pointer to a software structure into a hardware ABI?
If it's a hardware architecture structure, where is the documentation for it?
Quoted 6 lines
I think there is a misunderstanding here. We don't share the
tdcall_outputpointer with the TDX module. Current use cases of TDCALL (other than TDVMCALL)
do not use registers from R12-R15. Since the registers R12-R15 are free and available, we are using R12 as temporary storage to hold thetdcall_outputpointer.
In other words, 'struct tdcall_output' is a purely software concept. However, its pointer is manipulated literally next to all of the TDCALL register arguments and it has an *IDENTICAL* comment to all of those other moves.
Please make it clear that %r12 is not being used at all for the TDCALL instruction itself.
But, the bigger point here is that the code needs to be structured in a way that makes the function and interactions clear. If I want to know more about the "output pointer", where do I go? Do I go looking at the calling functions or the TDINFO instruction reference?
...
Quoted 7 lines
Quoted 3 lines
Please go look at some other assembly code in the kernel called from C. Do those functions do this? Why? Why not? Do they care about "passing stale values back up"?
Maybe I am being overly cautious here. Since TDX module is the trusted code, speculation attack is not a consideration here. I will remove this block of code.
Caution is OK. Caution without explanation somewhere as to why it is warranted is not.
Quoted 3 lines
Do we need to rename the helper functions ?
tdvmcall(),tdvmcall_out_r11()
Yes.
Quoted 4 lines
Also what about output structs?
struct
tdcall_output
structtdvmcall_output
Yes, they need sane, straightforward names which are not confusing too.
Quoted 22 lines
Quoted 15 lines
Quoted 10 lines
Quoted 9 lines
arch/x86/kernel/tdcall.S line 200
Quoted 7 lines
arch/x86/kernel/tdcall.S new file +7
-0+/* Helper function for standard type of TDVMCALL */ +SYM_FUNC_START(__tdvmcall) + /* Set TDVMCALL type info (0 - Standard, > 0 - vendor) in R10 */ + xor %r10, %r10 + call do_tdvmcall + retq +SYM_FUNC_END(__tdvmcall)Why do we need this helper? Why does it need to be in assembly?
Quoted 4 lines
Its simpler to do it in assembly. Also, grouping all register updates in the same file will make it easier for us to read or debug issues. Another
reason is, we also calldo_tdvmcall()from in/out instruction use case.Quoted 6 lines
Sathya, I seem to have to reverse-engineer what you are doing for all this stuff. Your answers to my questions are almost entirely orthogonal to the things I really want to know. I guess I need to be more precise with the questions I'm asking. But, this is yet another case where I think the burden for this series continues to fall on the reviewer rather than the submitter. Not the way I think it is best.
Quoted 3 lines
I have assumed that you are aware of reason for the existence of
do_tdvmcall()helper function. It is mainly created to hold common code between vendor specific and standard type of tdvmcall's.
No, I was not aware of that. Remember, you're not doing this for *ME*.
You're doing it for the hundred other people that are going to look
over the code and who won't have been aware of your reasoning.
Borislav Petkov
maintainerRe: [RFC v2 01/32] x86/paravirt: Introduce CONFIG_PARAVIRT_XL
In reply to Kuppuswamy Sathyanarayanan+ Jürgen.
Quoted 5 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
Split off halt paravirt calls from
CONFIG_PARAVIRT_XXLinto a separate config option. It provides a middle ground for not-so-deep paravirtulized environments.
Please introduce a spellchecker into your patch creation workflow.
Also, what does "not-so-deep" mean?
Quoted 3 lines
CONFIG_PARAVIRT_XLwill be used by TDX that needs couple of paravirt calls that were hidden underCONFIG_PARAVIRT_XXL, but the rest of the config would be a bloat for TDX.
Used how? Why is it bloat for TDX?
I'm sure that'll become clear in the remainder of the patches but you should state it here so that it is clear why you're doing what you're doing.
Quoted 37 lines
Signed-off-by Kirill A. Shutemov <[email protected]>Reviewed-by Andi Kleen <[email protected]>Reviewed-by Tony Luck <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>--- arch/x86/Kconfig | 4 +++ arch/x86/boot/compressed/misc.h | 1 + arch/x86/include/asm/irqflags.h | 38 +++++++++++++++------------ arch/x86/include/asm/paravirt.h | 22 +++++++++------- arch/x86/include/asm/paravirt_types.h | 3 ++- arch/x86/kernel/paravirt.c | 4 ++- arch/x86/mm/mem_encrypt_identity.c | 1 + 7 files changed, 44 insertions(+), 29 deletions(-)arch/x86/Kconfig +4
-0@@ -783,8 +783,12 @@ config PARAVIRT over full virtualization. However, when run without a hypervisor the kernel is theoretically slower and slightly larger. +config PARAVIRT_XL + bool + config PARAVIRT_XXL bool + select PARAVIRT_XL config PARAVIRT_DEBUG bool "paravirt-ops debugging"arch/x86/boot/compressed/misc.h +0
-0
arch/x86/boot/compressed/misc.h line 13
Quoted 5 lines
arch/x86/boot/compressed/misc.h +1
-0@@ -9,6 +9,7 @@* paravirt and debugging variants are added.) */ #undef CONFIG_PARAVIRT +#undef CONFIG_PARAVIRT_XL#undef CONFIG_PARAVIRT_XXL
So what happens if someone else needs even less pv and defines CONFIG_PARAVIRT_L. Or _M? Or _S?
Are we going to teleport into a clothing store each time we look at paravirt now? :)
So before this goes out of hand let's define explicitly, pls, what XXL means and XL. And rename them. They could be called PARAVIRT_FULL and PARAVIRT_HLT as apparently that thing is exposing only the PV ops related to HLT.
Or something to that effect.
Dunno, maybe Jürgen has a better idea, leaving in the rest quoted for him.
Thx.
Quoted 148 lines
arch/x86/boot/compressed/misc.h +0
-0#undef CONFIG_PARAVIRT_SPINLOCKS#undef CONFIG_KASANarch/x86/include/asm/irqflags.h +21
-17@@ -59,27 +59,11 @@ static inline __cpuidle void native_halt(void)#endif -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_XL #include <asm/paravirt.h> #else #ifndef __ASSEMBLY__#include <linux/types.h> - -static __always_inline unsigned long arch_local_save_flags (void ) -{ -return native_save_fl (); -} - -static __always_inline void arch_local_irq_disable(void) -{ -native_irq_disable (); -} - -static __always_inline void arch_local_irq_enable(void) -{ -native_irq_enable (); -} -/* * Used in the idle loop; sti takes one instruction cycle * to complete: @@ -97,6 +81,26 @@ static inline __cpuidle void halt(void) {native_halt (); } +#endif /* !__ASSEMBLY__ */ +#endif /* CONFIG_PARAVIRT_XL */ + +#ifndef CONFIG_PARAVIRT_XXL +#ifndef __ASSEMBLY__ + +static __always_inline unsigned long arch_local_save_flags (void ) +{ +return native_save_fl (); +} + +static __always_inline void arch_local_irq_disable(void) +{ +native_irq_disable (); +} + +static __always_inline void arch_local_irq_enable(void) +{ +native_irq_enable (); +}/* * For spinlocks, etc: arch/x86/include/asm/paravirt.h +12
-10@@ -84,6 +84,18 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct *mm)PVOP_VCALL1 (mmu.exit_mmap, mm); } +#ifdef CONFIG_PARAVIRT_XL +static inline void arch_safe_halt (void ) +{ +PVOP_VCALL0 (irq.safe_halt); +} + +static inline void halt (void ) +{ +PVOP_VCALL0 (irq.halt); +} +#endif +#ifdef CONFIG_PARAVIRT_XXL static inline void load_sp0 (unsigned long sp0) {@@ -145,16 +157,6 @@ static inline void __write_cr4(unsigned long x)PVOP_VCALL1 (cpu.write_cr4, x); } -static inline void arch_safe_halt (void ) -{ -PVOP_VCALL0 (irq.safe_halt); -} - -static inline void halt (void ) -{ -PVOP_VCALL0 (irq.halt); -} -static inline void wbinvd (void ) {PVOP_VCALL0 (cpu.wbinvd);arch/x86/include/asm/paravirt_types.h +2
-1@@ -177,7 +177,8 @@ struct pv_irq_ops {struct paravirt_callee_save save_fl;struct paravirt_callee_save irq_disable;struct paravirt_callee_save irq_enable; - +#endif +#ifdef CONFIG_PARAVIRT_XL void (*safe_halt)(void );void (*halt)(void );#endif arch/x86/kernel/paravirt.c +3
-1@@ -322,9 +322,11 @@ struct paravirt_patch_template pv_ops = { .irq.save_fl = __PV_IS_CALLEE_SAVE(native_save_fl), .irq.irq_disable = __PV_IS_CALLEE_SAVE(native_irq_disable), .irq.irq_enable = __PV_IS_CALLEE_SAVE(native_irq_enable), +#endif /* CONFIG_PARAVIRT_XXL */ +#ifdef CONFIG_PARAVIRT_XL .irq.safe_halt = native_safe_halt, .irq.halt = native_halt, -#endif /* CONFIG_PARAVIRT_XXL */ +#endif /* CONFIG_PARAVIRT_XL */ /* Mmu ops. */ .mmu.flush_tlb_user = native_flush_tlb_local,arch/x86/mm/mem_encrypt_identity.c +1
-0@@ -24,6 +24,7 @@* be extended when new paravirt and debugging variants are added.) */ #undef CONFIG_PARAVIRT +#undef CONFIG_PARAVIRT_XL#undef CONFIG_PARAVIRT_XXL#undef CONFIG_PARAVIRT_SPINLOCKSSignature · 1 line
2.25.1
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [RFC v2 05/32] x86/tdx: Add __tdcall() and __tdvmcall() helper functions
In reply to Dave HansenHi Dave,
Quoted 11 lines · Dave Hansen
Quoted 3 lines
Do we need to rename the helper functions ?
tdvmcall(),tdvmcall_out_r11()Yes.
Quoted 4 lines
Also what about output structs?
struct
tdcall_output
structtdvmcall_outputYes, they need sane, straightforward names which are not confusing too.
Following is the rename diff. Please let me know if you agree with the names used.
arch/x86/include/asm/tdx.h +0 -0
-struct tdcall_output {
+struct tdx_module_output {
u64 rcx;
u64 rdx;
u64 r8;
@@ -19,7 +19,7 @@ struct tdcall_output {
u64 r11;};
-struct tdvmcall_output {
+struct tdx_hypercall_output {
u64 r11;
u64 r12;
u64 r13;@@ -33,12 +33,12 @@ bool is_tdx_guest(void);
void __init tdx_early_init(void);
/* Helper function used to communicate with the TDX module */
-u64 __tdcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
- struct tdcall_output *out);
+u64 __tdx_module_call(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
+ struct tdx_module_output *out);
/* Helper function used to request services from VMM */
-u64 __tdvmcall(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15,
- struct tdvmcall_output *out);
+u64 __tdx_hypercall(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15,
+ struct tdx_hypercall_output *out);
arch/x86/kernel/tdx.c +4 -4
@@ -8,11 +8,11 @@/* * Wrapper for use case that checks for error code and print warning message. */ -static inline u64 tdvmcall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +static inline u64 tdx_hypercall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) {u64 err; - err =__tdvmcall (fn, r12, r13, r14, r15,NULL ); + err =__tdx_hypercall (fn, r12, r13, r14, r15,NULL );if (err) pr_warn_ratelimited("TDVMCALL fn:%llx failed with err:%llx\n",
@@ -24,13 +24,14 @@ static inline u64 tdvmcall(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15)/* * Wrapper for the semi-common case where we need single output value (R11). */ -static inline u64 tdvmcall_out_r11 (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +static inline u64 tdx_hypercall_out_r11 (u64 fn,u64 r12,u64 r13, +u64 r14,u64 r15) --
Sathyanarayanan Kuppuswamy
Linux Kernel Developer
Re: [RFC v2 05/32] x86/tdx: Add __tdcall() and __tdvmcall() helper functions
Quoted 2 lines · Sathyanarayanan
Following is the rename diff. Please let me know if you agree with the names used.
Look fine at a glance, but the real key is how they look when they get used.
PATCH 1/1 x86/tdx: Add __tdx_module_call() and __tdx_hypercall() helper functions
Guests communicate with VMMs with hypercalls. Historically, these are implemented using instructions that are known to cause VMEXITs like vmcall, vmlaunch, etc. However, with TDX, VMEXITs no longer expose guest state to the host. This prevents the old hypercall mechanisms from working. So to communicate with VMM, TDX specification defines a new instruction called "tdcall".
In TDX based VM, since VMM is an untrusted entity, a intermediary layer (TDX module) exists between host and guest to facilitate the secure communication. And "tdcall" instruction is used by the guest to request services from TDX module. And a variant of "tdcall" instruction (with specific arguments as defined by GHCI) is used by the guest to request services from VMM via the TDX module.
Implement common helper functions to communicate with the TDX Module and VMM (using TDCALL instruction).__tdx_hypercall() - function can be used to request services from
the VMM.
__tdx_module_call() - function can be used to communicate with the
TDX Module.
Also define two additional wrappers, tdx_hypercall() and tdx_hypercall_out_r11() to cover common use cases of __tdx_hypercall() function. Since each use case of __tdx_module_call() is different, we don't need such wrappers for it.
Implement __tdx_module_call() and __tdx_hypercall() helper functions in assembly.
Rationale behind choosing to use assembly over inline assembly are,
1. Since the number of lines of instructions (with comments) in __tdx_hypercall() implementation is over 70, using inline assembly to implement it will make it hard to read.
2. Also, since many registers (R8-R15, R[A-D]X)) will be used in TDCALL operation, if all these registers are included in in-line assembly constraints, some of the older compilers may not be able to meet this requirement.
Also, just like syscalls, not all TDVMCALL/TDCALLs use cases need to use the same set of argument registers. The implementation here picks the current worst-case scenario for TDCALL (4 registers). For TDCALLs with fewer than 4 arguments, there will end up being a few superfluous (cheap) instructions. But, this approach maximizes code reuse. The same argument applies to __tdx_hypercall() function as well.
Current implementation of __tdx_hypercall() includes error handling (ud2 on failure case) in assembly function instead of doing it in C wrapper function. The reason behind this choice is, when adding support for in/out instructions (refer to patch titled "x86/tdx: Handle port I/O" in this series), we use alternative_io() to substitute in/out instruction with __tdx_hypercall() calls. So use of C wrappers is not trivial in this case because the input parameters will be in the wrong registers and it's tricky to include proper buffer code to make this happen.
For registers used by TDCALL instruction, please check TDX GHCI specification, sec 2.4 and 3.
---
Hi Dave,
It includes all fixes suggested by you. Please let me know your comments.
Changes since v1:
* Renamed __tdcall()/__tdvmcall() to __tdx_module_call()/__tdx_hypercall().
* Used BIT() to derive TDVMCALL_EXPOSE_REGS_MASK.
* Removed unnecessary code in __tdcall() function.
* Fixed comments as per Dave's review.
arch/x86/include/asm/tdx.h | 26 ++++ arch/x86/kernel/Makefile | 2 +- arch/x86/kernel/asm-offsets.c | 22 ++++ arch/x86/kernel/tdcall.S | 215 ++++++++++++++++++++++++++++++++++ arch/x86/kernel/tdx.c | 39 ++++++ 5 files changed, 303 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kernel/tdcall.S
arch/x86/include/asm/tdx.h +26 -0
@@ -8,12 +8,38 @@#ifdef CONFIG_INTEL_TDX_GUEST #include <asm/cpufeature.h> +#include <linux/types.h> + +struct tdx_module_output { +u64 rcx; +u64 rdx; +u64 r8; +u64 r9; +u64 r10; +u64 r11; +}; + +struct tdx_hypercall_output { +u64 r11; +u64 r12; +u64 r13; +u64 r14; +u64 r15; +};/* Common API to check TDX support in decompression and common kernel code. */ bool is_tdx_guest (void );void __init tdx_early_init (void ); +/* Helper function used to communicate with the TDX module */ +u64 __tdx_module_call (u64 fn,u64 rcx,u64 rdx,u64 r8,u64 r9, +struct tdx_module_output *out); + +/* Helper function used to request services from VMM */ +u64 __tdx_hypercall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15, +struct tdx_hypercall_output *out); +#else // !CONFIG_INTEL_TDX_GUEST static inline bool is_tdx_guest (void )
arch/x86/kernel/Makefile +1 -1
@@ -127,7 +127,7 @@ obj-$(CONFIG_PARAVIRT_CLOCK) += pvclock.o obj-$(CONFIG_X86_PMEM_LEGACY_DEVICE) += pmem.o obj-$(CONFIG_JAILHOUSE_GUEST) += jailhouse.o -obj-$(CONFIG_INTEL_TDX_GUEST) += tdx.o +obj-$(CONFIG_INTEL_TDX_GUEST) += tdcall.o tdx.o obj-$(CONFIG_EISA) += eisa.o obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
arch/x86/kernel/asm-offsets.c +22 -0
@@ -23,6 +23,10 @@#include <xen/interface/xen.h> #endif +#ifdef CONFIG_INTEL_TDX_GUEST +#include <asm/tdx.h> +#endif +#ifdef CONFIG_X86_32 # include "asm-offsets_32.c" #else
@@ -75,6 +79,24 @@ static void __used common(void)OFFSET (XEN_vcpu_info_arch_cr2, vcpu_info, arch.cr2);#endif +#ifdef CONFIG_INTEL_TDX_GUEST +BLANK (); +/* Offset for fields in tdcall_output */ +OFFSET (TDX_MODULE_rcx, tdx_module_output, rcx); +OFFSET (TDX_MODULE_rdx, tdx_module_output, rdx); +OFFSET (TDX_MODULE_r8, tdx_module_output, r8); +OFFSET (TDX_MODULE_r9, tdx_module_output, r9); +OFFSET (TDX_MODULE_r10, tdx_module_output, r10); +OFFSET (TDX_MODULE_r11, tdx_module_output, r11); + +/* Offset for fields in tdvmcall_output */ +OFFSET (TDX_HYPERCALL_r11, tdx_hypercall_output, r11); +OFFSET (TDX_HYPERCALL_r12, tdx_hypercall_output, r12); +OFFSET (TDX_HYPERCALL_r13, tdx_hypercall_output, r13); +OFFSET (TDX_HYPERCALL_r14, tdx_hypercall_output, r14); +OFFSET (TDX_HYPERCALL_r15, tdx_hypercall_output, r15); +#endif +BLANK ();OFFSET (BP_scratch, boot_params, scratch);OFFSET (BP_secure_boot, boot_params, secure_boot);
arch/x86/kernel/tdcall.S new file +215 -0
215 lines not on this page · raw message
arch/x86/kernel/tdx.c +39 -0
@@ -1,8 +1,47 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (C) 2020 Intel Corporation */ +#define pr_fmt (fmt) "TDX: " fmt +#include <asm/tdx.h> +/* +* Wrapper for simple hypercalls that only return a success/error code. +*/ +static inline u64 tdx_hypercall (u64 fn,u64 r12,u64 r13,u64 r14,u64 r15) +{ +u64 err; + + err =__tdx_hypercall (fn, r12, r13, r14, r15,NULL ); + +if (err) +pr_warn_ratelimited ("TDVMCALL fn:%llx failed with err:%llx\n" , + fn, err); + +return err; +} + +/* +* Wrapper for the semi-common case where we need single output +* value (R11). Callers of this function does not care about the +* hypercall error code (mainly for IN or MMIO usecase). +*/ +static inline u64 tdx_hypercall_out_r11 (u64 fn,u64 r12,u64 r13, +u64 r14,u64 r15) +{ + +struct tdx_hypercall_output out = {0 }; +u64 err; + + err =__tdx_hypercall (fn, r12, r13, r14, r15, &out); + +if (err) +pr_warn_ratelimited ("TDVMCALL fn:%llx failed with err:%llx\n" , + fn, err); + +return out.r11; +} +static inline bool cpuid_has_tdx_guest (void ) {u32 eax, signature[3 ];
Signature · 1 line
2.25.1
Hi Peter/Andy,
Quoted 1 line · Kuppuswamy Sathyanarayanan
Hi All,
Just a gentle ping. Please let me know your comments on this patch set. I hope it addressed concerns raised by you in RFC v1.
Quoted 164 lines
NOTE: This series is not ready for wide public review. It is being specifically posted so that Peter Z and other experts on the entry code can look for problems with the new exception handler (#VE). That's also why x86@ is not being spammed.
Intel's Trust Domain Extensions (TDX) protect guest VMs from malicious hosts and some physical attacks. This series adds the bare-minimum support to run a TDX guest. The host-side support will be submitted separately. Also support for advanced TD guest features like attestation or debug-mode will be submitted separately. Also, at this point it is not secure with some known holes in drivers, and also hasn’t been fully audited and fuzzed yet.
TDX has a lot of similarities to SEV. It enhances confidentiality and of guest memory and state (like registers) and includes a new exception (#VE) for the same basic reasons as SEV-ES. Like SEV-SNP (not merged yet), TDX limits the host's ability to effect changes in the guest physical address space.
In contrast to the SEV code in the kernel, TDX guest memory is integrity protected and isolated; the host is prevented from accessing guest memory (even ciphertext).
The TDX architecture also includes a new CPU mode called Secure-Arbitration Mode (SEAM). The software (TDX module) running in this mode arbitrates interactions between host and guest and implements many of the guarantees of the TDX architecture.
Some of the key differences between TD and regular VM is,
1. Multi CPU bring-up is done using the ACPI MADT wake-up table.
2. A new #VE exception handler is added. The TDX module injects #VE exception to the guest TD in cases of instructions that need to be emulated, disallowed MSR accesses, subset of CPUID leaves, etc.
3. By default memory is marked as private, and TD will selectively share it with VMM based on need.
4. Remote attestation is supported to enable a third party (either the owner of the workload or a user of the services provided by the workload) to establish that the workload is running on an Intel-TDX-enabled platform located within a TD prior to providing that workload data.You can find TDX related documents in the following link.
https://software.intel.com/content/www/br/pt/develop/articles/intel-trust-domain-extensions.html
Changes since v1:
* Implementedtdcall()andtdvmcall()helper functions in assembly and renamed them as__tdcall()and__tdvmcall().
* Addeddo_general_protection()helper function to re-use protection code between #GP exception and TDX #VE exception handlers.
* Addressed syscall gap issue in #VE handler support (for details check the commit log in "x86/traps: Add #VE support for TDX guest").
* Modified patch titled "x86/tdx: Handle port I/O" to re-use commontdvmcall()helper function.
* Added error handling support to MADT CPU wakeup code.
* Introduced enumtdx_map_typeto identify SHARED vs PRIVATE memory type.
* Enabled shared memory in IOAPIC driver.
* Added BINUTILS version info for TDCALL.
* Changed the TDVMCALL vendor id from 0 to "TDX.KVM".
* ReplacedWARN()withpr_warn_ratelimited()in__tdvmcall()wrappers.
* Fixed commit log and code comments related review comments.
* Renamed patch titled # "x86/topology: Disable CPU hotplug support for TDX platforms" to "x86/topology: Disable CPU online/offline control for TDX guest"
* Rebased on top of v5.12 kernel.Erik Kaneda (1):
ACPICA: ACPI 6.4: MADT: add Multiprocessor Wakeup StructureIsaku Yamahata (1):
x86/tdx: ioapic: Add shared bit for IOAPIC base addressKirill A. Shutemov (16):
x86/paravirt: IntroduceCONFIG_PARAVIRT_XLx86/tdx: Get TD execution environment information via TDINFO x86/traps: Add #VE support for TDX guest
x86/tdx: Add HLT support for TDX guest
x86/tdx: Wire up KVM hypercalls
x86/tdx: Add MSR support for TDX guest
x86/tdx: Handle CPUID via #VE
x86/io: Allow to overrideinX()andoutX()implementation x86/tdx: Handle port I/O
x86/tdx: Handle in-kernel MMIO
x86/mm: Moveforce_dma_unencrypted()to common code x86/tdx: Exclude Shared bit from__PHYSICAL_MASKx86/tdx: Make pages shared inioremap()
x86/tdx: Add helper to do MapGPA TDVMALL
x86/tdx: Make DMA pages shared
x86/kvm: Use bounce buffers for TD guestKuppuswamy Sathyanarayanan (10):
x86/tdx: IntroduceINTEL_TDX_GUESTconfig option x86/cpufeatures: Add TDX Guest CPU feature
x86/x86: Addis_tdx_guest()interface
x86/tdx: Add__tdcall()and__tdvmcall()helper functions x86/traps: Add do_general_protection() helper function x86/tdx: Handle MWAIT, MONITOR and WBINVD
ACPICA: ACPI 6.4: MADT: add Multiprocessor Wakeup Mailbox Structure ACPI/table: Print MADT Wake table information
x86/acpi, x86/boot: Add multiprocessor wake-up support x86/topology: Disable CPU online/offline control for TDX guestSean Christopherson (4):
x86/boot: Add a trampoline for APs booting in 64-bit mode x86/boot: Avoid #VE during compressed boot for TDX platforms x86/boot: Avoid unnecessary #VE during boot process x86/tdx: Forcefully disable legacy PIC for TDX guestsarch/x86/Kconfig | 28 +- arch/x86/boot/compressed/Makefile | 2 + arch/x86/boot/compressed/head_64.S | 10 +- arch/x86/boot/compressed/misc.h | 1 + arch/x86/boot/compressed/pgtable.h | 2 +- arch/x86/boot/compressed/tdcall.S | 9 + arch/x86/boot/compressed/tdx.c | 32 ++ arch/x86/include/asm/apic.h | 3 + arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/idtentry.h | 4 + arch/x86/include/asm/io.h | 24 +- arch/x86/include/asm/irqflags.h | 38 +- arch/x86/include/asm/kvm_para.h | 21 + arch/x86/include/asm/paravirt.h | 22 +- arch/x86/include/asm/paravirt_types.h | 3 +- arch/x86/include/asm/pgtable.h | 3 + arch/x86/include/asm/realmode.h | 1 + arch/x86/include/asm/tdx.h | 176 +++++++++ arch/x86/kernel/Makefile | 1 + arch/x86/kernel/acpi/boot.c | 79 ++++ arch/x86/kernel/apic/apic.c | 8 + arch/x86/kernel/apic/io_apic.c | 12 +- arch/x86/kernel/asm-offsets.c | 22 ++ arch/x86/kernel/head64.c | 3 + arch/x86/kernel/head_64.S | 13 +- arch/x86/kernel/idt.c | 6 + arch/x86/kernel/paravirt.c | 4 +- arch/x86/kernel/pci-swiotlb.c | 2 +- arch/x86/kernel/smpboot.c | 5 + arch/x86/kernel/tdcall.S | 361 +++++++++++++++++ arch/x86/kernel/tdx-kvm.c | 45 +++ arch/x86/kernel/tdx.c | 480 +++++++++++++++++++++++ arch/x86/kernel/topology.c | 3 +- arch/x86/kernel/traps.c | 81 ++-- arch/x86/mm/Makefile | 2 + arch/x86/mm/ioremap.c | 8 +- arch/x86/mm/mem_encrypt.c | 75 ---- arch/x86/mm/mem_encrypt_common.c | 85 ++++ arch/x86/mm/mem_encrypt_identity.c | 1 + arch/x86/mm/pat/set_memory.c | 48 ++- arch/x86/realmode/rm/header.S | 1 + arch/x86/realmode/rm/trampoline_64.S | 49 ++- arch/x86/realmode/rm/trampoline_common.S | 5 +- drivers/acpi/tables.c | 11 + include/acpi/actbl2.h | 26 +- 45 files changed, 1654 insertions(+), 162 deletions(-)create mode 100644
arch/x86/boot/compressed/tdcall.Screate mode 100644arch/x86/boot/compressed/tdx.ccreate mode 100644arch/x86/include/asm/tdx.hcreate mode 100644arch/x86/kernel/tdcall.Screate mode 100644arch/x86/kernel/tdx-kvm.ccreate mode 100644arch/x86/kernel/tdx.ccreate mode 100644arch/x86/mm/mem_encrypt_common.c
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 01/32] x86/paravirt: Introduce CONFIG_PARAVIRT_XL
In reply to Borislav PetkovQuoted 1 line · Borislav Petkov
Or something to that effect.
See the couple of attached patches. Does look along the lines you wanted?
The first one renames PARAVIRT_XXL and the second one introduces PARAVIRT_HLT.
Signature · 1 line
Kirill A. Shutemov
From 2671a044687da0e0c7105beb3467a270b8863a1b Mon Sep 17 00:00:00 2001
From: "Kirill A. Shutemov" <[email protected]>
Date: Thu, 6 May 2021 17:04:42 +0300
Subject: [PATCH 1/2] x86/paravirt: Rename PARAVIRT_XXL to PARAVIRT_FULL
PARAVIRT_XXL provides a way to hook up a full set paravirt ops. Rename it to PARAVIRT_FULL to be more self-descriptive.
It's a preparation for the next patch.
--- arch/x86/Kconfig | 2 +- arch/x86/boot/compressed/misc.h | 2 +- arch/x86/entry/vdso/vdso32/vclock_gettime.c | 2 +- arch/x86/include/asm/debugreg.h | 2 +- arch/x86/include/asm/desc.h | 4 ++-- arch/x86/include/asm/fixmap.h | 4 ++-- arch/x86/include/asm/io_bitmap.h | 2 +- arch/x86/include/asm/irqflags.h | 4 ++-- arch/x86/include/asm/mmu_context.h | 4 ++-- arch/x86/include/asm/msr.h | 4 ++-- arch/x86/include/asm/paravirt.h | 12 ++++++------ arch/x86/include/asm/paravirt_types.h | 10 +++++----- arch/x86/include/asm/pgalloc.h | 2 +- arch/x86/include/asm/pgtable.h | 6 +++--- arch/x86/include/asm/processor.h | 4 ++-- arch/x86/include/asm/ptrace.h | 2 +- arch/x86/include/asm/required-features.h | 2 +- arch/x86/include/asm/special_insns.h | 4 ++-- arch/x86/kernel/asm-offsets.c | 2 +- arch/x86/kernel/asm-offsets_64.c | 2 +- arch/x86/kernel/paravirt.c | 18 +++++++++--------- arch/x86/kernel/paravirt_patch.c | 6 +++--- arch/x86/mm/mem_encrypt_identity.c | 2 +- arch/x86/xen/Kconfig | 2 +- tools/arch/x86/include/asm/required-features.h | 2 +- 25 files changed, 53 insertions(+), 53 deletions(-)
arch/x86/Kconfig +1 -1
@@ -783,7 +783,7 @@ config PARAVIRT over full virtualization. However, when run without a hypervisor the kernel is theoretically slower and slightly larger. -config PARAVIRT_XXL +config PARAVIRT_FULL bool config PARAVIRT_DEBUG
arch/x86/boot/compressed/misc.h +1 -1
@@ -9,7 +9,7 @@* paravirt and debugging variants are added.) */ #undef CONFIG_PARAVIRT -#undef CONFIG_PARAVIRT_XXL +#undef CONFIG_PARAVIRT_FULL#undef CONFIG_PARAVIRT_SPINLOCKS#undef CONFIG_KASAN#undef CONFIG_KASAN_GENERIC
arch/x86/entry/vdso/vdso32/vclock_gettime.c +1 -1
@@ -14,7 +14,7 @@#undef CONFIG_ILLEGAL_POINTER_VALUE#undef CONFIG_SPARSEMEM_VMEMMAP#undef CONFIG_NR_CPUS -#undef CONFIG_PARAVIRT_XXL +#undef CONFIG_PARAVIRT_FULL#define CONFIG_X86_32 1#define CONFIG_PGTABLE_LEVELS 2
arch/x86/include/asm/debugreg.h +1 -1
@@ -8,7 +8,7 @@ DECLARE_PER_CPU(unsigned long, cpu_dr7); -#ifndef CONFIG_PARAVIRT_XXL +#ifndef CONFIG_PARAVIRT_FULL /* * These special macros can be used to get or set a debugging register */
arch/x86/include/asm/desc.h +2 -2
@@ -103,7 +103,7 @@ static inline int desc_empty(const void *ptr)return !(desc[0 ] | desc[1 ]); } -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #include <asm/paravirt.h> #else #define load_TR_desc () native_load_tr_desc()
@@ -129,7 +129,7 @@ static inline void paravirt_alloc_ldt(struct desc_struct *ldt, unsigned entries)static inline void paravirt_free_ldt (struct desc_struct *ldt,unsigned entries) { } -#endif /* CONFIG_PARAVIRT_XXL */ +#endif /* CONFIG_PARAVIRT_FULL */ #define store_ldt (ldt) asm("sldt %0" : "=m"(ldt))
arch/x86/include/asm/fixmap.h +2 -2
@@ -105,7 +105,7 @@ enum fixed_addresses { FIX_PCIE_MCFG,#endif #endif -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL FIX_PARAVIRT_BOOTMAP,#endif
@@ -160,7 +160,7 @@ void __native_set_fixmap(enum fixed_addresses idx, pte_t pte);void native_set_fixmap (unsigned /* enum fixed_addresses */ idx,phys_addr_t phys,pgprot_t flags); -#ifndef CONFIG_PARAVIRT_XXL +#ifndef CONFIG_PARAVIRT_FULL static inline void __set_fixmap (enum fixed_addresses idx,phys_addr_t phys,pgprot_t flags) {
arch/x86/include/asm/io_bitmap.h +1 -1
@@ -36,7 +36,7 @@ static inline void native_tss_invalidate_io_bitmap(void)void native_tss_update_io_bitmap (void ); -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #include <asm/paravirt.h> #else #define tss_update_io_bitmap native_tss_update_io_bitmap
arch/x86/include/asm/irqflags.h +2 -2
@@ -59,7 +59,7 @@ static inline __cpuidle void native_halt(void)#endif -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #include <asm/paravirt.h> #else #ifndef __ASSEMBLY__
@@ -124,7 +124,7 @@ static __always_inline unsigned long arch_local_irq_save(void)#endif #endif /* __ASSEMBLY__ */ -#endif /* CONFIG_PARAVIRT_XXL */ +#endif /* CONFIG_PARAVIRT_FULL */ #ifndef __ASSEMBLY__ static __always_inline int arch_irqs_disabled_flags(unsigned long flags)
arch/x86/include/asm/mmu_context.h +2 -2
@@ -15,12 +15,12 @@extern atomic64_t last_mm_ctx_id; -#ifndef CONFIG_PARAVIRT_XXL +#ifndef CONFIG_PARAVIRT_FULL static inline void paravirt_activate_mm (struct mm_struct *prev,struct mm_struct *next) { } -#endif /* !CONFIG_PARAVIRT_XXL */ +#endif /* !CONFIG_PARAVIRT_FULL */ #ifdef CONFIG_PERF_EVENTS DECLARE_STATIC_KEY_FALSE (rdpmc_never_available_key);
arch/x86/include/asm/msr.h +2 -2
@@ -251,7 +251,7 @@ static inline unsigned long long native_read_pmc(int counter)return EAX_EDX_VAL (val, low, high); } -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #include <asm/paravirt.h> #else #include <linux/errno.h>
@@ -314,7 +314,7 @@ do { \#define rdpmcl (counter, val) ((val) = native_read_pmc(counter)) -#endif /* !CONFIG_PARAVIRT_XXL */ +#endif /* !CONFIG_PARAVIRT_FULL */ /* * 64-bit version of wrmsr_safe():
arch/x86/include/asm/paravirt.h +6 -6
@@ -84,7 +84,7 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct *mm)PVOP_VCALL1 (mmu.exit_mmap, mm); } -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL static inline void load_sp0 (unsigned long sp0) {PVOP_VCALL1 (cpu.load_sp0, sp0);
@@ -642,7 +642,7 @@ bool __raw_callee_save___native_vcpu_is_preempted(long cpu);#define __PV_IS_CALLEE_SAVE (func) \ ((struct paravirt_callee_save) { func }) -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL static inline notrace unsigned long arch_local_save_flags (void ) { return PVOP_CALLEE0(unsigned long, irq.save_fl);
@@ -748,7 +748,7 @@ extern void default_banner(void);#define PARA_INDIRECT (addr) *%cs:addr#endif -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #define INTERRUPT_RETURN \ PARA_SITE(PARA_PATCH(PV_CPU_iret), \ ANNOTATE_RETPOLINE_SAFE; \
@@ -770,7 +770,7 @@ extern void default_banner(void);#endif #ifdef CONFIG_X86_64 -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #ifdef CONFIG_DEBUG_ENTRY #define SAVE_FLAGS (clobbers) \ PARA_SITE(PARA_PATCH(PV_IRQ_save_fl), \
@@ -779,7 +779,7 @@ extern void default_banner(void); call PARA_INDIRECT(pv_ops+PV_IRQ_save_fl); \ PV_RESTORE_REGS(clobbers | CLBR_CALLEE_SAVE);)#endif -#endif /* CONFIG_PARAVIRT_XXL */ +#endif /* CONFIG_PARAVIRT_FULL */ #endif /* CONFIG_X86_64 */ #endif /* __ASSEMBLY__ */
@@ -788,7 +788,7 @@ extern void default_banner(void);#endif /* !CONFIG_PARAVIRT */ #ifndef __ASSEMBLY__ -#ifndef CONFIG_PARAVIRT_XXL +#ifndef CONFIG_PARAVIRT_FULL static inline void paravirt_arch_dup_mmap(struct mm_struct *oldmm, struct mm_struct *mm) {
arch/x86/include/asm/paravirt_types.h +5 -5
@@ -66,7 +66,7 @@ struct paravirt_callee_save {/* general info */ struct pv_info { -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL u16 extra_user_64bit_cs;/* __USER_CS if none */ #endif
@@ -86,7 +86,7 @@ struct pv_init_ops { unsigned long addr, unsigned len); } __no_randomize_layout; -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL struct pv_lazy_ops { /* Set deferred update mode, used for batching operations. */ void (*enter)(void);
@@ -104,7 +104,7 @@ struct pv_cpu_ops {/* hooks for various privileged instructions */ void (*io_delay)(void ); -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL unsigned long (*get_debugreg)(int regno);void (*set_debugreg)(int regno,unsigned long value);
@@ -166,7 +166,7 @@ struct pv_cpu_ops { } __no_randomize_layout;struct pv_irq_ops { -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL /* * Get/set interrupt state. save_fl is expected to use X86_EFLAGS_IF; * all other bits returned from save_fl are undefined.
@@ -196,7 +196,7 @@ struct pv_mmu_ops {/* Hook for intercepting the destruction of an mm_struct. */ void (*exit_mmap)(struct mm_struct *mm); -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL struct paravirt_callee_save read_cr2;void (*write_cr2)(unsigned long );
arch/x86/include/asm/pgalloc.h +1 -1
@@ -12,7 +12,7 @@static inline int __paravirt_pgd_alloc (struct mm_struct *mm) {return 0 ; } -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #include <asm/paravirt.h> #else #define paravirt_pgd_alloc (mm) __paravirt_pgd_alloc(mm)
arch/x86/include/asm/pgtable.h +3 -3
@@ -59,9 +59,9 @@ extern struct mm_struct *pgd_page_get_mm(struct page *page);extern pmdval_t early_pmd_flags; -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #include <asm/paravirt.h> -#else /* !CONFIG_PARAVIRT_XXL */ +#else /* !CONFIG_PARAVIRT_FULL */ #define set_pte (ptep, pte) native_set_pte(ptep, pte)#define set_pte_atomic (ptep, pte) \
@@ -115,7 +115,7 @@ extern pmdval_t early_pmd_flags;#define __pte (x) native_make_pte(x)#define arch_end_context_switch (prev) do {} while(0) -#endif /* CONFIG_PARAVIRT_XXL */ +#endif /* CONFIG_PARAVIRT_FULL */ /* * The following only work if pte_present() is true.
arch/x86/include/asm/processor.h +2 -2
@@ -580,7 +580,7 @@ static inline bool on_thread_stack(void) current_stack_pointer) < THREAD_SIZE; } -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #include <asm/paravirt.h> #else #define __cpuid native_cpuid
@@ -590,7 +590,7 @@ static inline void load_sp0(unsigned long sp0) native_load_sp0(sp0); } -#endif /* CONFIG_PARAVIRT_XXL */ +#endif /* CONFIG_PARAVIRT_FULL */ /* Free all resources held by a thread. */ extern void release_thread (struct task_struct *);
arch/x86/include/asm/ptrace.h +1 -1
@@ -146,7 +146,7 @@ static inline int v8086_mode(struct pt_regs *regs)static inline bool user_64bit_mode (struct pt_regs *regs) {#ifdef CONFIG_X86_64 -#ifndef CONFIG_PARAVIRT_XXL +#ifndef CONFIG_PARAVIRT_FULL /* * On non-paravirt systems, this is the only long mode CPL 3 * selector. We do not allow long mode selectors in the LDT.
arch/x86/include/asm/required-features.h +1 -1
@@ -54,7 +54,7 @@#endif #ifdef CONFIG_X86_64 -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL /* Paravirtualized systems may not have PSE or PGE available */ #define NEED_PSE 0#define NEED_PGE 0
arch/x86/include/asm/special_insns.h +2 -2
@@ -148,7 +148,7 @@ static inline unsigned long __read_cr4(void)return native_read_cr4 (); } -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #include <asm/paravirt.h> #else
@@ -205,7 +205,7 @@ static inline void load_gs_index(unsigned int selector)#endif -#endif /* CONFIG_PARAVIRT_XXL */ +#endif /* CONFIG_PARAVIRT_FULL */ static inline void clflush (volatile void *__p) {
arch/x86/kernel/asm-offsets.c +1 -1
@@ -61,7 +61,7 @@ static void __used common(void)OFFSET (IA32_RT_SIGFRAME_sigcontext, rt_sigframe_ia32, uc.uc_mcontext);#endif -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL BLANK ();OFFSET (PV_IRQ_irq_disable, paravirt_patch_template, irq.irq_disable);OFFSET (PV_IRQ_irq_enable, paravirt_patch_template, irq.irq_enable);
arch/x86/kernel/asm-offsets_64.c +1 -1
@@ -12,7 +12,7 @@int main (void ) {#ifdef CONFIG_PARAVIRT -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL #ifdef CONFIG_DEBUG_ENTRY OFFSET (PV_IRQ_save_fl, paravirt_patch_template, irq.save_fl);#endif
arch/x86/kernel/paravirt.c +9 -9
@@ -79,7 +79,7 @@ static unsigned paravirt_patch_call(void *insn_buff, const void *target,return call_len; } -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL /* identity function, which can be inlined */ u64 notrace _paravirt_ident_64(u64 x) {
@@ -130,7 +130,7 @@ unsigned paravirt_patch_default(u8 type, void *insn_buff,else if (opfunc == _paravirt_nop) ret =0 ; -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL /* identity functions just return their single argument */ else if (opfunc == _paravirt_ident_64) ret =paravirt_patch_ident_64 (insn_buff, len);
@@ -227,7 +227,7 @@ void paravirt_flush_lazy_mmu(void)preempt_enable (); } -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL void paravirt_start_context_switch (struct task_struct *prev) {BUG_ON (preemptible ());
@@ -260,7 +260,7 @@ enum paravirt_lazy_mode paravirt_get_lazy_mode(void)struct pv_info pv_info = { .name ="bare hardware" , -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL .extra_user_64bit_cs = __USER_CS, #endif };
@@ -279,7 +279,7 @@ struct paravirt_patch_template pv_ops = {/* Cpu ops. */ .cpu.io_delay = native_io_delay, -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL .cpu.cpuid = native_cpuid, .cpu.get_debugreg = native_get_debugreg, .cpu.set_debugreg = native_set_debugreg,
@@ -324,7 +324,7 @@ struct paravirt_patch_template pv_ops = { .irq.irq_enable = __PV_IS_CALLEE_SAVE(native_irq_enable), .irq.safe_halt = native_safe_halt, .irq.halt = native_halt, -#endif /* CONFIG_PARAVIRT_XXL */ +#endif /* CONFIG_PARAVIRT_FULL */ /* Mmu ops. */ .mmu.flush_tlb_user = native_flush_tlb_local,
@@ -336,7 +336,7 @@ struct paravirt_patch_template pv_ops = { .mmu.exit_mmap = paravirt_nop, -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL .mmu.read_cr2 =__PV_IS_CALLEE_SAVE (native_read_cr2), .mmu.write_cr2 = native_write_cr2, .mmu.read_cr3 = __native_read_cr3,
@@ -393,7 +393,7 @@ struct paravirt_patch_template pv_ops = { }, .mmu.set_fixmap = native_set_fixmap, -#endif /* CONFIG_PARAVIRT_XXL */ +#endif /* CONFIG_PARAVIRT_FULL */ #if defined(CONFIG_PARAVIRT_SPINLOCKS )/* Lock ops. */
@@ -409,7 +409,7 @@ struct paravirt_patch_template pv_ops = {#endif }; -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL /* At this point, native_get/set_debugreg has real function entries */ NOKPROBE_SYMBOL (native_get_debugreg);NOKPROBE_SYMBOL (native_set_debugreg);
arch/x86/kernel/paravirt_patch.c +3 -3
@@ -17,7 +17,7 @@case PARAVIRT_PATCH (ops.m): \ return PATCH(data, ops##_##m, insn_buff, len) -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL struct patch_xxl { const unsigned char irq_irq_disable[1]; const unsigned char irq_irq_enable[1];
@@ -44,7 +44,7 @@ unsigned int paravirt_patch_ident_64(void *insn_buff, unsigned int len) {return PATCH (xxl, mov64, insn_buff, len); } -# endif /* CONFIG_PARAVIRT_XXL */ +# endif /* CONFIG_PARAVIRT_FULL */ #ifdef CONFIG_PARAVIRT_SPINLOCKS struct patch_lock {
@@ -68,7 +68,7 @@ unsigned int native_patch(u8 type, void *insn_buff, unsigned long addr, {switch (type) { -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL PATCH_CASE (irq, save_fl, xxl, insn_buff, len);PATCH_CASE (irq, irq_enable, xxl, insn_buff, len);PATCH_CASE (irq, irq_disable, xxl, insn_buff, len);
arch/x86/mm/mem_encrypt_identity.c +1 -1
@@ -24,7 +24,7 @@* be extended when new paravirt and debugging variants are added.) */ #undef CONFIG_PARAVIRT -#undef CONFIG_PARAVIRT_XXL +#undef CONFIG_PARAVIRT_FULL#undef CONFIG_PARAVIRT_SPINLOCKS#include <linux/kernel.h>
arch/x86/xen/Kconfig +1 -1
@@ -20,7 +20,7 @@ config XEN_PV default y depends on XEN depends on X86_64 - select PARAVIRT_XXL + select PARAVIRT_FULL select XEN_HAVE_PVMMU select XEN_HAVE_VPMU help
tools/arch/x86/include/asm/required-features.h +1 -1
@@ -54,7 +54,7 @@#endif #ifdef CONFIG_X86_64 -#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_PARAVIRT_FULL /* Paravirtualized systems may not have PSE or PGE available */ #define NEED_PSE 0#define NEED_PGE 0
Signature · 1 line
2.26.3
From 08e03cebb65b48e6b7e5bf0d805762cc661d09f0 Mon Sep 17 00:00:00 2001
From: "Kirill A. Shutemov" <[email protected]>
Date: Tue, 19 Nov 2019 14:56:03 +0300
Subject: [PATCH 2/2] x86/paravirt: Introduce CONFIG_PARAVIRT_HLT
CONFIG_PARAVIRT_FULL provides a way to hook up the full set of paravirt ops, but TDX only needs two of them: halt() and safe_halt().
Split off halt paravirt calls from CONFIG_PARAVIRT_FULL into a separate config option -- CONFIG_PARAVIRT_HLT.
--- arch/x86/Kconfig | 4 +++ arch/x86/boot/compressed/misc.h | 1 + arch/x86/include/asm/irqflags.h | 38 +++++++++++++++------------ arch/x86/include/asm/paravirt.h | 22 +++++++++------- arch/x86/include/asm/paravirt_types.h | 3 ++- arch/x86/kernel/paravirt.c | 4 ++- arch/x86/mm/mem_encrypt_identity.c | 1 + 7 files changed, 44 insertions(+), 29 deletions(-)
arch/x86/Kconfig +4 -0
@@ -783,8 +783,12 @@ config PARAVIRT over full virtualization. However, when run without a hypervisor the kernel is theoretically slower and slightly larger. +config PARAVIRT_HLT + bool + config PARAVIRT_FULL bool + select PARAVIRT_HLT config PARAVIRT_DEBUG bool "paravirt-ops debugging"
arch/x86/boot/compressed/misc.h +1 -0
@@ -9,6 +9,7 @@* paravirt and debugging variants are added.) */ #undef CONFIG_PARAVIRT +#undef CONFIG_PARAVIRT_HLT#undef CONFIG_PARAVIRT_FULL#undef CONFIG_PARAVIRT_SPINLOCKS#undef CONFIG_KASAN
arch/x86/include/asm/irqflags.h +21 -17
@@ -59,27 +59,11 @@ static inline __cpuidle void native_halt(void)#endif -#ifdef CONFIG_PARAVIRT_FULL +#ifdef CONFIG_PARAVIRT_HLT #include <asm/paravirt.h> #else #ifndef __ASSEMBLY__#include <linux/types.h> - -static __always_inline unsigned long arch_local_save_flags (void ) -{ -return native_save_fl (); -} - -static __always_inline void arch_local_irq_disable(void) -{ -native_irq_disable (); -} - -static __always_inline void arch_local_irq_enable(void) -{ -native_irq_enable (); -} -/* * Used in the idle loop; sti takes one instruction cycle * to complete:
@@ -97,6 +81,26 @@ static inline __cpuidle void halt(void) {native_halt (); } +#endif /* !__ASSEMBLY__ */ +#endif /* CONFIG_PARAVIRT_HLT */ + +#ifndef CONFIG_PARAVIRT_FULL +#ifndef __ASSEMBLY__ + +static __always_inline unsigned long arch_local_save_flags (void ) +{ +return native_save_fl (); +} + +static __always_inline void arch_local_irq_disable(void) +{ +native_irq_disable (); +} + +static __always_inline void arch_local_irq_enable(void) +{ +native_irq_enable (); +}/* * For spinlocks, etc:
arch/x86/include/asm/paravirt.h +12 -10
@@ -84,6 +84,18 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct *mm)PVOP_VCALL1 (mmu.exit_mmap, mm); } +#ifdef CONFIG_PARAVIRT_HLT +static inline void arch_safe_halt (void ) +{ +PVOP_VCALL0 (irq.safe_halt); +} + +static inline void halt (void ) +{ +PVOP_VCALL0 (irq.halt); +} +#endif +#ifdef CONFIG_PARAVIRT_FULL static inline void load_sp0 (unsigned long sp0) {
@@ -145,16 +157,6 @@ static inline void __write_cr4(unsigned long x)PVOP_VCALL1 (cpu.write_cr4, x); } -static inline void arch_safe_halt (void ) -{ -PVOP_VCALL0 (irq.safe_halt); -} - -static inline void halt (void ) -{ -PVOP_VCALL0 (irq.halt); -} -static inline void wbinvd (void ) {PVOP_VCALL0 (cpu.wbinvd);
arch/x86/include/asm/paravirt_types.h +2 -1
@@ -177,7 +177,8 @@ struct pv_irq_ops {struct paravirt_callee_save save_fl;struct paravirt_callee_save irq_disable;struct paravirt_callee_save irq_enable; - +#endif +#ifdef CONFIG_PARAVIRT_HLT void (*safe_halt)(void );void (*halt)(void );#endif
arch/x86/kernel/paravirt.c +3 -1
@@ -322,9 +322,11 @@ struct paravirt_patch_template pv_ops = { .irq.save_fl = __PV_IS_CALLEE_SAVE(native_save_fl), .irq.irq_disable = __PV_IS_CALLEE_SAVE(native_irq_disable), .irq.irq_enable = __PV_IS_CALLEE_SAVE(native_irq_enable), +#endif /* CONFIG_PARAVIRT_FULL */ +#ifdef CONFIG_PARAVIRT_HLT .irq.safe_halt = native_safe_halt, .irq.halt = native_halt, -#endif /* CONFIG_PARAVIRT_FULL */ +#endif /* CONFIG_PARAVIRT_HLT */ /* Mmu ops. */ .mmu.flush_tlb_user = native_flush_tlb_local,
arch/x86/mm/mem_encrypt_identity.c +1 -0
@@ -24,6 +24,7 @@* be extended when new paravirt and debugging variants are added.) */ #undef CONFIG_PARAVIRT +#undef CONFIG_PARAVIRT_HLT#undef CONFIG_PARAVIRT_FULL#undef CONFIG_PARAVIRT_SPINLOCKS
Signature · 1 line
2.26.3
Re: [RFC v2 07/32] x86/traps: Add do_general_protection() helper function
In reply to Kuppuswamy SathyanarayananQuoted 1 line · Kuppuswamy Sathyanarayanan
TDX guest #VE exception handler treats unsupported exceptions
^ The
Quoted 1 line
as #GP. So to handle the #GP, move the protection fault handler
s/So to/To/
Also, it does not "treat them as #GP". It handles them in the same way that a #GP is handled. There's a difference between literally making them a #GP and having a similar end result. This description conflates them.
Quoted 2 lines
code to out of
exc_general_protection()and create new helper function for it.
I wouldn't name the functions. Just say that you want the #GP behavior from #VE so you need a common helper.
Quoted 1 line
Also since exception handler is responsible to decide when to
^ an
Quoted 2 lines
turn on/off IRQ, move cond_local_irq_{enable/disable)() calls out of do_general_protection().
This paragraph doesn't really say anything meaningful. Yes, exception handlers reenable interrupts. Try to *SAY* something about why they do this and why you have to move the code around. Or, just axe it.
Quoted 63 lines
This is a preparatory patch for adding #VE exception handler support for TDX guests.
Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>--- arch/x86/kernel/traps.c | 51 ++++++++++++++++++++++------------------- 1 file changed, 27 insertions(+), 24 deletions(-)arch/x86/kernel/traps.c +5
-21@@ -527,44 +527,28 @@ static enum kernel_gp_hint get_kernel_gp_address(struct pt_regs *regs,#define GPFSTR "general protection fault" -DEFINE_IDTENTRY_ERRORCODE (exc_general_protection) +static void do_general_protection (struct pt_regs *regs,long error_code) {char desc[sizeof (GPFSTR ) +50 +2 *sizeof (unsigned long ) +1 ] =GPFSTR ;enum kernel_gp_hint hint =GP_NO_HINT ; -struct task_struct *tsk; +struct task_struct *tsk = current;unsigned long gp_addr;int ret; -cond_local_irq_enable (regs); - -if (static_cpu_has (X86_FEATURE_UMIP )) { -if (user_mode (regs) &&fixup_umip_exception (regs)) - goto exit; - } - -if (v8086_mode (regs)) { -local_irq_enable (); -handle_vm86_fault ((struct kernel_vm86_regs *) regs, error_code); -local_irq_disable (); -return ; - } - - tsk = current; -if (user_mode (regs)) { tsk->thread.error_code = error_code; tsk->thread.trap_nr =X86_TRAP_GP ;if (fixup_vdso_exception (regs,X86_TRAP_GP , error_code,0 )) - goto exit; +return ;show_signal (tsk,SIGSEGV ,"" , desc, regs, error_code);force_sig (SIGSEGV ); - goto exit; +return ; }if (fixup_exception (regs,X86_TRAP_GP , error_code,0 )) - goto exit; +return ; tsk->thread.error_code = error_code; tsk->thread.trap_nr =X86_TRAP_GP ;
arch/x86/kernel/traps.c DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) · line 565
Quoted 7 lines
arch/x86/kernel/traps.c +1
-1@@ -576,11 +560,11 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)if (!preemptible () &&kprobe_running () &&kprobe_fault_handler (regs,X86_TRAP_GP )) - goto exit; +return ; ret =notify_die (DIE_GPF , desc, regs, error_code,X86_TRAP_GP ,SIGSEGV );
So... We're going to send signals based on #VE which use this bit in the ABI which is documented as:
#define X86_TRAP_GP 13 /* General Protection Fault */
Considering that there is also a:
#define X86_TRAP_VE 20 /* Virtualization Exception */
this seems like a stretch.
Also, isnt there a lot of truly #GP-specific code in there, like fixup_exception()? Why do you need to call that for #VE? How did you decide what remains in the handler versus what gets separated out?
Quoted 34 lines
arch/x86/kernel/traps.c +21
-2if (ret ==NOTIFY_STOP ) - goto exit; +return ;if (error_code)snprintf (desc,sizeof (desc),"segment-related " GPFSTR );@@ -601,8 +585,27 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) gp_addr =0 ;die_addr (desc, regs, error_code, gp_addr); +} -exit: +DEFINE_IDTENTRY_ERRORCODE (exc_general_protection) +{ +cond_local_irq_enable (regs); + +if (static_cpu_has (X86_FEATURE_UMIP )) { +if (user_mode (regs) &&fixup_umip_exception (regs)) { +cond_local_irq_disable (regs); +return ; + } + } + +if (v8086_mode (regs)) { +local_irq_enable (); +handle_vm86_fault ((struct kernel_vm86_regs *) regs, error_code); +local_irq_disable (); +return ; + } + +do_general_protection (regs, error_code);cond_local_irq_disable (regs); }
Re: [RFC v2 08/32] x86/traps: Add #VE support for TDX guest
In reply to Kuppuswamy SathyanarayananOn 4/26/21 11:01 AM, Kuppuswamy Sathyanarayanan wrote:
...
Quoted 5 lines
The #VE cannot be nested before TDGETVEINFO is called, if there is any reason for it to nest the TD would shut down. The TDX module guarantees that no NMIs (or #MC or similar) can happen in this window. After TDGETVEINFO the #VE handler can nest if needed, although we don’t expect it to happen normally.
I think this description really needs some work. Does "The #VE cannot be nested" mean that "hardware guarantees that #VE will not be generated", or "the #VE must not be nested"?
What does "the TD would shut down" mean? I think you mean that instead of delivering a nested #VE the hardware would actually exit to the host and TDX would prevent the guest from being reentered. Right?
Quoted 29 lines
arch/x86/include/asm/idtentry.h +4
-0@@ -619,6 +619,10 @@ DECLARE_IDTENTRY_XENCB(X86_TRAP_OTHER, exc_xen_hypervisor_callback);DECLARE_IDTENTRY_RAW (X86_TRAP_OTHER , exc_xen_unknown_trap);#endif +#ifdef CONFIG_INTEL_TDX_GUEST +DECLARE_IDTENTRY (X86_TRAP_VE , exc_virtualization_exception); +#endif +/* Device interrupts common/spurious */ DECLARE_IDTENTRY_IRQ (X86_TRAP_OTHER , common_interrupt);#ifdef CONFIG_X86_LOCAL_APIC arch/x86/include/asm/tdx.h +1
-0@@ -11,6 +11,7 @@#include <linux/types.h> #define TDINFO 1 +#define TDGETVEINFO 3struct tdcall_output {u64 rcx;@@ -29,6 +30,20 @@ struct tdvmcall_output {u64 r15;
arch/x86/include/asm/tdx.h struct tdvmcall_output { · line 40
Quoted 10 lines
arch/x86/include/asm/tdx.h +8
-0}; +struct ve_info { +u64 exit_reason; +u64 exit_qual; +u64 gla; +u64 gpa; +u32 instr_len; +u32 instr_info; +};
Is this an architectural structure or some software construct?
Quoted 43 lines
arch/x86/include/asm/tdx.h +5
-0+unsigned long tdg_get_ve_info (struct ve_info *ve); + +int tdg_handle_virtualization_exception (struct pt_regs *regs, +struct ve_info *ve); +/* Common API to check TDX support in decompression and common kernel code. */ bool is_tdx_guest (void );arch/x86/kernel/idt.c +6
-0@@ -64,6 +64,9 @@ static const __initconst struct idt_data early_idts[] = {*/ INTG(X86_TRAP_PF, asm_exc_page_fault),#endif +#ifdef CONFIG_INTEL_TDX_GUEST + INTG(X86_TRAP_VE, asm_exc_virtualization_exception), +#endif };/* @@ -87,6 +90,9 @@ static const __initconst struct idt_data def_idts[] = {INTG (X86_TRAP_MF , asm_exc_coprocessor_error),INTG (X86_TRAP_AC , asm_exc_alignment_check),INTG (X86_TRAP_XF , asm_exc_simd_coprocessor_error), +#ifdef CONFIG_INTEL_TDX_GUEST +INTG (X86_TRAP_VE , asm_exc_virtualization_exception), +#endif #ifdef CONFIG_X86_32 TSKG(X86_TRAP_DF, GDT_ENTRY_DOUBLEFAULT_TSS),arch/x86/kernel/tdx.c +3
-0@@ -82,6 +82,44 @@ static void tdg_get_info(void) td_info.attributes = out.rdx; } +unsigned long tdg_get_ve_info (struct ve_info *ve) +{ +u64 ret;
arch/x86/kernel/tdx.c static void tdg_get_info(void) · line 97
Quoted 10 lines
arch/x86/kernel/tdx.c +10
-0+struct tdcall_output out = {0 }; + +/* +* The #VE cannot be nested before TDGETVEINFO is called, +* if there is any reason for it to nest the TD would shut +* down. The TDX module guarantees that no NMIs (or #MC or +* similar) can happen in this window. After TDGETVEINFO +* the #VE handler can nest if needed, although we don’t +* expect it to happen normally. +*/
I find that description a bit unsatisfying. Could we make this a bit more concrete? By the way, what about *normal* interrupts?
Maybe we should talk about this in terms of *rules* that folks need to follow. Maybe:
NMIs and machine checks are suppressed. Before this point any #VE is fatal. After this point, NMIs and additional #VEs are permitted.
Quoted 45 lines
arch/x86/kernel/tdx.c +24
-0+ ret =__tdcall (TDGETVEINFO ,0 ,0 ,0 ,0 , &out); + + ve->exit_reason = out.rcx; + ve->exit_qual = out.rdx; + ve->gla = out.r8; + ve->gpa = out.r9; + ve->instr_len = out.r10 &UINT_MAX ; + ve->instr_info = out.r10 >>32 ; + +return ret; +} + +int tdg_handle_virtualization_exception (struct pt_regs *regs, +struct ve_info *ve) +{ +/* +* TODO: Add handler support for various #VE exit +* reasons. It will be added by other patches in +* the series. +*/ +pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason); +return -EFAULT ; +} +void __init tdx_early_init (void ) {if (!cpuid_has_tdx_guest ())arch/x86/kernel/traps.c +3
-0@@ -61,6 +61,7 @@#include <asm/insn.h> #include <asm/insn-eval.h> #include <asm/vdso.h> +#include <asm/tdx.h> #ifdef CONFIG_X86_64 #include <asm/x86_init.h> @@ -1140,6 +1141,35 @@ DEFINE_IDTENTRY(exc_device_not_available) } } +#ifdef CONFIG_INTEL_TDX_GUEST +DEFINE_IDTENTRY (exc_virtualization_exception)
arch/x86/kernel/traps.c DEFINE_IDTENTRY(exc_device_not_available) · line 1155
Quoted 10 lines
arch/x86/kernel/traps.c +10
-0+{ +struct ve_info ve; +int ret; + +RCU_LOCKDEP_WARN (!rcu_is_watching (),"entry code didn't wake RCU" ); + +/* +* Consume #VE info before re-enabling interrupts. It will be +* re-enabled after executing the TDGETVEINFO TDCALL. +*/
"It" is nebulous here. Is this talking about NMIs, or the cond_local_irq_enable() that is "after" TDGETVEINFO?
Quoted 21 lines
arch/x86/kernel/traps.c +17
-0+ ret =tdg_get_ve_info (&ve); + +cond_local_irq_enable (regs); + +if (!ret) + ret =tdg_handle_virtualization_exception (regs, &ve); +/* +* If tdg_handle_virtualization_exception() could not process +* it successfully, treat it as #GP(0) and handle it. +*/ +if (ret) +do_general_protection (regs,0 ); + +cond_local_irq_disable (regs); +} +#endif +#ifdef CONFIG_X86_32 DEFINE_IDTENTRY_SW (iret_error) {
Re: [RFC v2 10/32] x86/tdx: Wire up KVM hypercalls
In reply to Kuppuswamy SathyanarayananQuoted 3 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
KVM hypercalls have to be wrapped into vendor-specific TDVMCALLs.
How about:
KVM hypercalls use the "vmcall" or "vmmcall" instructions. Although the ABI is similar, those instructions no longer function for TDX guests. Make TDVMCALLs instead of VMCALL/VMCALL.
Quoted 13 lines
arch/x86/include/asm/kvm_para.h +1
-0@@ -6,6 +6,7 @@#include <asm/alternative.h> #include <linux/interrupt.h> #include <uapi/asm/kvm_para.h> +#include <asm/tdx.h> extern void kvmclock_init (void );
arch/x86/include/asm/kvm_para.h static inline bool kvm_check_and_clear_guest_paused(void) · line 40
Quoted 6 lines
arch/x86/include/asm/kvm_para.h +3
-0@@ -34,6 +35,10 @@ static inline bool kvm_check_and_clear_guest_paused(void)static inline long kvm_hypercall0 (unsigned int nr) {long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall0 (nr);
... all of these look OK.
Quoted 14 lines
arch/x86/include/asm/kvm_para.h +0
-0#endif /* _ASM_X86_TDX_H */ arch/x86/kernel/tdcall.S +1
-0@@ -11,6 +11,7 @@ * refer to TDX GHCI specification). */ #define TDVMCALL_EXPOSE_REGS_MASK 0xfc00 +#define TDVMCALL_VENDOR_KVM 0x4d564b2e584454 /* "TDX.KVM" */ /* * TDX guests use the TDCALL instruction to make
arch/x86/kernel/tdcall.S SYM_FUNC_START(__tdvmcall) · line 207
Quoted 9 lines
arch/x86/kernel/tdcall.S +6
-0@@ -198,3 +199,9 @@ SYM_FUNC_START(__tdvmcall) call do_tdvmcall retq SYM_FUNC_END(__tdvmcall) + +SYM_FUNC_START(__tdvmcall_vendor_kvm) + movq $TDVMCALL_VENDOR_KVM, %r10 + call do_tdvmcall + retq +SYM_FUNC_END(__tdvmcall_vendor_kvm)
Granted, this is not a ton of assembly. But, it does look a bit weird.
It needs a comment and/or a mention in the changelog.
R10 is not part of the function call ABI, but it is a part of the TDVMCALL ABI. This little assembly wrapper lets us reuse do_tdvmcall() for both KVM-specific hypercalls TDVMCALL_VENDOR_KVM and the more generic __tdvmcalls.
arch/x86/kernel/tdx.c line 17
Quoted 13 lines
arch/x86/kernel/tdx.c +4
-0@@ -8,6 +8,10 @@#include <linux/cpu.h> +#ifdef CONFIG_KVM_GUEST +#include "tdx-kvm.c" +#endif + static struct { unsigned int gpa_width; unsigned long attributes;
I know KVM does weird stuff. But, this is *really* weird. Why are we #including a .c file into another .c file?
Re: [RFC v2 15/32] x86/tdx: Handle in-kernel MMIO
In reply to Kuppuswamy SathyanarayananQuoted 2 lines · Kuppuswamy Sathyanarayanan
Handle #VE due to MMIO operations. MMIO triggers #VE with
EPT_VIOLATIONexit reason.
This needs a bit of a history lesson. "In traditional VMs, MMIO tends to be implemented by giving a guest access to an mapping which will cause a VMEXIT on access. That's not possible in a TDX guest..."
Quoted 2 lines
For now we only handle subset of instruction that kernel uses for MMIO oerations. User-space access triggers SIGBUS.
I still don't think that TDX guests should be doing things that they *KNOW* will cause #VE, including MMIO. I really want to hear a more discrete story about why this is the *best* way to do this for Linux instead of just a hack from the Windows binary driver ecosystem that seemed expedient.
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
In reply to Kuppuswamy SathyanarayananQuoted 1 line
+++
b/arch/x86/mm/mem_encrypt_common.c
...
Quoted 15 lines
arch/x86/mm/mem_encrypt_common.c new file +15
-0+/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ +bool force_dma_unencrypted (struct device *dev) +{ +/* +* For SEV, all DMA must be to unencrypted/shared addresses. +*/ +if (sev_active ()) +return true; + +/* +* For SME, all DMA must be to unencrypted addresses if the +* device does not support DMA to addresses that include the +* encryption mask. +*/ +if (sme_active ()) {
arch/x86/mm/mem_encrypt_common.c line 38
Quoted 10 lines
arch/x86/mm/mem_encrypt_common.c new file +10
-0+u64 dma_enc_mask =DMA_BIT_MASK (__ffs64 (sme_me_mask)); +u64 dma_dev_mask =min_not_zero (dev->coherent_dma_mask, + dev->bus_dma_limit); + +if (dma_dev_mask <= dma_enc_mask) +return true; + } + +return false; +}
This doesn't seem much like common code to me. It seems like 100% SEV code. Is this really where we want to move it?
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
In reply to Kuppuswamy Sathyanarayananarch/x86/mm/ioremap.c static unsigned int __ioremap_check_ram(struct resource *res) · line 96
Quoted 5 lines · Kuppuswamy Sathyanarayanan
static unsigned int
__ioremap_check_encrypted(struct resource *res){arch/x86/mm/ioremap.c +1
-1-if (!sev_active ()) +if (!sev_active () && !is_tdx_guest ())return 0 ;
I think it's time to come up with a real name for all of the code that's
under: (sev_active() || is_tdx_guest()).
"encrypted" isn't it, for sure.
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
Quoted 10 lines · Dave Hansen
arch/x86/mm/ioremap.c static unsigned int __ioremap_check_ram(struct resource *res) · line 96
Quoted 5 lines · Kuppuswamy Sathyanarayanan
static unsigned int __ioremap_check_encrypted(struct resource *res) {arch/x86/mm/ioremap.c +1
-1-if (!sev_active ()) +if (!sev_active () && !is_tdx_guest ())return 0 ;I think it's time to come up with a real name for all of the code that's
under: (sev_active()||is_tdx_guest())."encrypted" isn't it, for sure.
I called it protected_guest() in some other patches.
-Andi
Re: [RFC v2 32/32] x86/tdx: ioapic: Add shared bit for IOAPIC base address
In reply to Kuppuswamy SathyanarayananQuoted 7 lines · Kuppuswamy Sathyanarayanan
From: Isaku Yamahata <[email protected]>
IOAPIC is emulated by KVM which means its MMIO address is shared by host. Add shared bit for base address of IOAPIC. Most MMIO region is handled by ioremap which is already marked as shared for TDX guest platform, but IOAPIC is an exception which uses fixed map.
Ho hum... I guess I'll rewrite the changelog:
The kernel interacts with each bare-metal IOAPIC with a special MMIO page. When running under KVM, the guest's IOAPICs are emulated by KVM.
When running as a TDX guest, the guest needs to mark each IOAPIC mapping as "shared" with the host. This ensures that TDX private protections are not applied to the page, which allows the TDX host emulation to work.
Earlier patches in this series modified ioremap() so that ioremap()-created mappings such as virtio will be marked as shared. However, the IOAPIC code does not use ioremap() and instead uses the fixmap mechanism.
Introduce a special fixmap helper just for the IOAPIC code. Ensure that it marks IOAPIC pages as "shared". This replaces set_fixmap_nocache() with __set_fixmap() since __set_fixmap() allows custom 'prot' values.
Quoted 8 lines
arch/x86/kernel/apic/io_apic.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)arch/x86/kernel/apic/io_apic.c +0
-0
arch/x86/kernel/apic/io_apic.c static struct resource * __init ioapic_setup_resources(void) · line 2684
Quoted 10 lines
arch/x86/kernel/apic/io_apic.c +7
-0@@ -2675,6 +2675,14 @@ static struct resource * __init ioapic_setup_resources(void)return res; } +static void io_apic_set_fixmap_nocache (enum fixed_addresses idx,phys_addr_t phys) +{ +pgprot_t flags =FIXMAP_PAGE_NOCACHE ; +if (is_tdx_guest ()) + flags =pgprot_tdg_shared (flags); +__set_fixmap (idx, phys, flags); +}
^ This seems like it could at least use a one-liner comment.
Quoted 22 lines
arch/x86/kernel/apic/io_apic.c +2
-2void __init io_apic_init_mappings (void ) {unsigned long ioapic_phys, idx =FIX_IO_APIC_BASE_0 ;@@ -2707,7 +2715,7 @@ void __init io_apic_init_mappings(void) __func__, PAGE_SIZE, PAGE_SIZE); ioapic_phys =__pa (ioapic_phys); } -set_fixmap_nocache (idx, ioapic_phys); +io_apic_set_fixmap_nocache (idx, ioapic_phys);apic_printk (APIC_VERBOSE ,"mapped IOAPIC to %08lx (%08lx)\n" ,__fix_to_virt (idx) + (ioapic_phys & ~PAGE_MASK ), ioapic_phys);@@ -2836,7 +2844,7 @@ int mp_register_ioapic(int id, u32 address, u32 gsi_base, ioapics[idx].mp_config.flags =MPC_APIC_USABLE ; ioapics[idx].mp_config.apicaddr = address; -set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address); +io_apic_set_fixmap_nocache (FIX_IO_APIC_BASE_0 + idx, address);if (bad_ioapic_register (idx)) {clear_fixmap (FIX_IO_APIC_BASE_0 + idx);return -ENODEV ;
Re: [RFC v2 10/32] x86/tdx: Wire up KVM hypercalls
In reply to Dave HansenQuoted 2 lines · Dave Hansen
I know KVM does weird stuff. But, this is*really* weird. Why are we #including a .c file into another .c file?
I think Kirill implemented it this way to skip Makefile changes for it. I don't see any other KVM direct dependencies in tdx.c.
I will fix it in next version.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Juergen Gross
maintainerRe: [RFC v2 01/32] x86/paravirt: Introduce CONFIG_PARAVIRT_XL
In reply to Borislav PetkovQuoted 11 lines · Borislav Petkov
+ Jürgen.
Quoted 5 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
Split off halt paravirt calls from
CONFIG_PARAVIRT_XXLinto a separate config option. It provides a middle ground for not-so-deep paravirtulized environments.Please introduce a spellchecker into your patch creation workflow.
Quoted 7 lines
Also, what does "not-so-deep" mean?
Quoted 3 lines
CONFIG_PARAVIRT_XLwill be used by TDX that needs couple of paravirt calls that were hidden underCONFIG_PARAVIRT_XXL, but the rest of the config would be a bloat for TDX.Used how? Why is it bloat for TDX?
Is there any major downside to move the halt related pvops functions from CONFIG_PARAVIRT_XXL to CONFIG_PARAVIRT?
I'd rather introduce a new PARAVIRT level only in case of multiple pvops functions needed for a new guest type, or if a real hot path would be affected.
Juergen
Re: [RFC v2 01/32] x86/paravirt: Introduce CONFIG_PARAVIRT_XL
\
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
CONFIG_PARAVIRT_XLwill be used by TDX that needs couple of paravirt calls that were hidden underCONFIG_PARAVIRT_XXL, but the rest of the config would be a bloat for TDX.Used how? Why is it bloat for TDX?
Is there any major downside to move the halt related pvops functions from
CONFIG_PARAVIRT_XXLtoCONFIG_PARAVIRT?
I think the main motivation is to get rid of all the page table related hooks for modern configurations. These are the bulk of the annotations and cause bloat and worse code. Shadow page tables are really obscure these days and very few people still need them and it's totally reasonable to build even widely used distribution kernels without them. On contrast most of the other hooks are comparatively few and also on comparatively slow paths, so don't really matter too much.
I think it would be ok to have a CONFIG_PARAVIRT that does not have page table support, and a separate config option for those (that could be eventually deprecated).
But that would break existing .configs for those shadow stack users, that's why I think Kirill did it the other way around.
-Andi
Juergen Gross
maintainerRe: [RFC v2 01/32] x86/paravirt: Introduce CONFIG_PARAVIRT_XL
Quoted 7 lines · Andi Kleen
\
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines
CONFIG_PARAVIRT_XLwill be used by TDX that needs couple of paravirt calls that were hidden underCONFIG_PARAVIRT_XXL, but the rest of the config would be a bloat for TDX.Used how? Why is it bloat for TDX?
Quoted 8 lines
Quoted 2 lines
Is there any major downside to move the halt related pvops functions from
CONFIG_PARAVIRT_XXLtoCONFIG_PARAVIRT?I think the main motivation is to get rid of all the page table related hooks for modern configurations. These are the bulk of the annotations and cause bloat and worse code. Shadow page tables are really obscure these days and very few people still need them and it's totally reasonable to build even widely used distribution kernels without them. On contrast most of the other hooks are comparatively few and also on comparatively slow paths, so don't really matter too much.
I think it would be ok to have a
CONFIG_PARAVIRTthat does not have page table support, and a separate config option for those (that could be eventually deprecated).But that would break existing .configs for those shadow stack users, that's why I think Kirill did it the other way around.
No. We have PARAVIRT_XXL for Xen PV guests, and we have PARAVIRT for other hypervisor's guests, supporting basically the TLB flush operations and time related operations only. Adding the halt related operations to PARAVIRT wouldn't break anything.
Juergen
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Kuppuswamy SathyanarayananQuoted 2 lines · Kuppuswamy Sathyanarayanan
From: "Kirill A. Shutemov" <[email protected]>
While I do not expect that a patch in the middle of a series needs the full introduction of all concepts, the high expectations of the reader's context in this changelog make the patch actively painful to read.
Some connective tissue commentary to list assumptions and pointers to definitions is needed to make this patch stand alone when a future bisect lands on it and someone wonders where to get started debugging it.
Quoted 2 lines
Unroll string operations and handle port I/O through TDVMCALLs. Also handle #VE due to I/O operations with the same TDVMCALLs.
There is a mix of direct-TDVMCALL usage and handling #VE when and why is either approached used?
Quoted 2 lines
Decompression code uses port IO for earlyprintk. We must use paravirt calls there too if we want to allow earlyprintk.
What is the tradeoff between teaching the decompression code to handle #VE (the implied assumption) vs teaching it to avoid #VE with direct TDVMCALLs (the chosen direction)?
Rewrite without "we":
"Given the need to support earlyprintk for protected guests, deploy paravirt calls for the io*() and out*() usage in the decompress code."
This raises the question of why the cover letter switched from explicitly saying TDVMCALL to "paravirt" where it could be confused with the typical paravirt helpers?
Quoted 2 lines
Decompresion code cannot deal with alternatives: use branches
s/Decompresion/Decompression/
Quoted 6 lines
instead to implement
inX()andoutX()helpers.Since we use call instruction in place of in/out instruction, the argument passed to call instruction has to be in a register, it cannot be an immediate value like in/out instruction. So change constraint flag from "Nd" to "d"
Rewrite without "we":
With the approach to use a "call" instruction as an alternative for an "in/out" instruction it is no longer the case that the argument can be an immediate value. Change the asm constraint flag from "Nd" to "d" to accomodate.
Quoted 11 lines
Co-developed-by Kuppuswamy Sathyanarayanan <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>Signed-off-by Kirill A. Shutemov <[email protected]>Reviewed-by Andi Kleen <[email protected]>--- arch/x86/boot/compressed/Makefile | 1 + arch/x86/boot/compressed/tdcall.S | 9 ++ arch/x86/include/asm/io.h | 5 +- arch/x86/include/asm/tdx.h | 46 ++++++++- arch/x86/kernel/tdcall.S | 154 ++++++++++++++++++++++++++++++
Why is this named "tdcall" when it is implementing tdvmcalls? I must say those names don't really help me understand what they do. Can we have Linux names that don't mandate keeping the spec terminology in my brain's translation cache?
Quoted 22 lines
arch/x86/kernel/tdx.c | 33 +++++++ 6 files changed, 245 insertions(+), 3 deletions(-) create mode 100644 arch/x86/boot/compressed/tdcall.Sarch/x86/boot/compressed/Makefile +1
-0@@ -97,6 +97,7 @@ endif vmlinux-objs-$(CONFIG_ACPI) += $(obj)/acpi.o vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdx.o +vmlinux-objs-$(CONFIG_INTEL_TDX_GUEST) += $(obj)/tdcall.o vmlinux-objs-$(CONFIG_EFI_MIXED) += $(obj)/efi_thunk_$(BITS).o efi-obj-$(CONFIG_EFI_STUB) = $(objtree)/drivers/firmware/efi/libstub/lib.aarch/x86/boot/compressed/tdcall.S new file +0
-0
arch/x86/boot/compressed/tdcall.S line 7
Quoted 7 lines
arch/x86/boot/compressed/tdcall.S new file +7
-0@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include <asm/export.h> + +/* Do not export symbols in decompression code */ +#undef EXPORT_SYMBOL +#define EXPORT_SYMBOL(sym)
What's wrong with the existing:
KBUILD_CFLAGS += -D__DISABLE_EXPORTS
...in arch/x86/boot/compressed/Makefile?
Quoted 15 lines
arch/x86/boot/compressed/tdcall.S new file +2
-0+ +#include "../../kernel/tdcall.S"arch/x86/include/asm/io.h +1
-0@@ -43,6 +43,7 @@#include <asm/page.h> #include <asm/early_ioremap.h> #include <asm/pgtable_types.h> +#include <asm/tdx.h> #define build_mmio_read (name, size, type, reg, barrier) \ static inline type name(const volatile void __iomem *addr) \
arch/x86/include/asm/io.h static inline unsigned type in##bwl##_p(int port) \ · line 313
Quoted 5 lines
arch/x86/include/asm/io.h +1
-1@@ -309,7 +310,7 @@ static inline unsigned type in##bwl##_p(int port) \ \ static inline void outs##bwl(int port, const void *addr, unsigned long count) \ { \ - if (sev_key_active()) { \ + if (sev_key_active() || is_tdx_guest()) { \
Is there a unified Linux name these can be given to stop the proliferation of poor vendor names for similar concepts?
That routine
Quoted 26 lines
arch/x86/include/asm/io.h +1
-1unsigned type *value = (unsigned type *)addr; \while (count) { \ out##bwl(*value, port); \@@ -325,7 +326,7 @@ static inline void outs##bwl(int port, const void *addr, unsigned long count) \ \ static inline void ins##bwl(int port, void *addr, unsigned long count) \ { \ - if (sev_key_active()) { \ + if (sev_key_active() || is_tdx_guest()) { \ unsigned type *value = (unsigned type *)addr; \ while (count) { \ *value = in##bwl(port); \arch/x86/include/asm/tdx.h +2
-0@@ -5,6 +5,8 @@#define TDX_CPUID_LEAF_ID 0x21 +#ifndef __ASSEMBLY__ +#ifdef CONFIG_INTEL_TDX_GUEST #include <asm/cpufeature.h>
arch/x86/include/asm/tdx.h long tdx_kvm_hypercall3(unsigned int nr, unsigned long p1, unsigned long p2, · line 72
Quoted 4 lines
arch/x86/include/asm/tdx.h +1
-0@@ -67,6 +69,48 @@ long tdx_kvm_hypercall3(unsigned int nr, unsigned long p1, unsigned long p2,long tdx_kvm_hypercall4 (unsigned int nr,unsigned long p1,unsigned long p2,unsigned long p3,unsigned long p4); +/* Decompression code doesn't know how to handle alternatives */
Does it also not know how to handle #VE to keep it aligned with the runtime code?
Quoted 21 lines
arch/x86/include/asm/tdx.h +21
-0+#ifdef BOOT_COMPRESSED_MISC_H +#define __out (bwl, bw) \ +do { \ + if (is_tdx_guest()) { \ + asm volatile("call tdg_out" #bwl : : \ + "a"(value), "d"(port)); \ + } else { \ + asm volatile("out" #bwl " %" #bw "0, %w1" : : \ + "a"(value), "Nd"(port)); \ + } \ +} while (0) +#define __in (bwl, bw) \ +do { \ + if (is_tdx_guest()) { \ + asm volatile("call tdg_in" #bwl : \ + "=a"(value) : "d"(port)); \ + } else { \ + asm volatile("in" #bwl " %w1, %" #bw "0" : \ + "=a"(value) : "Nd"(port)); \ + } \ +} while (0)
arch/x86/include/asm/tdx.h long tdx_kvm_hypercall3(unsigned int nr, unsigned long p1, unsigned long p2, · line 103
Quoted 10 lines
arch/x86/include/asm/tdx.h +10
-0+#else +#define __out (bwl, bw) \ + alternative_input("out" #bwl " %" #bw "1, %w2", \ + "call tdg_out" #bwl, X86_FEATURE_TDX_GUEST, \ + "a"(value), "d"(port)) + +#define __in (bwl, bw) \ + alternative_io("in" #bwl " %w2, %" #bw "0", \ + "call tdg_in" #bwl, X86_FEATURE_TDX_GUEST, \ + "=a"(value), "d"(port))
Outside the boot decompression code isn't this branch of the "ifdef BOOT_COMPRESSED_MISC_H" handled by #VE? I also don't see any usage of __{in,out}() in this patch.
Quoted 33 lines
arch/x86/include/asm/tdx.h +11
-1+#endif + +void tdg_outb (unsigned char value,unsigned short port); +void tdg_outw (unsigned short value,unsigned short port); +void tdg_outl (unsigned int value,unsigned short port); + +unsigned char tdg_inb (unsigned short port); +unsigned short tdg_inw (unsigned short port); +unsigned int tdg_inl (unsigned short port); +#else // !CONFIG_INTEL_TDX_GUEST static inline bool is_tdx_guest (void )@@ -106,5 +150,5 @@ static inline long tdx_kvm_hypercall4(unsigned int nr, unsigned long p1, }#endif /* CONFIG_INTEL_TDX_GUEST */ - +#endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_TDX_H */ arch/x86/kernel/tdcall.S +1
-0@@ -3,6 +3,7 @@ #include <asm/asm.h> #include <asm/frame.h> #include <asm/unwind_hints.h> +#include <asm/export.h> #include <linux/linkage.h>
arch/x86/kernel/tdcall.S line 19
Quoted 7 lines
arch/x86/kernel/tdcall.S +4
-0@@ -12,6 +13,12 @@ */ #define TDVMCALL_EXPOSE_REGS_MASK 0xfc00 #define TDVMCALL_VENDOR_KVM 0x4d564b2e584454 /* "TDX.KVM" */ +#define EXIT_REASON_IO_INSTRUCTION 30 +/* + * Current size of struct tdvmcall_output is 40 bytes, + * but allocate double to account future changes.
What future changes? Why could they not be handled as future code changes?
arch/x86/kernel/tdcall.S line 21
Quoted 2 lines
arch/x86/kernel/tdcall.S +2
-0+ */ +#define TDVMCALL_OUTPUT_SIZE 80
Perhaps "PAYLOAD_SIZE" since it is used for both input and output?
If the ABI does not include the size of the payload then how would code detect if even 80 bytes was violated in the future?
Quoted 13 lines
arch/x86/kernel/tdcall.S +6
-0/* * TDX guests use the TDCALL instruction to make@@ -205,3 +212,150 @@ SYM_FUNC_START(__tdvmcall_vendor_kvm) call do_tdvmcall retq SYM_FUNC_END(__tdvmcall_vendor_kvm) + +.macro io_save_registers + push %rbp + push %rbx + push %rcx + push %rdx
arch/x86/kernel/tdcall.S SYM_FUNC_START(__tdvmcall_vendor_kvm) · line 230
Quoted 10 lines
arch/x86/kernel/tdcall.S +10
-0+ push %rdi + push %rsi + push %r8 + push %r9 + push %r10 + push %r11 + push %r12 + push %r13 + push %r14 + push %r15
Surely there's an existing macro for this pattern? Would PUSH_AND_CLEAR_REGS + POP_REGS be suitable? Besides code sharing it would eliminate clearing of %r8.
Quoted 104 lines
arch/x86/kernel/tdcall.S +104
-0+.endm +.macro io_restore_registers + pop %r15 + pop %r14 + pop %r13 + pop %r12 + pop %r11 + pop %r10 + pop %r9 + pop %r8 + pop %rsi + pop %rdi + pop %rdx + pop %rcx + pop %rbx + pop %rbp +.endm + +/* + * tdg_out{b,w,l}() - Write given data to the specified port. + * + * @arg1 (RAX) - Value to be written (passed via R8 to do_tdvmcall()). + * @arg2 (RDX) - Port id (passed via RCX to do_tdvmcall()). + * + */ +SYM_FUNC_START(tdg_outb) + io_save_registers + xor %r8, %r8 + /* Move data to R8 register */ + mov %al, %r8b + /* Set data width to 1 byte */ + mov $1, %rsi + jmp 1f + +SYM_FUNC_START(tdg_outw) + io_save_registers + xor %r8, %r8 + /* Move data to R8 register */ + mov %ax, %r8w + /* Set data width to 2 bytes */ + mov $2, %rsi + jmp 1f + +SYM_FUNC_START(tdg_outl) + io_save_registers + xor %r8, %r8 + /* Move data to R8 register */ + mov %eax, %r8d + /* Set data width to 4 bytes */ + mov $4, %rsi +1: + /* + * Since io_save_registers does not save rax + * state, save it here so that we can preserve + * the caller register state. + */ + push %rax + + mov %rdx, %rcx + /* Set 1 in RDX to select out operation */ + mov $1, %rdx + /* Set TDVMCALL function id in RDI */ + mov $EXIT_REASON_IO_INSTRUCTION, %rdi + /* Set TDVMCALL type info (0 - Standard, > 0 - vendor) in R10 */ + xor %r10, %r10 + /* Since we don't use tdvmcall output, set it to NULL */ + xor %r9, %r9 + + call do_tdvmcall + + pop %rax + io_restore_registers + ret +SYM_FUNC_END(tdg_outb) +SYM_FUNC_END(tdg_outw) +SYM_FUNC_END(tdg_outl) +EXPORT_SYMBOL(tdg_outb) +EXPORT_SYMBOL(tdg_outw) +EXPORT_SYMBOL(tdg_outl) + +/* + * tdg_in{b,w,l}() - Read data to the specified port. + * + * @arg1 (RDX) - Port id (passed via RCX to do_tdvmcall()). + * + * Returns data read via RAX register. + * + */ +SYM_FUNC_START(tdg_inb) + io_save_registers + /* Set data width to 1 byte */ + mov $1, %rsi + jmp 1f + +SYM_FUNC_START(tdg_inw) + io_save_registers + /* Set data width to 2 bytes */ + mov $2, %rsi + jmp 1f + +SYM_FUNC_START(tdg_inl) + io_save_registers + /* Set data width to 4 bytes */ + mov $4, %rsi
arch/x86/kernel/tdcall.S SYM_FUNC_START(__tdvmcall_vendor_kvm) · line 344
Quoted 10 lines
arch/x86/kernel/tdcall.S +10
-0+1: + mov %rdx, %rcx + /* Set 0 in RDX to select in operation */ + mov $0, %rdx + /* Set TDVMCALL function id in RDI */ + mov $EXIT_REASON_IO_INSTRUCTION, %rdi + /* Set TDVMCALL type info (0 - Standard, > 0 - vendor) in R10 */ + xor %r10, %r10 + /* Allocate memory in stack for Output */ + subq $TDVMCALL_OUTPUT_SIZE, %rsp
Why is this leaf function responsibility? I would expect the core do_tdvmcall (or whatever it is renamed to) helper to hide output buffer payload handling. tdg_in* only wants 1, 2, or 4 bytes, not 40 bytes of payload to handle.
arch/x86/kernel/tdcall.S SYM_FUNC_START(__tdvmcall_vendor_kvm) · line 351
Quoted 7 lines
arch/x86/kernel/tdcall.S +7
-0+ /* Move tdvmcall_output pointer to R9 */ + movq %rsp, %r9 + + call do_tdvmcall + + /* Move data read from port to RAX */ + mov TDVMCALL_r11(%r9), %eax
"TDVMCALL_r11" is unreadable, what is that doing?
Shouldn't failed in* calls signal failure with an all ones result?
Quoted 29 lines
arch/x86/kernel/tdcall.S +10
-0+ /* Free allocated memory */ + addq $TDVMCALL_OUTPUT_SIZE, %rsp + io_restore_registers + ret +SYM_FUNC_END(tdg_inb) +SYM_FUNC_END(tdg_inw) +SYM_FUNC_END(tdg_inl) +EXPORT_SYMBOL(tdg_inb) +EXPORT_SYMBOL(tdg_inw) +EXPORT_SYMBOL(tdg_inl)arch/x86/kernel/tdx.c +11
-0@@ -189,6 +189,36 @@ static void tdg_handle_cpuid(struct pt_regs *regs) regs->dx = out.r15; } +static void tdg_out (int size,int port,unsigned int value) +{ +tdvmcall (EXIT_REASON_IO_INSTRUCTION , size,1 , port, value); +} + +static unsigned int tdg_in (int size,int port) +{ +return tdvmcall_out_r11 (EXIT_REASON_IO_INSTRUCTION , size,0 , port,0 ); +} + +static void tdg_handle_io (struct pt_regs *regs,u32 exit_qual)
arch/x86/kernel/tdx.c static void tdg_handle_cpuid(struct pt_regs *regs) · line 212
Quoted 10 lines
arch/x86/kernel/tdx.c +10
-0+{ +bool string = exit_qual &16 ; +int out, size, port; + +/* I/O strings ops are unrolled at build time. */ +BUG_ON (string); + + out = (exit_qual &8 ) ?0 :1 ; + size = (exit_qual &7 ) +1 ; + port = exit_qual >>16 ;
This seems to be begging for exit_qual helpers to put symbolic names on these operations.
Quoted 25 lines
arch/x86/kernel/tdx.c +12
-1+ +if (out) { +tdg_out (size, port, regs->ax); + }else { + regs->ax &= ~GENMASK (8 * size,0 ); + regs->ax |=tdg_in (size, port) &GENMASK (8 * size,0 ); + } +} +unsigned long tdg_get_ve_info (struct ve_info *ve) {u64 ret;@@ -238,6 +268,9 @@ int tdg_handle_virtualization_exception(struct pt_regs *regs,case EXIT_REASON_CPUID :tdg_handle_cpuid (regs);break ; +case EXIT_REASON_IO_INSTRUCTION : +tdg_handle_io (regs, ve->exit_qual); +break ;default :pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason);return -EFAULT ; --2.25.1
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
In reply to Dave HansenQuoted 2 lines · Dave Hansen
This doesn't seem much like common code to me. It seems like 100% SEV code. Is this really where we want to move it?
Both SEV and TDX code has requirement to enable CONFIG_ARCH_HAS_FORCE_DMA_UNENCRYPTED and define force_dma_unencrypted() function.
force_dma_unencrypted() is modified by patch titled "x86/tdx: Make DMA pages shared" to add TDX guest specific support.
Since both SEV and TDX code uses it, its moved to common file.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
In reply to Andi KleenHi Dave,
Quoted 8 lines · Andi Kleen
Quoted 6 lines · Dave Hansen
Quoted 5 lines · Kuppuswamy Sathyanarayanan
static unsigned int
__ioremap_check_encrypted(struct resource *res){
- if (!sev_active())
+ if (!sev_active() && !is_tdx_guest())
return 0;
Quoted 6 lines
Quoted 4 lines
I think it's time to come up with a real name for all of the code that's
under: (sev_active()||is_tdx_guest())."encrypted" isn't it, for sure.
I called it
protected_guest()in some other patches.
If you are also fine with above mentioned function name, I can include it in this series. Since we have many use cases of above condition, it will be useful define it as helper function.
Quoted 3 lines
-Andi
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
In reply to Kuppuswamy, SathyanarayananQuoted 4 lines · Sathyanarayanan
Quoted 2 lines · Dave Hansen
This doesn't seem much like common code to me. It seems like 100% SEV code. Is this really where we want to move it?
Quoted 8 lines
Both SEV and TDX code has requirement to enable
CONFIG_ARCH_HAS_FORCE_DMA_UNENCRYPTEDand defineforce_dma_unencrypted()function.
force_dma_unencrypted()is modified by patch titled "x86/tdx: Make DMA pages shared" to add TDX guest specific support.Since both SEV and TDX code uses it, its moved to common file.
That's not an excuse to have a bunch of AMD (or Intel) feature-specific code in a file named "common". I'd make an attempt to keep them separate and then call into the two separate functions *from* the common function.
Re: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
In reply to Kuppuswamy, SathyanarayananQuoted 7 lines · Sathyanarayanan
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
- if (!sev_active())
+ if (!sev_active() && !is_tdx_guest())
return 0;I think it's time to come up with a real name for all of the code that's
under: (sev_active()||is_tdx_guest()).
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
"encrypted" isn't it, for sure.
I called it
protected_guest()in some other patches.If you are also fine with above mentioned function name, I can include it in this series. Since we have many use cases of above condition, it will be useful define it as helper function.
FWIW, I think sev_active() has a horrible name. Shouldn't that be "is_sev_guest()"? "sev_active()" could be read as "I'm a SEV host" or "I'm a SEV guest" and "SEV is active".
protected_guest() seems fine to cover both, despite the horrid SEV naming. It'll actually be nice to banish it from appearing in many of its uses. :)
Sean Christopherson
maintainerRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
+Boris, who has similar opinions on sev_active().
Quoted 12 lines · Dave Hansen
Quoted 11 lines · Sathyanarayanan
Quoted 10 lines
Quoted 8 lines
Quoted 4 lines
- if (!sev_active())
+ if (!sev_active() && !is_tdx_guest())
return 0;I think it's time to come up with a real name for all of the code that's
under: (sev_active()||is_tdx_guest())."encrypted" isn't it, for sure.
I called it
protected_guest()in some other patches.
Quoted 7 lines
Quoted 3 lines
If you are also fine with above mentioned function name, I can include it in this series. Since we have many use cases of above condition, it will be useful define it as helper function.
FWIW, I think
sev_active()has a horrible name. Shouldn't that be "is_sev_guest()"? "sev_active()" could be read as "I'm a SEV host" or "I'm a SEV guest" and "SEV is active".
I can't find the thread offhand, but Boris proposed something along the lines of cpu_has(), but specific to a given flavor of protected guest. IIRC, it was sev_guest_has(SEV_ES) or something like that.
I 100% agree that we should have actual feature bits somewhere for the various protected guest flavors.
Quoted 3 lines
protected_guest()seems fine to cover both, despite the horrid SEV naming. It'll actually be nice to banish it from appearing in many of its uses. :)
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Dan WilliamsQuoted 3 lines · Dan Williams
There is a mix of direct-TDVMCALL usage and handling #VE when and why is either approached used?
For the really early code in the decompressor or the main kernel we can't use #VE because the IDT needed for handling the exception is not set up, and some other infrastructure needed by the handler is missing. The early code needs to do port IO to be able to write the early serial console. To keep it all common it ended up that all port IO is paravirt. Actually for most the main kernel port IO calls we could just use #VE and it would result in smaller binaries, but then we would need to annotate all early portio with some special name. That's why port IO is all TDCALL.
For some others the only thing that really has to be #VE is MMIO because we don't want to annotate every MMIO read*/write* with an alternative (which would result in incredible binary bloat) For the others they have mostly become now direct calls.
Quoted 6 lines
Quoted 3 lines
Decompression code uses port IO for earlyprintk. We must use paravirt calls there too if we want to allow earlyprintk.
What is the tradeoff between teaching the decompression code to handle #VE (the implied assumption) vs teaching it to avoid #VE with direct TDVMCALLs (the chosen direction)?
The decompression code only really needs it to output something. But you couldn't debug anything until #VE is set up. Also the decompression code has a very basic environment that doesn't supply most kernel services, and the #VE handler is relatively complicated. It would probably need to be duplicated and the instruction decoder be ported to work in this environment. It would be all a lot of work, just to make the debug output work.
Quoted 11 lines
Quoted 11 lines
Co-developed-by Kuppuswamy Sathyanarayanan <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>Signed-off-by Kirill A. Shutemov <[email protected]>Reviewed-by Andi Kleen <[email protected]>--- arch/x86/boot/compressed/Makefile | 1 + arch/x86/boot/compressed/tdcall.S | 9 ++ arch/x86/include/asm/io.h | 5 +- arch/x86/include/asm/tdx.h | 46 ++++++++- arch/x86/kernel/tdcall.S | 154 ++++++++++++++++++++++++++++++
Quoted 4 lines
Why is this named "tdcall" when it is implementing tdvmcalls? I must say those names don't really help me understand what they do. Can we have Linux names that don't mandate keeping the spec terminology in my brain's translation cache?
The instruction is called TDCALL. It's always the same instruction
TDVMCALL is the variant when the host processes it (as opposed to the TDX module), but it's just a different name space in the call number.
\
Quoted 2 lines
Is there a unified Linux name these can be given to stop the proliferation of poor vendor names for similar concepts?
We could use protected_guest()
Quoted 3 lines
Does it also not know how to handle #VE to keep it aligned with the runtime code?
Not sure I understand the question, but the decompression code supports neither alternatives nor #VE. It's a very limited environment.
Quoted 4 lines
Outside the boot decompression code isn't this branch of the "ifdef
BOOT_COMPRESSED_MISC_H" handled by #VE? I also don't see any usage of __{in,out}() in this patch.
I thought it was all alternative after decompression, so the #VE code shouldn't be called. We still have it for some reason though.
Quoted 5 lines
Perhaps "
PAYLOAD_SIZE" since it is used for both input and output?If the ABI does not include the size of the payload then how would code detect if even 80 bytes was violated in the future?
The payload in memory is just a Linux concept. At the TDCALL level it's only registers.
Quoted 5 lines
5
Surely there's an existing macro for this pattern? WouldPUSH_AND_CLEAR_REGS+POP_REGSbe suitable? Besides code sharing it would eliminate clearing of %r8.
There used to be SAVE_ALL/SAVE_REGS, but they have been all removed in some past refactorings.
-Andi
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
Quoted 7 lines · Andi Kleen
Quoted 3 lines · Dan Williams
There is a mix of direct-TDVMCALL usage and handling #VE when and why is either approached used?
Quoted 9 lines
For the really early code in the decompressor or the main kernel we can't use #VE because the IDT needed for handling the exception is not set up, and some other infrastructure needed by the handler is missing. The early code needs to do port IO to be able to write the early serial console. To keep it all common it ended up that all port IO is paravirt. Actually for most the main kernel port IO calls we could just use #VE and it would result in smaller binaries, but then we would need to annotate all early portio with some special name. That's why port IO is all TDCALL.
Thanks Andi. Sathya, please include the above in the next posting.
Quoted 38 lines
For some others the only thing that really has to be #VE is MMIO because we don't want to annotate every MMIO read*/write* with an alternative (which would result in incredible binary bloat) For the others they have mostly become now direct calls.
Quoted 6 lines
Quoted 3 lines
Decompression code uses port IO for earlyprintk. We must use paravirt calls there too if we want to allow earlyprintk.
What is the tradeoff between teaching the decompression code to handle #VE (the implied assumption) vs teaching it to avoid #VE with direct TDVMCALLs (the chosen direction)?
The decompression code only really needs it to output something. But you couldn't debug anything until #VE is set up. Also the decompression code has a very basic environment that doesn't supply most kernel services, and the #VE handler is relatively complicated. It would probably need to be duplicated and the instruction decoder be ported to work in this environment. It would be all a lot of work, just to make the debug output work.
Quoted 11 lines
Quoted 11 lines
Co-developed-by Kuppuswamy Sathyanarayanan <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>Signed-off-by Kirill A. Shutemov <[email protected]>Reviewed-by Andi Kleen <[email protected]>--- arch/x86/boot/compressed/Makefile | 1 + arch/x86/boot/compressed/tdcall.S | 9 ++ arch/x86/include/asm/io.h | 5 +- arch/x86/include/asm/tdx.h | 46 ++++++++- arch/x86/kernel/tdcall.S | 154 ++++++++++++++++++++++++++++++Quoted 4 lines
Why is this named "tdcall" when it is implementing tdvmcalls? I must say those names don't really help me understand what they do. Can we have Linux names that don't mandate keeping the spec terminology in my brain's translation cache?
Quoted 6 lines
The instruction is called TDCALL. It's always the same instruction
TDVMCALL is the variant when the host processes it (as opposed to the TDX module), but it's just a different name space in the call number.
Ok.
Quoted 6 lines
\Quoted 2 lines
Is there a unified Linux name these can be given to stop the proliferation of poor vendor names for similar concepts?
We could use
protected_guest()
Looks good.
Quoted 9 lines
Quoted 5 lines
Does it also not know how to handle #VE to keep it aligned with the runtime code?
Not sure I understand the question, but the decompression code supports neither alternatives nor #VE. It's a very limited environment.
Yes, that addresses the question.
Quoted 8 lines
Quoted 5 lines
Outside the boot decompression code isn't this branch of the "ifdef
BOOT_COMPRESSED_MISC_H" handled by #VE? I also don't see any usage of __{in,out}() in this patch.I thought it was all alternative after decompression, so the #VE code shouldn't be called. We still have it for some reason though.
Right, I'm struggling to understand where these spurious in/out instructions are coming from that are not replaced by the alternative's code? Shouldn't those be dropped on the floor and warned about rather than handled? I.e. shouldn't port-io instruction escapes that would cause #VE be precluded at build-time?
Quoted 12 lines
Quoted 7 lines
Perhaps "
PAYLOAD_SIZE" since it is used for both input and output?If the ABI does not include the size of the payload then how would code detect if even 80 bytes was violated in the future?
The payload in memory is just a Linux concept. At the TDCALL level it's only registers.
If it's only a Linux concept why does this code need to "prepare for the future"?
Quoted 9 lines
Quoted 5 lines
5
Surely there's an existing macro for this pattern? WouldPUSH_AND_CLEAR_REGS+POP_REGSbe suitable? Besides code sharing it would eliminate clearing of %r8.There used to be SAVE_ALL/SAVE_REGS, but they have been all removed in some past refactorings.
Not a huge deal, but at a minimum it seems a generic construct that deserves to be declared centrally rather than tdx-guest-port-io local.
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
Quoted 10 lines · Dan Williams
Quoted 6 lines
Quoted 3 lines
Outside the boot decompression code isn't this branch of the "ifdef
BOOT_COMPRESSED_MISC_H" handled by #VE? I also don't see any usage of __{in,out}() in this patch.I thought it was all alternative after decompression, so the #VE code shouldn't be called. We still have it for some reason though.
Right, I'm struggling to understand where these spurious in/out instructions are coming from that are not replaced by the alternative's code?
There should be nothing in the main tree at least.
Quoted 2 lines
Shouldn't those be dropped on the floor and warned about rather than handled?
It might be related to eventually handling them in ring 3, but I believe we disallow that currently too and it's not all that useful anyways. So yes it could be forbidden.
Quoted 2 lines
I.e. shouldn't port-io instruction escapes
that would cause #VE be precluded at build-time?
You mean in objtool? That would seem like overkill for a more theoretical problem.
Quoted 5 lines
Quoted 2 lines
There used to be SAVE_ALL/SAVE_REGS, but they have been all removed in some past refactorings.
Not a huge deal, but at a minimum it seems a generic construct that deserves to be declared centrally rather than tdx-guest-port-io local.
Yes I agree. We should just bring SAVE_ALL/SAVE_REGS back.
-Andi
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
On Mon, May 10, 2021 at 5:01 PM Andi Kleen <ak@lin [..]
Quoted 5 lines
Quoted 2 lines
I.e. shouldn't port-io instruction escapes
that would cause #VE be precluded at build-time?You mean in objtool? That would seem like overkill for a more theoretical problem.
Oh, sorry, no, I was not implying objtool overkill, just that the mainline kernel should not be surprised by spurious instruction usage.
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Dan WilliamsQuoted 18 lines · Dan Williams
Quoted 7 lines · Andi Kleen
Quoted 3 lines · Dan Williams
There is a mix of direct-TDVMCALL usage and handling #VE when and why is either approached used?
Quoted 9 lines
For the really early code in the decompressor or the main kernel we can't use #VE because the IDT needed for handling the exception is not set up, and some other infrastructure needed by the handler is missing. The early code needs to do port IO to be able to write the early serial console. To keep it all common it ended up that all port IO is paravirt. Actually for most the main kernel port IO calls we could just use #VE and it would result in smaller binaries, but then we would need to annotate all early portio with some special name. That's why port IO is all TDCALL.
Quoted 1 line
Thanks Andi. Sathya, please include the above in the next posting.
Will include it.
Quoted 93 lines
Quoted 39 lines
For some others the only thing that really has to be #VE is MMIO because we don't want to annotate every MMIO read*/write* with an alternative (which would result in incredible binary bloat) For the others they have mostly become now direct calls.
Quoted 6 lines
Quoted 3 lines
Decompression code uses port IO for earlyprintk. We must use paravirt calls there too if we want to allow earlyprintk.
What is the tradeoff between teaching the decompression code to handle #VE (the implied assumption) vs teaching it to avoid #VE with direct TDVMCALLs (the chosen direction)?
The decompression code only really needs it to output something. But you couldn't debug anything until #VE is set up. Also the decompression code has a very basic environment that doesn't supply most kernel services, and the #VE handler is relatively complicated. It would probably need to be duplicated and the instruction decoder be ported to work in this environment. It would be all a lot of work, just to make the debug output work.
Quoted 11 lines
Quoted 11 lines
Co-developed-by Kuppuswamy Sathyanarayanan <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>Signed-off-by Kirill A. Shutemov <[email protected]>Reviewed-by Andi Kleen <[email protected]>--- arch/x86/boot/compressed/Makefile | 1 + arch/x86/boot/compressed/tdcall.S | 9 ++ arch/x86/include/asm/io.h | 5 +- arch/x86/include/asm/tdx.h | 46 ++++++++- arch/x86/kernel/tdcall.S | 154 ++++++++++++++++++++++++++++++Quoted 4 lines
Why is this named "tdcall" when it is implementing tdvmcalls? I must say those names don't really help me understand what they do. Can we have Linux names that don't mandate keeping the spec terminology in my brain's translation cache?
Quoted 6 lines
The instruction is called TDCALL. It's always the same instruction
TDVMCALL is the variant when the host processes it (as opposed to the TDX module), but it's just a different name space in the call number.
Ok.
Quoted 6 lines
\Quoted 2 lines
Is there a unified Linux name these can be given to stop the proliferation of poor vendor names for similar concepts?
We could use
protected_guest()Looks good.
Quoted 9 lines
Quoted 5 lines
Does it also not know how to handle #VE to keep it aligned with the runtime code?
Not sure I understand the question, but the decompression code supports neither alternatives nor #VE. It's a very limited environment.
Yes, that addresses the question.
Quoted 8 lines
Quoted 5 lines
Outside the boot decompression code isn't this branch of the "ifdef
BOOT_COMPRESSED_MISC_H" handled by #VE? I also don't see any usage of __{in,out}() in this patch.I thought it was all alternative after decompression, so the #VE code shouldn't be called. We still have it for some reason though.
Right, I'm struggling to understand where these spurious in/out instructions are coming from that are not replaced by the alternative's code? Shouldn't those be dropped on the floor and warned about rather than handled? I.e. shouldn't port-io instruction escapes that would cause #VE be precluded at build-time?
Quoted 9 lines
Quoted 7 lines
Perhaps "
PAYLOAD_SIZE" since it is used for both input and output?If the ABI does not include the size of the payload then how would code detect if even 80 bytes was violated in the future?
Quoted 6 lines
Quoted 3 lines
The payload in memory is just a Linux concept. At the TDCALL level it's only registers.
If it's only a Linux concept why does this code need to "prepare for the future"?
It is the software only structure. It is created to group all the output registers used by VMM. You can find more details about it in patch titled # "[RFC v2 05/32] x86/tdx: Add __tdcall() and __tdvmcall() helper functions"
It is mainly used by functions like __tdx_hypercall(),__tdx_hypercall_vendor_kvm() and tdx_in{b,w,l}.
u64 __tdx_hypercall(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15,
struct tdx_hypercall_output *out);
u64 __tdx_hypercall_vendor_kvm(u64 fn, u64 r12, u64 r13, u64 r14,
u64 r15, struct tdx_hypercall_output *out);struct tdx_hypercall_output {
u64 r11;
u64 r12;
u64 r13;
u64 r14;
u64 r15;
};Functions like __tdx_hypercall() and __tdx_hypercall_vendor_kvm() are used by TDX guest to request services (like RDMSR, WRMSR,GetQuote, etc) from VMM using TDCALL instruction. do_tdx_hypercall() is the helper function (in tdcall.S) which actually implements this ABI.
As per current ABI, VMM will use registers R11-R15 to share the output values with the guest. So we have defined the structure
struct tdx_hypercall_output to group all output registers and make it easier to share it with users of the TDCALLs. This is Linux defined structure.
If there are any changes in TDCALL ABI for VMM, we might have to extend this structure to accommodate new output register changes. So if we define TDVMCALL_OUTPUT_SIZE as 40, we will have modify this value for any future struct tdx_hypercall_output changes. So to avoid it, we have allocated double the size.
May be I should define it as,
#define TDVMCALL_OUTPUT_SIZE sizeof(struct tdx_hypercall_output)
But currently we don't include the asm/tdx.h (which defines struct tdx_hypercall_output) in tdcall.S. So I have defined the size as constant value.
Quoted 15 lines
Quoted 11 lines
Quoted 7 lines
5
Surely there's an existing macro for this pattern? WouldPUSH_AND_CLEAR_REGS+POP_REGSbe suitable? Besides code sharing it would eliminate clearing of %r8.There used to be SAVE_ALL/SAVE_REGS, but they have been all removed in some past refactorings.
Not a huge deal, but at a minimum it seems a generic construct that deserves to be declared centrally rather than tdx-guest-port-io local.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Dan WilliamsQuoted 8 lines · Dan Williams
Quoted 6 lines
Quoted 3 lines
Surely there's an existing macro for this pattern? Would
PUSH_AND_CLEAR_REGS+POP_REGSbe suitable? Besides code sharing it would eliminate clearing of %r8.There used to be SAVE_ALL/SAVE_REGS, but they have been all removed in some past refactorings.
Not a huge deal, but at a minimum it seems a generic construct that deserves to be declared centrally rather than tdx-guest-port-io local.
I can define SAVE_ALL_REGS/RESTORE_ALL_REGS. Do you want to move it outside TDX code? I don't know if there will be other users for it?
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Kuppuswamy, SathyanarayananOn Mon, May 10, 2021 at 5:30 PM Kuppuswamy, Sathyanarayanan <[email protected]> wrote:
[..]
Quoted 8 lines
It is mainly used by functions like __tdx_hypercall(),__tdx_hypercall_vendor_kvm() and tdx_in{b,w,l}.
u64 __tdx_hypercall(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15,
struct tdx_hypercall_output *out); u64 __tdx_hypercall_vendor_kvm(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15, struct tdx_hypercall_output *out);
Quoted 7 lines
struct tdx_hypercall_output {
u64 r11;
u64 r12;
u64 r13;
u64 r14;
u64 r15;
};Why is this by register name and not something like:
struct tdx_hypercall_payload {
u64 data[5];
};
...because the code in this patch is reading the payload out of a stack relative offset, not r11.
Quoted 9 lines
Functions like
__tdx_hypercall()and__tdx_hypercall_vendor_kvm()are used by TDX guest to request services (like RDMSR, WRMSR,GetQuote, etc) from VMM using TDCALL instruction.do_tdx_hypercall()is the helper function (intdcall.S) which actually implements this ABI.As per current ABI, VMM will use registers R11-R15 to share the output values with the guest.
Which ABI, __tdx_hypercall_vendor_kvm()? The code is putting the payload on the stack, so I'm not sure what ABI you are referring to?
Quoted 10 lines
So we have defined the structure
structtdx_hypercall_outputto group all output registers and make it easier to share it with users of the TDCALLs. This is Linux defined structure.If there are any changes in TDCALL ABI for VMM, we might have to extend this structure to accommodate new output register changes. So if we define
TDVMCALL_OUTPUT_SIZEas 40, we will have modify this value for any future structtdx_hypercall_outputchanges. So to avoid it, we have allocated double the size.
Quoted 3 lines
May be I should define it as,
#define
TDVMCALL_OUTPUT_SIZEsizeof(struct tdx_hypercall_output)
An arrangement like that seems more reasonable than a seemingly arbitrary number and an ominous warning about things that may happen in the future.
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
In reply to Kuppuswamy SathyanarayananQuoted 10 lines · Kuppuswamy Sathyanarayanan
When running as a TDX guest, there are a number of existing, privileged instructions that do not work. If the guest kernel uses these instructions, the hardware generates a #VE.
You can find the list of unsupported instructions in Intel Trust Domain Extensions (Intel® TDX) Module specification, sec 9.2.2 and in Guest-Host Communication Interface (GHCI) Specification for Intel TDX, sec 2.4.1.
Ah, better than the "handle port io" patch, these details at least give the reader a chance.
Quoted 4 lines
To prevent TD guest from using MWAIT/MONITOR instructions, support for these instructions are already disabled by TDX module (SEAM). So CPUID flags for these instructions should be in disabled state.
Why does this not result in a #UD if the instruction is disabled by SEAM? How is it possible to execute a disabled instruction (one precluded by CPUID) to the point where it triggers #VE instead of #UD?
Quoted 6 lines
After the above mentioned preventive measures, if TD guests still execute these instructions, add appropriate warning messages in #VE handler. For WBIND instruction, since it's related to memory writeback and cache flushes, it's mainly used in context of IO devices. Since TDX 1.0 does not support non-virtual I/O devices, skipping it should not cause any fatal issues.
WBINVD is in a different class than MWAIT/MONITOR since it is not identified by CPUID, it can't possibly have the same #UD behaviour. It's not clear why WBINVD is included in the same patch as MWAIT/MONITOR?
I disagree with the assertion that WBINVD is mainly used in the context of I/O devices, it's also used for ACPI power management paths. WBINVD dependent functionality should be dynamically disabled rather than warned about.
Does a TDX guest support out-of-tree modules? The kernel is already tainted when out-of-tree modules are loaded. In other words in-tree modules preclude forbidden instructions because they can just be audited, and out-of-tree modules are ok to trigger abrupt failure if they attempt to use forbidden instructions.
Quoted 3 lines
But to let users know about its usage, use
WARN()to report about it.. For MWAIT/MONITOR instruction, since its unsupported useWARN()to report unsupported usage.
I'm not sure how useful warning is outside of a kernel developer's debug environment. The kernel should know what instructions are disabled and which are available. WBINVD in particular has potential data integrity implications. Code that might lead to a WBINVD usage should be disabled, not run all the way up to where WBINVD is attempted and then trigger an after-the-fact WARN_ONCE().
The WBINVD change deserves to be split off from MWAIT/MONITOR, and more thought needs to be put into where these spurious instruction usages are arising.
Quoted 36 lines
Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>Reviewed-by Andi Kleen <[email protected]>--- arch/x86/kernel/tdx.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)arch/x86/kernel/tdx.c +15
-1@@ -371,6 +371,21 @@ int tdg_handle_virtualization_exception(struct pt_regs *regs,case EXIT_REASON_EPT_VIOLATION : ve->instr_len =tdg_handle_mmio (regs, ve);break ; +case EXIT_REASON_WBINVD : +/* +* WBINVD is not supported inside TDX guests. All in- +* kernel uses should have been disabled. +*/ +WARN_ONCE (1 ,"TD Guest used unsupported WBINVD instruction\n" ); +break ; +case EXIT_REASON_MONITOR_INSTRUCTION : +case EXIT_REASON_MWAIT_INSTRUCTION : +/* +* Something in the kernel used MONITOR or MWAIT despite +* X86_FEATURE_MWAIT being cleared for TDX guests. +*/ +WARN_ONCE (1 ,"TD Guest used unsupported MWAIT/MONITOR instruction\n" ); +break ;default :pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason);return -EFAULT ; --2.25.1
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
Quoted 6 lines
Quoted 4 lines
To prevent TD guest from using MWAIT/MONITOR instructions, support for these instructions are already disabled by TDX module (SEAM). So CPUID flags for these instructions should be in disabled state.
Why does this not result in a #UD if the instruction is disabled by SEAM?
It's just the TDX module (SEAM is the execution mode used by the TDX module)
Quoted 2 lines
How is it possible to execute a disabled instruction (one precluded by CPUID) to the point where it triggers #VE instead of #UD?
That's how the TDX module works. It never injects anything else other than #VE. You can still get other exceptions of course, but they won't come from the TDX module.
Quoted 10 lines
Quoted 6 lines
After the above mentioned preventive measures, if TD guests still execute these instructions, add appropriate warning messages in #VE handler. For WBIND instruction, since it's related to memory writeback and cache flushes, it's mainly used in context of IO devices. Since TDX 1.0 does not support non-virtual I/O devices, skipping it should not cause any fatal issues.
WBINVD is in a different class than MWAIT/MONITOR since it is not identified by CPUID, it can't possibly have the same #UD behaviour. It's not clear why WBINVD is included in the same patch as MWAIT/MONITOR?
Because these are all instructions we never expect to execute, so nothing special is needed for them. That's a unique class that logically fits together.
Quoted 4 lines
I disagree with the assertion that WBINVD is mainly used in the context of I/O devices, it's also used for ACPI power management paths.
You mean S3? That's of course also not supported inside TDX.
Quoted 8 lines
WBINVD dependent functionality should be dynamically disabled
rather than warned about.Does a TDX guest support out-of-tree modules? The kernel is already tainted when out-of-tree modules are loaded. In other words in-tree modules preclude forbidden instructions because they can just be audited, and out-of-tree modules are ok to trigger abrupt failure if they attempt to use forbidden instructions.
We already did a lot of bi^wdiscussion on this on the last review.
Originally we had a different handling, this was the result of previous feedback.
It doesn't really matter because it should never happen.
Quoted 10 lines
Quoted 4 lines
But to let users know about its usage, use
WARN()to report about it.. For MWAIT/MONITOR instruction, since its unsupported useWARN()to report unsupported usage.I'm not sure how useful warning is outside of a kernel developer's debug environment. The kernel should know what instructions are disabled and which are available. WBINVD in particular has potential data integrity implications. Code that might lead to a WBINVD usage should be disabled, not run all the way up to where WBINVD is attempted and then trigger an after-the-fact
WARN_ONCE().
We don't expect the warning to ever happen. Yes all of this will be disabled. Nearly all are in code paths that cannot happen inside TDX anyways due to missing PCI-IDs or different cpuids, and S3 is explicitly disabled and would be impossible anyways due to lack of BIOS support.
Quoted 4 lines
The WBINVD change deserves to be split off from MWAIT/MONITOR, and more thought needs to be put into where these spurious instruction usages are arising.
I disagree. We already spent a lot of cycles on this. WBINVD makes never sense in current TDX and all the code will be disabled.
-Andi
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Kuppuswamy, SathyanarayananQuoted 7 lines · Sathyanarayanan
Quoted 4 lines · Dan Williams
Quoted 4 lines
Quoted 3 lines
Surely there's an existing macro for this pattern? Would
PUSH_AND_CLEAR_REGS+POP_REGSbe suitable? Besides code sharing it would eliminate clearing of %r8.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
There used to be SAVE_ALL/SAVE_REGS, but they have been all removed in some past refactorings.
Not a huge deal, but at a minimum it seems a generic construct that deserves to be declared centrally rather than tdx-guest-port-io local.
I can define SAVE_ALL_REGS/RESTORE_ALL_REGS. Do you want to move it outside TDX code? I don't know if there will be other users for it?
The old name was SAVE_ALL / SAVE_REGS.
Yes please put it outside tdx code into some include file.
-Andi
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Dan WilliamsQuoted 19 lines · Dan Williams
On Mon, May 10, 2021 at 5:30 PM Kuppuswamy, Sathyanarayanan <[email protected]> wrote:
[..]Quoted 8 lines
It is mainly used by functions like __tdx_hypercall(),__tdx_hypercall_vendor_kvm() and tdx_in{b,w,l}.
u64 __tdx_hypercall(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15,
struct tdx_hypercall_output *out); u64 __tdx_hypercall_vendor_kvm(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15, struct tdx_hypercall_output *out);Quoted 7 lines
struct tdx_hypercall_output { u64 r11; u64 r12; u64 r13; u64 r14; u64 r15; };
Quoted 8 lines
Why is this by register name and not something like:
struct tdx_hypercall_payload { u64 data[5]; };...because the code in this patch is reading the payload out of a stack relative offset, not r11.
Since this patch allocates this memory in ASM code, we read it via offset. If you see other use cases in tdx.c, you will notice the use of register names.
static void tdg_handle_cpuid(struct pt_regs *regs)
{
u64 ret;
struct tdx_hypercall_output out = {0};
ret = __tdx_hypercall(EXIT_REASON_CPUID, regs->ax,
regs->cx, 0, 0, &out);
WARN_ON(ret);
regs->ax = out.r12;
regs->bx = out.r13;
regs->cx = out.r14;
regs->dx = out.r15;
}static u64 tdg_read_msr_safe(unsigned int msr, int *err)
{
u64 ret;
struct tdx_hypercall_output out = {0};
WARN_ON_ONCE(tdg_is_context_switched_msr(msr));
/*
* Since CSTAR MSR is not used by Intel CPUs as SYSCALL
* instruction, just ignore it. Even raising TDVMCALL
* will lead to same result.
*/
if (msr == MSR_CSTAR)
return 0;
ret = __tdx_hypercall(EXIT_REASON_MSR_READ, msr, 0, 0, 0, &out);
*err = ret ? -EIO : 0;
return out.r11;
}Quoted 8 lines
Quoted 8 lines
Functions like
__tdx_hypercall()and__tdx_hypercall_vendor_kvm()are used by TDX guest to request services (like RDMSR, WRMSR,GetQuote, etc) from VMM using TDCALL instruction.do_tdx_hypercall()is the helper function (intdcall.S) which actually implements this ABI.
Quoted 4 lines
Quoted 2 lines
As per current ABI, VMM will use registers R11-R15 to share the output values with the guest.
Which ABI,
TDCALL ABI (see sections 3.1 to 3.12 and look for Output Operands in each TDVMCALL variant).
__tdx_hypercall_vendor_kvm()? The code is putting the
Quoted 13 lines
payload on the stack, so I'm not sure what ABI you are referring to?
Quoted 10 lines
So we have defined the structure
structtdx_hypercall_outputto group all output registers and make it easier to share it with users of the TDCALLs. This is Linux defined structure.If there are any changes in TDCALL ABI for VMM, we might have to extend this structure to accommodate new output register changes. So if we define
TDVMCALL_OUTPUT_SIZEas 40, we will have modify this value for any future structtdx_hypercall_outputchanges. So to avoid it, we have allocated double the size.
Quoted 7 lines
Quoted 3 lines
May be I should define it as,
#define
TDVMCALL_OUTPUT_SIZEsizeof(struct tdx_hypercall_output)An arrangement like that seems more reasonable than a seemingly arbitrary number and an ominous warning about things that may happen in the future.
I will use the above format.
Quoted 1 line
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
In reply to Andi KleenQuoted 8 lines · Andi Kleen
Quoted 6 lines
Quoted 4 lines
To prevent TD guest from using MWAIT/MONITOR instructions, support for these instructions are already disabled by TDX module (SEAM). So CPUID flags for these instructions should be in disabled state.
Why does this not result in a #UD if the instruction is disabled by SEAM?
It's just the TDX module (SEAM is the execution mode used by the TDX module)
If it is disabled by the TDX Module, we should never execute it. But for some reason, if we still come across this instruction (buggy TDX module?), we add appropriate warning in #VE handler.
Quoted 13 lines
Quoted 4 lines
How is it possible to execute a disabled instruction (one precluded by CPUID) to the point where it triggers #VE instead of #UD?
That's how the TDX module works. It never injects anything else other than #VE. You can still get other exceptions of course, but they won't come from the TDX module.
Quoted 6 lines
Quoted 6 lines
After the above mentioned preventive measures, if TD guests still execute these instructions, add appropriate warning messages in #VE handler. For WBIND instruction, since it's related to memory writeback and cache flushes, it's mainly used in context of IO devices. Since TDX 1.0 does not support non-virtual I/O devices, skipping it should not cause any fatal issues.
Quoted 6 lines
Quoted 4 lines
WBINVD is in a different class than MWAIT/MONITOR since it is not identified by CPUID, it can't possibly have the same #UD behaviour. It's not clear why WBINVD is included in the same patch as MWAIT/MONITOR?
Because these are all instructions we never expect to execute, so nothing special is needed for them. That's a unique class that logically fits together.
Yes, for all these three instruction we don't need any special handling code. So they are grouped together.
Quoted 30 lines
Quoted 6 lines
I disagree with the assertion that WBINVD is mainly used in the context of I/O devices, it's also used for ACPI power management paths.
You mean S3? That's of course also not supported inside TDX.
Quoted 8 lines
WBINVD dependent functionality should be dynamically disabled rather than warned about.
Does a TDX guest support out-of-tree modules? The kernel is already tainted when out-of-tree modules are loaded. In other words in-tree modules preclude forbidden instructions because they can just be audited, and out-of-tree modules are ok to trigger abrupt failure if they attempt to use forbidden instructions.
We already did a lot of bi^wdiscussion on this on the last review.
Originally we had a different handling, this was the result of previous feedback.
It doesn't really matter because it should never happen.
Quoted 4 lines
Quoted 4 lines
But to let users know about its usage, use
WARN()to report about it.. For MWAIT/MONITOR instruction, since its unsupported useWARN()to report unsupported usage.
Quoted 8 lines
Quoted 6 lines
I'm not sure how useful warning is outside of a kernel developer's debug environment. The kernel should know what instructions are disabled and which are available. WBINVD in particular has potential data integrity implications. Code that might lead to a WBINVD usage should be disabled, not run all the way up to where WBINVD is attempted and then trigger an after-the-fact
WARN_ONCE().We don't expect the warning to ever happen. Yes all of this will be disabled. Nearly all are in code paths that cannot happen inside TDX anyways due to missing PCI-IDs or different cpuids, and S3 is explicitly disabled and would be impossible anyways due to lack of BIOS support.
We have added WARN to let user know about its usage and fix it. By default we should never hit this path.
Quoted 15 lines
Quoted 8 lines
The WBINVD change deserves to be split off from MWAIT/MONITOR, and more thought needs to be put into where these spurious instruction usages are arising.
I disagree. We already spent a lot of cycles on this. WBINVD makes never sense in current TDX and all the code will be disabled.
-Andi
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
Quoted 7 lines · Sathyanarayanan
Quoted 4 lines · Andi Kleen
Quoted 4 lines
Quoted 4 lines
To prevent TD guest from using MWAIT/MONITOR instructions, support for these instructions are already disabled by TDX module (SEAM). So CPUID flags for these instructions should be in disabled state.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Why does this not result in a #UD if the instruction is disabled by SEAM?
It's just the TDX module (SEAM is the execution mode used by the TDX module)
If it is disabled by the TDX Module, we should never execute it. But for some reason, if we still come across this instruction (buggy TDX module?), we add appropriate warning in #VE handler.
I think the only case where it could happen is if the kernel jumps to a random address due to a bug and the destination happens to be these instruction bytes. Of course it is exceedingly unlikely.
Or we make some mistake, but that's hopefully fixed quickly.
-Andi
Borislav Petkov
maintainerRe: [RFC v2 28/32] x86/tdx: Make pages shared in ioremap()
In reply to Sean ChristophersonQuoted 6 lines · Sean Christopherson
I can't find the thread offhand, but Boris proposed something along the lines of
cpu_has(), but specific to a given flavor of protected guest. IIRC, it wassev_guest_has(SEV_ES)or something like that.I 100% agree that we should have actual feature bits somewhere for the various protected guest flavors.
Preach brother! :)
/me goes and greps mailboxes...
ah, do you mean this, per chance:
https://lore.kernel.org/kvm/[email protected]/
?
And yes, this has "sev" in the name and dhansen makes sense to me in wishing to unify all the protected guest feature queries under a common name. And then depending on the vendor, that common name will call the respective vendor's helper to answer the protected guest aspect asked about.
This way, generic code will call
protected_guest_has()
or so and be nicely abstracted away from the underlying implementation.
Hohumm, yap, sounds nice to me.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
In reply to Dan WilliamsQuoted 7 lines · Dan Williams
Quoted 4 lines
To prevent TD guest from using MWAIT/MONITOR instructions, support for these instructions are already disabled by TDX module (SEAM). So CPUID flags for these instructions should be in disabled state.
Why does this not result in a #UD if the instruction is disabled by SEAM? How is it possible to execute a disabled instruction (one precluded by CPUID) to the point where it triggers #VE instead of #UD?
This is actually a vestige of VMX. It's quite possible toady to have a feature which isn't enumerated in CPUID which still exists and "works" in the silicon. There are all kinds of pitfalls to doing this, but folks evidently do it in public clouds all the time.
The CPUID virtualization basically just traps into the hypervisor and lets the hypervisor set whatever register values it wants to appear when CPUID "returns".
But, the controls for what instructions generate #UD are actually quite separate and unrelated to CPUID itself.
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Kuppuswamy, SathyanarayananQuoted 28 lines · Sathyanarayanan
Quoted 27 lines · Dan Williams
On Mon, May 10, 2021 at 5:30 PM Kuppuswamy, Sathyanarayanan <[email protected]> wrote:
[..]Quoted 10 lines
It is mainly used by functions like
__tdx_hypercall(),__tdx_hypercall_vendor_kvm()
and tdx_in{b,w,l}.u64
__tdx_hypercall(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15, struct tdx_hypercall_output *out); u64__tdx_hypercall_vendor_kvm(u64 fn, u64 r12, u64 r13, u64 r14, u64 r15, struct tdx_hypercall_output *out);Quoted 7 lines
struct
tdx_hypercall_output{
u64 r11;
u64 r12;
u64 r13;
u64 r14;
u64 r15;
};Why is this by register name and not something like:
struct
tdx_hypercall_payload{
u64 data[5];
};
Quoted 6 lines
Quoted 2 lines
...because the code in this patch is reading the payload out of a stack relative offset, not r11.
Since this patch allocates this memory in ASM code, we read it via offset. If you see other use cases in
tdx.c, you will notice the use of register names.
To what you do you refer by "this patch allocates this memory in ASM code"? Could you point to the specific ASM code that "allocates memory"?
Dan I'll try to answer your question. TDX has both a "hypercall" interface for guests to call into hosts and a "seamcall" interface where guests or hosts can talk to the TDX/SEAM module.
Both of these represent an ABI which _resembles_ a system call ABI. Values are placed in registers, including a "function" register which is very similar to a the system call number we place in RAX.
*But* those ABIs was actually designed to (IIRC) resemble the Windows/Microsoft ABI, not the Linux ABI. So the register conventions are unfamiliar. There is assembly code to convert between the ELF function call ABI and the TDX ABIs.
For instance, if you are in C code and you call:
__tdx_hypercall_vendor_kvm(u64 fn, u64 r12, ...
The value for "fn" will be placed in RAX and "r12" will be placed in RDI for the function call itself. The assembly code will, for instance, take the "r12" *VARIABLE* and ensure it gets into the R12 *REGISTER* for the hypercall.
The same thing happens on the output side. The TDX ABIs specify "return" values in certain registers (r11-r15). However, those registers are not preserved in our function return ABI. So, they must be stashed off in memory into a place where the caller can retrieve them.
Rather than being unstructured "data[]", the value in tdx_hypercall_output->r11 was actually in register R11 at some point. If you look at the spec, you can see the functions that use R11.
Let's say there's a hypercall to check for whether puppies are cute. Here's the kernel side:
bool tdx_hypercall_puppies_are_cute()
{
struct tdx_hypercall_output out;
u64 ret;
ret = __tdx_hypercall_vendor_kvm(HOST_LIKES_PUPPIES, ..., &out);
/* Did the hypercall even succeed? */
if (ret != SUCCESS)
return -EINVAL;
if (out->r11 == TDX_WHATEVER_CUTE_BIT)
return true;
// Nope, I guess puppies are not cute
return false;
}The spec would actually say, "Blah blah, puppies are cute if TDX_WHATEVER_CUTE_BIT is set in r11". So, this whole setup actually results in really nice C code that you can sit side-by-side with the spec and see if they agree.
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
Quoted 2 lines · Dave Hansen
To what you do you refer by "this patch allocates this memory in ASM code"? Could you point to the specific ASM code that "allocates memory"?
We use 40 bytes in stack for storing the output register values. It is in function tdg_inl().
subq $TDVMCALL_OUTPUT_SIZE, %rsp
+SYM_FUNC_START(tdg_inl)
+ io_save_registers
+ /* Set data width to 4 bytes */
+ mov $4, %rsi
+1:
+ mov %rdx, %rcx
+ /* Set 0 in RDX to select in operation */
+ mov $0, %rdx
+ /* Set TDVMCALL function id in RDI */
+ mov $EXIT_REASON_IO_INSTRUCTION, %rdi
+ /* Set TDVMCALL type info (0 - Standard, > 0 - vendor) in R10 */
+ xor %r10, %r10
+ /* Allocate memory in stack for Output */
+ subq $TDVMCALL_OUTPUT_SIZE, %rsp
+ /* Move tdvmcall_output pointer to R9 */
+ movq %rsp, %r9
+
+ call do_tdvmcall
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Dan WilliamsQuoted 5 lines · Dan Williams
Quoted 2 lines
Decompression code uses port IO for earlyprintk. We must use paravirt calls there too if we want to allow earlyprintk.
What is the tradeoff between teaching the decompression code to handle #VE (the implied assumption) vs teaching it to avoid #VE with direct TDVMCALLs (the chosen direction)?
To me, the tradeoff is not just "teaching" the code to handle a #VE, but ensuring that the entire architecture works.
Intentionally invoking a #VE is like making a function call that *MIGHT* recurse on itself. Sure, you can try to come up with a story about bounding the recursion. But, I don't see any semblance of that in this series.
Exception-based recursion is really nasty because it's implicit, not explicit. That's why I'm advocating for a design where the kernel never intentionally causes a #VE: it never intentionally recurses without bounds.
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
In reply to Andi KleenQuoted 79 lines · Andi Kleen
Quoted 7 lines
Quoted 5 lines
To prevent TD guest from using MWAIT/MONITOR instructions, support for these instructions are already disabled by TDX module (SEAM). So CPUID flags for these instructions should be in disabled state.
Why does this not result in a #UD if the instruction is disabled by SEAM?
It's just the TDX module (SEAM is the execution mode used by the TDX module)
Quoted 2 lines
How is it possible to execute a disabled instruction (one precluded by CPUID) to the point where it triggers #VE instead of #UD?
That's how the TDX module works. It never injects anything else other than #VE. You can still get other exceptions of course, but they won't come from the TDX module.
Quoted 10 lines
Quoted 6 lines
After the above mentioned preventive measures, if TD guests still execute these instructions, add appropriate warning messages in #VE handler. For WBIND instruction, since it's related to memory writeback and cache flushes, it's mainly used in context of IO devices. Since TDX 1.0 does not support non-virtual I/O devices, skipping it should not cause any fatal issues.
WBINVD is in a different class than MWAIT/MONITOR since it is not identified by CPUID, it can't possibly have the same #UD behaviour. It's not clear why WBINVD is included in the same patch as MWAIT/MONITOR?
Because these are all instructions we never expect to execute, so nothing special is needed for them. That's a unique class that logically fits together.
Quoted 4 lines
I disagree with the assertion that WBINVD is mainly used in the context of I/O devices, it's also used for ACPI power management paths.
You mean S3? That's of course also not supported inside TDX.
Quoted 8 lines
WBINVD dependent functionality should be dynamically disabled
rather than warned about.Does a TDX guest support out-of-tree modules? The kernel is already tainted when out-of-tree modules are loaded. In other words in-tree modules preclude forbidden instructions because they can just be audited, and out-of-tree modules are ok to trigger abrupt failure if they attempt to use forbidden instructions.
We already did a lot of bi^wdiscussion on this on the last review.
Originally we had a different handling, this was the result of previous feedback.
It doesn't really matter because it should never happen.
Quoted 10 lines
Quoted 4 lines
But to let users know about its usage, use
WARN()to report about it.. For MWAIT/MONITOR instruction, since its unsupported useWARN()to report unsupported usage.I'm not sure how useful warning is outside of a kernel developer's debug environment. The kernel should know what instructions are disabled and which are available. WBINVD in particular has potential data integrity implications. Code that might lead to a WBINVD usage should be disabled, not run all the way up to where WBINVD is attempted and then trigger an after-the-fact
WARN_ONCE().We don't expect the warning to ever happen. Yes all of this will be disabled. Nearly all are in code paths that cannot happen inside TDX anyways due to missing PCI-IDs or different cpuids, and S3 is explicitly disabled and would be impossible anyways due to lack of BIOS support.
Quoted 1 line
Quoted 6 lines
Quoted 3 lines
The WBINVD change deserves to be split off from MWAIT/MONITOR, and more thought needs to be put into where these spurious instruction usages are arising.
I disagree. We already spent a lot of cycles on this. WBINVD makes never sense in current TDX and all the code will be disabled.
Why not just drop the patch if it continues to cause people to spend cycles on it and it addresses a problem that will never happen?
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
Quoted 3 lines · Dan Williams
O
Why not just drop the patch if it continues to cause people to spend cycles on it and it addresses a problem that will never happen?
We want to at least get some kind of warning if there is really a mistake. Just dropping such an ability wouldn't seem right.
That's all that the patch does really.
-Andi
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Dave HansenQuoted 11 lines · Dave Hansen
Quoted 5 lines · Dan Williams
Quoted 2 lines
Decompression code uses port IO for earlyprintk. We must use paravirt calls there too if we want to allow earlyprintk.
What is the tradeoff between teaching the decompression code to handle #VE (the implied assumption) vs teaching it to avoid #VE with direct TDVMCALLs (the chosen direction)?
To me, the tradeoff is not just "teaching" the code to handle a #VE, but ensuring that the entire architecture works.
Quoted 8 lines
Intentionally invoking a #VE is like making a function call that *MIGHT* recurse on itself. Sure, you can try to come up with a story about bounding the recursion. But, I don't see any semblance of that in this series.
Exception-based recursion is really nasty because it's implicit, not explicit. That's why I'm advocating for a design where the kernel never intentionally causes a #VE: it never intentionally recurses without bounds.
Thanks Dave, this really helps.
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
In reply to Dan WilliamsQuoted 4 lines · Dan Williams
Quoted 2 lines
I disagree. We already spent a lot of cycles on this. WBINVD makes never sense in current TDX and all the code will be disabled.
Why not just drop the patch if it continues to cause people to spend cycles on it and it addresses a problem that will never happen?
If someone calls WBINVD, we have a bug. Not a little bug, either. It probably means there's some horribly confused kernel code that's now facing broken cache coherency. To me, it's a textbook place to use BUG_ON().
This also doesn't "address" the problem, it just helps produce a more coherent warning message. It's why we have OOPS messages in the page fault handler: it never makes any sense to dereference a NULL pointer, yet we have code to make debugging them easier. It's well worth the ~20 lines of code that this costs us for ease of debugging.
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
Quoted 12 lines · Dave Hansen
Quoted 4 lines · Dan Williams
Quoted 2 lines
I disagree. We already spent a lot of cycles on this. WBINVD makes never sense in current TDX and all the code will be disabled.
Why not just drop the patch if it continues to cause people to spend cycles on it and it addresses a problem that will never happen?
If someone calls WBINVD, we have a bug. Not a little bug, either. It probably means there's some horribly confused kernel code that's now facing broken cache coherency. To me, it's a textbook place to use
BUG_ON().
Quoted 5 lines
This also doesn't "address" the problem, it just helps produce a more coherent warning message. It's why we have OOPS messages in the page fault handler: it never makes any sense to dereference a NULL pointer, yet we have code to make debugging them easier. It's well worth the ~20 lines of code that this costs us for ease of debugging.
The 'default' case in this 'switch' prints the exit reason and faults, can't that also trigger a backtrace that dumps the exception stack and the faulting instruction? In other words shouldn't this just fail with a common way to provide better debug on any unhandled #VE and not try to continue running past something that "can't" happen?
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
Quoted 6 lines
The 'default' case in this 'switch' prints the exit reason and faults, can't that also trigger a backtrace that dumps the exception stack and the faulting instruction? In other words shouldn't this just fail with a common way to provide better debug on any unhandled #VE and not try to continue running past something that "can't" happen?
It will use the #GP common code which will do all the backtracing etc.
We didn't think we would need anything else than what #GP already does.
-Andi
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
In reply to Kuppuswamy SathyanarayananQuoted 1 line · Kuppuswamy Sathyanarayanan
For WBIND instruction, since it's related to memory writeback
^ WBINVD
Quoted 3 lines
and cache flushes, it's mainly used in context of IO devices. Since TDX 1.0 does not support non-virtual I/O devices, skipping it should not cause any fatal issues. But
Do me a favor:
grep -ri wbinvd arch/x86/
How many I/O devices do you see?
Please get your ducks in a row here. Come up with a coherent changelog about why the arch/x86 use of WBINVD doesn't apply to TDX guests. Explain the audit that you did. You *DID* do an audit, right?
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
In reply to Andi KleenQuoted 9 lines · Andi Kleen
Quoted 5 lines
The 'default' case in this 'switch' prints the exit reason and faults, can't that also trigger a backtrace that dumps the exception stack and the faulting instruction? In other words shouldn't this just fail with a common way to provide better debug on any unhandled #VE and not try to continue running past something that "can't" happen?
It will use the #GP common code which will do all the backtracing etc.
We didn't think we would need anything else than what #GP already does.
How do these end up in practice? Do they still say "general protection fault..."?
Isn't that really mean for anyone that goes trying to figure out what caused these? If they see a "general protection fault" from WBINVD and go digging in the SDM for how a #GP can come from WBINVD, won't they be sorely disappointed?
Sean Christopherson
maintainerRe: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
In reply to Dave HansenQuoted 5 lines · Dave Hansen
Quoted 4 lines · Dan Williams
Quoted 4 lines
To prevent TD guest from using MWAIT/MONITOR instructions, support for these instructions are already disabled by TDX module (SEAM). So CPUID flags for these instructions should be in disabled state.
Quoted 7 lines
Quoted 3 lines
Why does this not result in a #UD if the instruction is disabled by SEAM? How is it possible to execute a disabled instruction (one precluded by CPUID) to the point where it triggers #VE instead of #UD?
This is actually a vestige of VMX. It's quite possible toady to have a feature which isn't enumerated in CPUID which still exists and "works" in the silicon.
No, virtualization holes are something else entirely.
MONITOR/MWAIT are a bit weird; they do have an enable bit in IA32_MISC_ENABLE, but most VMMs don't context switch IA32_MISC_ENABLE (load guest value on entry, load host value on exit) because that would add ~250 cycles to every host<->guest transition. And IA32_MISC_ENABLE is shared between SMT siblings, which further complicates loading the guest's value into hardware. In the end, it's easier to leave MONITOR/MWAIT enabled in hardware and instead force a VM-Exit.
As for why TDX injects #VE instead of #UD, I suspect it's for the same reason that KVM emulates MONITOR/MWAIT as nops instead of injecting a #UD. The CPUID bit for MONITOR/MWAIT reflects their enabling in IA32_MISC_ENABLE, not raw support in hardware. That means there's no definitive way to enumerate to BIOS that MONITOR/MWAIT are not supported, e.g. AFAICT, EDKII blindly assumes it can enable MONITOR/MWAIT in IA32_MISC_ENABLE. To justify #UD instead of #VE, TDX would have to inject #GP on WRMSR to set IA32_MISC_ENABLE.ENABLE_MONITOR, and even then there would be weirdness with respect to VMM behavior in response to TDVMCALL(WRMSR) since the VMM could allow the virtual write. In the end, it's again simpler to inject #VE.
Quoted 2 lines
There are all kinds of pitfalls to doing this, but folks evidently do it in public clouds all the time.
Virtualization holes are when instructions/features are enumerated via CPUID, but don't have a control to hide the feature from the guest (or in the case of CET, multiple feature are buried behind a single control). So even if the VMM hides the feature via CPUID, the guest can still _cleanly_ execute the instruction if it's supported by the underlying hardware.
Quoted 6 lines
The CPUID virtualization basically just traps into the hypervisor and lets the hypervisor set whatever register values it wants to appear when CPUID "returns".
But, the controls for what instructions generate #UD are actually quite separate and unrelated to CPUID itself.
Eh, any sane VMM will accurately represent its virtual CPU model via CPUID insofar as possible, there are just too many creaky corners in x86 to make things 100% bombproof.
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
Quoted 7 lines · Sean Christopherson
Quoted 6 lines
Quoted 3 lines
Why does this not result in a #UD if the instruction is disabled by SEAM? How is it possible to execute a disabled instruction (one precluded by CPUID) to the point where it triggers #VE instead of #UD?
This is actually a vestige of VMX. It's quite possible toady to have a feature which isn't enumerated in CPUID which still exists and "works" in the silicon.
No, virtualization holes are something else entirely.
I think the bigger point is that *CPUID* doesn't enable or disable instructions in and of itself.
It can *reflect* enabling (like OSPKE), but nothing is actually enabled or disabled via CPUID.
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
In reply to Dave Hansenneed anything else than what #GP already does.
Quoted 2 lines
How do these end up in practice? Do they still say "general protection fault..."?
Yes, but there's a #VE specific message before it that prints the exit reason.
Quoted 5 lines
Isn't that really mean for anyone that goes trying to figure out what caused these? If they see a "general protection fault" from WBINVD and go digging in the SDM for how a #GP can come from WBINVD, won't they be sorely disappointed?
They'll see both the message and also that it isn't a true #VE in the backtrace.
-Andi
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
Quoted 6 lines · Andi Kleen
Quoted 2 lines
How do these end up in practice? Do they still say "general protection fault..."?
Yes, but there's a #VE specific message before it that prints the exit reason.
Quoted 7 lines
Quoted 4 lines
Isn't that really mean for anyone that goes trying to figure out what caused these? If they see a "general protection fault" from WBINVD and go digging in the SDM for how a #GP can come from WBINVD, won't they be sorely disappointed?
They'll see both the message and also that it isn't a true #VE in the backtrace.
Is there a good reason for the enduring "general protection fault..." message other than an aversion to refactoring the code?
Re: [RFC v2 16/32] x86/tdx: Handle MWAIT, MONITOR and WBINVD
Quoted 3 lines
Is there a good reason for the enduring "general protection fault..." message other than an aversion to refactoring the code?
You're the first ever to think it's a problem.
We're assuming that kernel developers are smart enough to understand this.
Please I implore everyone to move on from this patch. This is my last email on this topic.
-Andi
Re: [RFC v2 14/32] x86/tdx: Handle port I/O
In reply to Dave HansenQuoted 11 lines · Dave Hansen
Quoted 5 lines · Dan Williams
Quoted 2 lines
Decompression code uses port IO for earlyprintk. We must use paravirt calls there too if we want to allow earlyprintk.
What is the tradeoff between teaching the decompression code to handle #VE (the implied assumption) vs teaching it to avoid #VE with direct TDVMCALLs (the chosen direction)?
To me, the tradeoff is not just "teaching" the code to handle a #VE, but ensuring that the entire architecture works.
Quoted 8 lines
Intentionally invoking a #VE is like making a function call that *MIGHT* recurse on itself. Sure, you can try to come up with a story about bounding the recursion. But, I don't see any semblance of that in this series.
Exception-based recursion is really nasty because it's implicit, not explicit. That's why I'm advocating for a design where the kernel never intentionally causes a #VE: it never intentionally recurses without bounds.
So this circles back to the common problem with the mwait/monitor/wbinvd patch and this one. "Can't happen" #VE conditions should be fatal. I.e. have a nice clear message about why the kernel failed and halt. All the uses of these #VE triggering instructions can be eliminated ahead of time with auditing and people that load unaudited out-of-tree modules that trigger #VE get to keep the pieces. Said pieces will be described to them by the #VE triggered fail message. This isn't like split lock disable where the code is difficult to audit.
What am I missing?
Re: [RFC v2 01/32] x86/paravirt: Introduce CONFIG_PARAVIRT_XL
In reply to Juergen GrossQuoted 23 lines · Juergen Gross
Quoted 22 lines · Andi Kleen
\
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
CONFIG_PARAVIRT_XLwill be used by TDX that needs couple of paravirt calls that were hidden underCONFIG_PARAVIRT_XXL, but the rest of the config would be a bloat for TDX.Used how? Why is it bloat for TDX?
Is there any major downside to move the halt related pvops functions from
CONFIG_PARAVIRT_XXLtoCONFIG_PARAVIRT?I think the main motivation is to get rid of all the page table related hooks for modern configurations. These are the bulk of the annotations and cause bloat and worse code. Shadow page tables are really obscure these days and very few people still need them and it's totally reasonable to build even widely used distribution kernels without them. On contrast most of the other hooks are comparatively few and also on comparatively slow paths, so don't really matter too much.
I think it would be ok to have a
CONFIG_PARAVIRTthat does not have page table support, and a separate config option for those (that could be eventually deprecated).
Quoted 7 lines
Quoted 2 lines
But that would break existing .configs for those shadow stack users, that's why I think Kirill did it the other way around.
No. We have
PARAVIRT_XXLfor Xen PV guests, and we have PARAVIRT for other hypervisor's guests, supporting basically the TLB flush operations and time related operations only. Adding the halt related operations to PARAVIRT wouldn't break anything.
Yeah, I think we can do this. It should be fine.
Signature · 1 line
Kirill A. Shutemov
Re: [RFC v2 10/32] x86/tdx: Wire up KVM hypercalls
In reply to Kuppuswamy, SathyanarayananQuoted 10 lines · Sathyanarayanan
Quoted 2 lines · Dave Hansen
I know KVM does weird stuff. But, this is*really* weird. Why are we #including a .c file into another .c file?
I think Kirill implemented it this way to skip
Makefilechanges for it. I don't see any other KVM direct dependencies intdx.c.I will fix it in next version.
This has to be compiled only for TDX+KVM.
Signature · 1 line
Kirill A. Shutemov
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
In reply to Dave HansenQuoted 12 lines · Dave Hansen
Quoted 11 lines · Sathyanarayanan
Quoted 2 lines · Dave Hansen
This doesn't seem much like common code to me. It seems like 100% SEV code. Is this really where we want to move it?
Both SEV and TDX code has requirement to enable
CONFIG_ARCH_HAS_FORCE_DMA_UNENCRYPTEDand defineforce_dma_unencrypted()function.
force_dma_unencrypted()is modified by patch titled "x86/tdx: Make DMA pages shared" to add TDX guest specific support.
Quoted 6 lines
Quoted 1 line
Since both SEV and TDX code uses it, its moved to common file.
That's not an excuse to have a bunch of AMD (or Intel) feature-specific code in a file named "common". I'd make an attempt to keep them separate and then call into the two separate functions *from* the common function.
But why? What good does the additional level of inderection brings?
It's like saying arch/x86/kernel/cpu/common.c shouldn't have anything AMD or Intel specific. If a function can cover both vendors I don't see a point for additinal complexity.
Signature · 1 line
Kirill A. Shutemov
Peter Zijlstra
maintainerRe: [RFC v2 01/32] x86/paravirt: Introduce CONFIG_PARAVIRT_XL
In reply to Juergen GrossQuoted 4 lines · Juergen Gross
No. We have
PARAVIRT_XXLfor Xen PV guests, and we have PARAVIRT for other hypervisor's guests, supporting basically the TLB flush operations and time related operations only. Adding the halt related operations to PARAVIRT wouldn't break anything.
Also, I don't think anything modern should actually ever hit any of the HLT instructions, most everything should end up at an MWAIT.
Still, do we wants to give arch_safe_halt() and halt() the PVOP_ALT_VCALL0() treatment?
Re: [RFC v2 01/32] x86/paravirt: Introduce CONFIG_PARAVIRT_XL
Quoted 11 lines · Peter Zijlstra
Quoted 4 lines · Juergen Gross
No. We have
PARAVIRT_XXLfor Xen PV guests, and we have PARAVIRT for other hypervisor's guests, supporting basically the TLB flush operations and time related operations only. Adding the halt related operations to PARAVIRT wouldn't break anything.Also, I don't think anything modern should actually ever hit any of the HLT instructions, most everything should end up at an MWAIT.
Still, do we wants to give
arch_safe_halt()andhalt()thePVOP_ALT_VCALL0()treatment?
From performance reasons it's pointless to patch. HLT (and MWAIT) are so slow anyways that using patching or an indirect pointer is completely in the noise. So I would use whatever is cleanest in the code.
-Andi
Juergen Gross
maintainerRe: [RFC v2 01/32] x86/paravirt: Introduce CONFIG_PARAVIRT_XL
Quoted 8 lines · Andi Kleen
Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · Juergen Gross
No. We have
PARAVIRT_XXLfor Xen PV guests, and we have PARAVIRT for other hypervisor's guests, supporting basically the TLB flush operations and time related operations only. Adding the halt related operations to PARAVIRT wouldn't break anything.
Quoted 8 lines
Quoted 5 lines
Also, I don't think anything modern should actually ever hit any of the HLT instructions, most everything should end up at an MWAIT.
Still, do we wants to give
arch_safe_halt()andhalt()thePVOP_ALT_VCALL0()treatment?From performance reasons it's pointless to patch. HLT (and MWAIT) are
so slow anyways that using patching or an indirect pointer is completely in the noise. So I would use whatever is cleanest in the code.
This would probably be x86_platform_ops.hyper hooks.
Juergen
Re: [RFC v2 10/32] x86/tdx: Wire up KVM hypercalls
In reply to Kirill A. ShutemovQuoted 1 line · Kirill A. Shutemov
This has to be compiled only for TDX+KVM.
Got it. So if we want to remove the "C" file include, we will have to add #ifdef CONFIG_KVM_GUEST in Makefile.
ifdef CONFIG_KVM_GUEST
obj-$(CONFIG_INTEL_TDX_GUEST) += tdx-kvm.o
#endif
Dave, do you prefer above change over "C" file include?
25 #ifdef CONFIG_KVM_GUEST
26 #include "tdx-kvm.c"
27 #endif
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 10/32] x86/tdx: Wire up KVM hypercalls
Quoted 9 lines · Sathyanarayanan
Quoted 1 line · Kirill A. Shutemov
This has to be compiled only for TDX+KVM.
Got it. So if we want to remove the "C" file include, we will have to add #ifdef
CONFIG_KVM_GUESTinMakefile.ifdef
CONFIG_KVM_GUEST
obj-$(CONFIG_INTEL_TDX_GUEST) += tdx-kvm.o
#endif
Is there truly no dependency between CONFIG_KVM_GUEST and CONFIG_INTEL_TDX_GUEST?
If there isn't, then the way we do it is adding another (invisible) Kconfig variable to express the dependency for tdx-kvm.o:
config INTEL_TDX_GUEST_KVM
bool depends on KVM_GUEST && INTEL_TDX_GUEST
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
In reply to Kirill A. ShutemovQuoted 9 lines · Kirill A. Shutemov
Quoted 4 lines
That's not an excuse to have a bunch of AMD (or Intel) feature-specific code in a file named "common". I'd make an attempt to keep them separate and then call into the two separate functions *from* the common function.
But why? What good does the additional level of inderection brings?
It's like saying
arch/x86/kernel/cpu/common.cshouldn't have anything AMD or Intel specific. If a function can cover both vendors I don't see a point for additinal complexity.
Because the code is already separate. You're actually going to some trouble to move the SEV-specific code and then combine it with the TDX-specific code.
Anyway, please just give it a shot. Should take all of ten minutes. If it doesn't work out in practice, fine. You'll have a good paragraph for the changelog.
Sean Christopherson
maintainerRe: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
Quoted 11 lines · Dave Hansen
Quoted 9 lines · Kirill A. Shutemov
Quoted 4 lines
That's not an excuse to have a bunch of AMD (or Intel) feature-specific code in a file named "common". I'd make an attempt to keep them separate and then call into the two separate functions *from* the common function.
But why? What good does the additional level of inderection brings?
It's like saying
arch/x86/kernel/cpu/common.cshouldn't have anything AMD or Intel specific. If a function can cover both vendors I don't see a point for additinal complexity.
Quoted 7 lines
Because the code is already separate. You're actually going to some trouble to move the SEV-specific code and then combine it with the TDX-specific code.
Anyway, please just give it a shot. Should take all of ten minutes. If it doesn't work out in practice, fine. You'll have a good paragraph for the changelog.
Or maybe wait to see how Boris' propose protected_guest_has() pans out? E.g. if we can do "protected_guest_has(MEMORY_ENCRYPTION)" or whatever, then the truly common bits could be placed into common.c without any vendor-specific logic.
Re: [RFC v2 21/32] x86/boot: Add a trampoline for APs booting in 64-bit mode
In reply to Kuppuswamy SathyanarayananQuoted 6 lines · Kuppuswamy Sathyanarayanan
From: Sean Christopherson <[email protected]>
Add a trampoline for booting APs in 64-bit mode via a software handoff with BIOS, and use the new trampoline for the ACPI MP wake protocol used by TDX.
Lets add a spec reference:
See section "4.1 ACPI-MADT-AP-Wakeup Table" in the Guest-Host Communication Interface specification for TDX.
Although, there is not much "wake protocol" in this patch, this appears to be the end of the process after the CPU has been messaged to start.
Quoted 33 lines
Extend the real mode IDT pointer by four bytes to support LIDT in 64-bit mode. For the GDT pointer, create a new entry as the existing storage for the pointer occupies the zero entry in the GDT itself.
Reported-by Kai Huang <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Reviewed-by Andi Kleen <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>--- arch/x86/include/asm/realmode.h | 1 + arch/x86/kernel/smpboot.c | 5 +++ arch/x86/realmode/rm/header.S | 1 + arch/x86/realmode/rm/trampoline_64.S | 49 +++++++++++++++++++++++- arch/x86/realmode/rm/trampoline_common.S | 5 ++- 5 files changed, 58 insertions(+), 3 deletions(-)arch/x86/include/asm/realmode.h +1
-0@@ -25,6 +25,7 @@ struct real_mode_header {u32 sev_es_trampoline_start;#endif #ifdef CONFIG_X86_64 +u32 trampoline_start64;u32 trampoline_pgd;#endif /* ACPI S3 wakeup */ arch/x86/kernel/smpboot.c +0
-0
arch/x86/kernel/smpboot.c static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle, · line 1042
Quoted 7 lines
arch/x86/kernel/smpboot.c +4
-0@@ -1036,6 +1036,11 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,unsigned long boot_error =0 ;unsigned long timeout; +#ifdef CONFIG_X86_64 +if (is_tdx_guest ()) + start_ip = real_mode_header->trampoline_start64; +#endif
Perhaps wrap this into an inline helper in arch/x86/include/asm/realmode.h so that this routine only does one assignment to @start_ip at function entry?
Quoted 25 lines
arch/x86/kernel/smpboot.c +1
-0+ idle->thread.sp = (unsigned long )task_pt_regs (idle); early_gdt_descr.address = (unsigned long )get_cpu_gdt_rw (cpu); initial_code = (unsigned long )start_secondary;arch/x86/realmode/rm/header.S +1
-0@@ -24,6 +24,7 @@ SYM_DATA_START(real_mode_header) .long pa_sev_es_trampoline_start #endif #ifdef CONFIG_X86_64 + .long pa_trampoline_start64 .long pa_trampoline_pgd; #endif /* ACPI S3 wakeup */arch/x86/realmode/rm/trampoline_64.S +1
-0@@ -143,13 +143,20 @@ SYM_CODE_START(startup_32) movl %eax, %cr3 # Set up EFER + movl $MSR_EFER, %ecx
arch/x86/realmode/rm/trampoline_64.S SYM_CODE_START(startup_32) · line 155
Quoted 10 lines
arch/x86/realmode/rm/trampoline_64.S +6
-1+ rdmsr + cmp pa_tr_efer, %eax + jne .Lwrite_efer + cmp pa_tr_efer + 4, %edx + je .Ldone_efer +.Lwrite_efer: movl pa_tr_efer, %eax movl pa_tr_efer + 4, %edx - movl $MSR_EFER, %ecx wrmsr
Is this hunk just a performance optimization to save an unnecessary wrmsr when it is pre-populated with the right value? Is it required for this patch? If "yes", it was not clear to me from the changelog, if "no" seems like it belongs in a standalone optimization patch.
arch/x86/realmode/rm/trampoline_64.S SYM_CODE_START(startup_32) · line 159
Quoted 5 lines
arch/x86/realmode/rm/trampoline_64.S +2
-1+.Ldone_efer: # Enable paging and in turn activate Long Mode - movl $(X86_CR0_PG | X86_CR0_WP | X86_CR0_PE), %eax + movl $(X86_CR0_PG | X86_CR0_WP | X86_CR0_NE | X86_CR0_PE), %eax
It seems setting X86_CR0_NE is redundant when coming through pa_trampoline_compat, is this a standalone fix to make sure that 'numeric-error' is enabled before startup_64?
Quoted 67 lines
arch/x86/realmode/rm/trampoline_64.S +38
-0movl %eax, %cr0 /*@@ -161,6 +168,19 @@ SYM_CODE_START(startup_32) ljmpl $__KERNEL_CS, $pa_startup_64 SYM_CODE_END(startup_32) +SYM_CODE_START(pa_trampoline_compat) + /* + * In compatibility mode. Prep ESP and DX for startup_32, then disable + * paging and complete the switch to legacy 32-bit mode. + */ + movl $rm_stack_end, %esp + movw $__KERNEL_DS, %dx + + movl $(X86_CR0_NE | X86_CR0_PE), %eax + movl %eax, %cr0 + ljmpl $__KERNEL32_CS, $pa_startup_32 +SYM_CODE_END(pa_trampoline_compat) + .section ".text64","ax" .code64 .balign 4@@ -169,6 +189,20 @@ SYM_CODE_START(startup_64) jmpq *tr_start(%rip) SYM_CODE_END(startup_64) +SYM_CODE_START(trampoline_start64) + /* + * APs start here on a direct transfer from 64-bit BIOS with identity + * mapped page tables. Load the kernel's GDT in order to gear down to + * 32-bit mode (to handle 4-level vs. 5-level paging), and to (re)load + * segment registers. Load the zero IDT so any fault triggers a + * shutdown instead of jumping back into BIOS. + */ + lidt tr_idt(%rip) + lgdt tr_gdt64(%rip) + + ljmpl *tr_compat(%rip) +SYM_CODE_END(trampoline_start64) + .section ".rodata","a" # Duplicate the global descriptor table # so the kernel can live anywhere@@ -182,6 +216,17 @@ SYM_DATA_START(tr_gdt) .quad 0x00cf93000000ffff # __KERNEL_DS SYM_DATA_END_LABEL(tr_gdt, SYM_L_LOCAL, tr_gdt_end) +SYM_DATA_START(tr_gdt64) + .short tr_gdt_end - tr_gdt - 1 # gdt limit + .long pa_tr_gdt + .long 0 +SYM_DATA_END(tr_gdt64) + +SYM_DATA_START(tr_compat) + .long pa_trampoline_compat + .short __KERNEL32_CS +SYM_DATA_END(tr_compat) + .bss .balign PAGE_SIZE SYM_DATA(trampoline_pgd, .space PAGE_SIZE)arch/x86/realmode/rm/trampoline_common.S +0
-0
arch/x86/realmode/rm/trampoline_common.S line 7
Quoted 8 lines
arch/x86/realmode/rm/trampoline_common.S +4
-1@@ -1,4 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ .section ".rodata","a" .balign 16 -SYM_DATA_LOCAL(tr_idt, .fill 1, 6, 0) +SYM_DATA_START_LOCAL(tr_idt) + .short 0 + .quad 0 +SYM_DATA_END(tr_idt)
Curious, is the following not equivalent?
-SYM_DATA_LOCAL(tr_idt, .fill 1, 6, 0)
+SYM_DATA_LOCAL(tr_idt, .fill 1, 10, 0)
Re: [RFC v2 22/32] x86/boot: Avoid #VE during compressed boot for TDX platforms
In reply to Kuppuswamy SathyanarayananQuoted 6 lines · Kuppuswamy Sathyanarayanan
From: Sean Christopherson <[email protected]>
Avoid operations which will inject #VE during compressed boot, which is obviously fatal for TDX platforms.
Quoted 6 lines
Details are,
1. TDX module injects #VE if a TDX guest attempts to write EFER. So skip the WRMSR to set EFER.LME=1 if it's already set. TDX also forces EFER.LME=1, i.e. the branch will always be taken and thus the #VE avoided.
Ah here's the justification for that hunk in the previous patch, are you sure that hunk belongs in the trampoline patch?
Quoted 5 lines
2. TDX module also injects a #VE if the guest attempts to clear CR0.NE. Ensure CR0.NE is set when loading CR0 during compressed boot. The Setting CR0.NE should be a nop on all CPUs that support 64-bit mode.
Ah, here's the justification for CR0.NE in the previous patch. Did something go wrong in the patch splitting?
Quoted 49 lines
Signed-off-by Sean Christopherson <[email protected]>Reviewed-by Andi Kleen <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>--- arch/x86/boot/compressed/head_64.S | 5 +++-- arch/x86/boot/compressed/pgtable.h | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-)arch/x86/boot/compressed/head_64.S +3
-2@@ -616,8 +616,9 @@ SYM_CODE_START(trampoline_32bit_src) movl $MSR_EFER, %ecx rdmsr btsl $_EFER_LME, %eax + jc 1f wrmsr - popl %edx +1: popl %edx popl %ecx /* Enable PAE and LA57 (if required) paging modes */@@ -636,7 +637,7 @@ SYM_CODE_START(trampoline_32bit_src) pushl %eax /* Enable paging again */ - movl $(X86_CR0_PG | X86_CR0_PE), %eax + movl $(X86_CR0_PG | X86_CR0_NE | X86_CR0_PE), %eax movl %eax, %cr0 lretarch/x86/boot/compressed/pgtable.h +1
-2@@ -6,7 +6,7 @@#define TRAMPOLINE_32BIT_PGTABLE_OFFSET 0#define TRAMPOLINE_32BIT_CODE_OFFSET PAGE_SIZE -#define TRAMPOLINE_32BIT_CODE_SIZE 0x70 +#define TRAMPOLINE_32BIT_CODE_SIZE 0x80#define TRAMPOLINE_32BIT_STACK_END TRAMPOLINE_32BIT_SIZE --2.25.1
Re: [RFC v2 23/32] x86/boot: Avoid unnecessary #VE during boot process
In reply to Kuppuswamy SathyanarayananQuoted 54 lines · Kuppuswamy Sathyanarayanan
From: Sean Christopherson <[email protected]>
Skip writing EFER during
secondary_startup_64()if the current value is also the desired value. This avoids a #VE when running as a TDX guest, as the TDX-Module does not allow writes to EFER (even when writing the current, fixed value).Also, preserve CR4.MCE instead of clearing it during boot to avoid a #VE when running as a TDX guest. The TDX-Module (effectively part of the hypervisor) requires CR4.MCE to be set at all times and injects a #VE if the guest attempts to clear CR4.MCE.
Signed-off-by Sean Christopherson <[email protected]>Reviewed-by Andi Kleen <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>--- arch/x86/boot/compressed/head_64.S | 5 ++++- arch/x86/kernel/head_64.S | 13 +++++++++++-- 2 files changed, 15 insertions(+), 3 deletions(-)arch/x86/boot/compressed/head_64.S +4
-1@@ -622,7 +622,10 @@ SYM_CODE_START(trampoline_32bit_src) popl %ecx /* Enable PAE and LA57 (if required) paging modes */ - movl $X86_CR4_PAE, %eax + movl %cr4, %eax + /* Clearing CR4.MCE will #VE on TDX guests. Leave it alone. */ + andl $X86_CR4_MCE, %eax + orl $X86_CR4_PAE, %eax testl %edx, %edx jz 1f orl $X86_CR4_LA57, %eaxarch/x86/kernel/head_64.S +4
-1@@ -141,7 +141,10 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) 1: /* Enable PAE mode, PGE and LA57 */ - movl $(X86_CR4_PAE | X86_CR4_PGE), %ecx + movq %cr4, %rcx + /* Clearing CR4.MCE will #VE on TDX guests. Leave it alone. */ + andl $X86_CR4_MCE, %ecx + orl $(X86_CR4_PAE | X86_CR4_PGE), %ecx #ifdef CONFIG_X86_5LEVEL testl $1, __pgtable_l5_enabled(%rip) jz 1f
arch/x86/kernel/head_64.S SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) · line 235
Quoted 4 lines
arch/x86/kernel/head_64.S +1
-0@@ -229,13 +232,19 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) /* Setup EFER (Extended Feature Enable Register) */ movl $MSR_EFER, %ecx rdmsr + movl %eax, %edx
Maybe comment that EFER is being saved here to check if the following enables are nops, but not a big deal.
...modulo whether the EFER wrmsr avoidance in PATCH 21 should move here.
Quoted 19 lines
arch/x86/kernel/head_64.S +6
-2btsl $_EFER_SCE, %eax /* Enable System Call */ btl $20,%edi /* No Execute supported? */ jnc 1f btsl $_EFER_NX, %eax btsq $_PAGE_BIT_NX,early_pmd_flags(%rip) -1: wrmsr /* Make changes effective */ + /* Skip the WRMSR if the current value matches the desired value. */ +1: cmpl %edx, %eax + je 1f + xor %edx, %edx + wrmsr /* Make changes effective */ +1: /* Setup cr0 */ movl $CR0_STATE, %eax /* Make changes effective */ --2.25.1
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
In reply to Sean ChristophersonQuoted 16 lines · Sean Christopherson
Quoted 15 lines · Dave Hansen
Quoted 9 lines · Kirill A. Shutemov
Quoted 4 lines
That's not an excuse to have a bunch of AMD (or Intel) feature-specific code in a file named "common". I'd make an attempt to keep them separate and then call into the two separate functions *from* the common function.
But why? What good does the additional level of inderection brings?
It's like saying
arch/x86/kernel/cpu/common.cshouldn't have anything AMD or Intel specific. If a function can cover both vendors I don't see a point for additinal complexity.Because the code is already separate. You're actually going to some trouble to move the SEV-specific code and then combine it with the TDX-specific code.
Quoted 7 lines
Quoted 3 lines
Anyway, please just give it a shot. Should take all of ten minutes. If it doesn't work out in practice, fine. You'll have a good paragraph for the changelog.
Or maybe wait to see how Boris' propose
protected_guest_has()pans out? E.g. if we can do "protected_guest_has(MEMORY_ENCRYPTION)" or whatever, then the truly common bits could be placed intocommon.cwithout any vendor-specific logic.
How about following abstraction? This patch was initially created to enable us use is_tdx_guest() outside of arch/x86 code. But extended it to support bitmap flags.
commit 188bdd3c97e49020b2bda9efd992a22091423b85
Author: Kuppuswamy Sathyanarayanan <[email protected]> Date: Wed May 12 11:35:13 2021 -0700
tdx: Introduce generic protected_guest abstraction
Add a generic way to check if we run with an encrypted guest, without requiring x86 specific ifdefs. This can then be used in non architecture specific code. Enablethis when running under TDX/SEV.
Also add helper functions to set/test encrypted guest feature flags.
arch/Kconfig +3 -0
@@ -956,6 +956,9 @@ config HAVE_ARCH_NVRAM_OPS config ISA_BUS_API def_bool ISA +config ARCH_HAS_PROTECTED_GUEST + bool + # # ABI hall of shame #
arch/x86/Kconfig +2 -0
@@ -882,6 +882,7 @@ config INTEL_TDX_GUEST select PARAVIRT_XL select X86_X2APIC select SECURITY_LOCKDOWN_LSM + select ARCH_HAS_PROTECTED_GUEST help Provide support for running in a trusted domain on Intel processors equipped with Trusted Domain eXtenstions. TDX is a new Intel
@@ -1537,6 +1538,7 @@ config AMD_MEM_ENCRYPT select ARCH_USE_MEMREMAP_PROT select ARCH_HAS_FORCE_DMA_UNENCRYPTED select INSTRUCTION_DECODER + select ARCH_HAS_PROTECTED_GUEST help Say yes to enable support for the encryption of system memory. This requires an AMD processor that supports Secure Memory
arch/x86/kernel/setup.c +5 -0
@@ -21,6 +21,7 @@#include <linux/usb/xhci-dbgp.h> #include <linux/static_call.h> #include <linux/swiotlb.h> +#include <linux/protected_guest.h> #include <uapi/linux/mount.h>
@@ -107,6 +108,10 @@ static struct resource bss_resource = { .flags = IORESOURCE_BUSY | IORESOURCE_SYSTEM_RAM }; +#ifdef CONFIG_ARCH_HAS_PROTECTED_GUEST +DECLARE_BITMAP (protected_guest_flags,PROTECTED_GUEST_BITMAP_LEN ); +EXPORT_SYMBOL (protected_guest_flags); +#endif #ifdef CONFIG_X86_32 /* CPU data as detected by the assembly code in head_32.S */
arch/x86/kernel/sev-es.c +4 -0
@@ -19,6 +19,7 @@#include <linux/memblock.h> #include <linux/kernel.h> #include <linux/mm.h> +#include <linux/protected_guest.h> #include <asm/cpu_entry_area.h> #include <asm/stacktrace.h>
@@ -680,6 +681,9 @@ static void __init init_ghcb(int cpu) data->ghcb_active = false; data->backup_ghcb_active = false; + +set_protected_guest_flag (GUEST_TYPE_SEV ); +set_protected_guest_flag (MEMORY_ENCRYPTION ); }void __init sev_es_init_vc_handling (void )
arch/x86/kernel/tdx.c +4 -0
@@ -7,6 +7,7 @@#include <asm/vmx.h> #include <linux/cpu.h> +#include <linux/protected_guest.h> static struct { unsigned int gpa_width;
@@ -92,6 +93,9 @@ void __init tdx_early_init(void)setup_force_cpu_cap (X86_FEATURE_TDX_GUEST ); +set_protected_guest_flag (GUEST_TYPE_TDX ); +set_protected_guest_flag (MEMORY_ENCRYPTION ); +tdg_get_info ();pr_info ("TDX guest is initialized\n" );
include/linux/protected_guest.h new file +37 -0
@@ -0,0 +1,37 @@ +#ifndef _LINUX_PROTECTED_GUEST_H +#define _LINUX_PROTECTED_GUEST_H 1 + +#define PROTECTED_GUEST_BITMAP_LEN 128 + +/* Protected Guest vendor types */ +#define GUEST_TYPE_TDX (1) +#define GUEST_TYPE_SEV (2) + +/* Protected Guest features */ +#define MEMORY_ENCRYPTION (20) + +#ifdef CONFIG_ARCH_HAS_PROTECTED_GUEST +extern DECLARE_BITMAP(protected_guest_flags, PROTECTED_GUEST_BITMAP_LEN); + +static bool protected_guest_has (unsigned long flag) +{ +return test_bit (flag, protected_guest_flags); +} + +static inline void set_protected_guest_flag (unsigned long flag) +{ +__set_bit (flag, protected_guest_flags); +} + +static inline bool is_protected_guest (void ) +{ +return (protected_guest_has (GUEST_TYPE_TDX ) | +protected_guest_has (GUEST_TYPE_SEV ) ); +} +#else +static inline bool protected_guest_has (unsigned long flag) {return false; } +static inline void set_protected_guest_flag (unsigned long flag) { } +static inline bool is_protected_guest (void ) {return false; } +#endif + +#endif
Quoted 1 line
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
include/linux/protected_guest.h line 11
Quoted 9 lines · Sathyanarayanan
include/linux/protected_guest.h new file +8
-0+#define PROTECTED_GUEST_BITMAP_LEN 128 + +/* Protected Guest vendor types */ +#define GUEST_TYPE_TDX (1) +#define GUEST_TYPE_SEV (2) + +/* Protected Guest features */ +#define MEMORY_ENCRYPTION (20)
I was assuming we'd reuse the X86_FEATURE infrastructure somehow. Is there a good reason not to?
That gives us all the compile-time optimization (via en/disabled-features.h) and static branches for "free".
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
Quoted 13 lines · Dave Hansen
include/linux/protected_guest.h line 11
Quoted 9 lines · Sathyanarayanan
include/linux/protected_guest.h new file +8
-0+#define PROTECTED_GUEST_BITMAP_LEN 128 + +/* Protected Guest vendor types */ +#define GUEST_TYPE_TDX (1) +#define GUEST_TYPE_SEV (2) + +/* Protected Guest features */ +#define MEMORY_ENCRYPTION (20)I was assuming we'd reuse the
X86_FEATUREinfrastructure somehow. Is there a good reason not to?
My assumption is, protected guest abstraction can be also used by non-x86 arch's in future. So I have tried to keep these definitions in common code.
Quoted 4 lines
That gives us all the compile-time optimization (via
en/disabled-features.h) and static branches for "free".
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 10/32] x86/tdx: Wire up KVM hypercalls
In reply to Dave HansenQuoted 7 lines · Dave Hansen
Quoted 6 lines · Sathyanarayanan
Quoted 1 line · Kirill A. Shutemov
This has to be compiled only for TDX+KVM.
Got it. So if we want to remove the "C" file include, we will have to add #ifdef
CONFIG_KVM_GUESTinMakefile.
Quoted 6 lines
Quoted 3 lines
ifdef
CONFIG_KVM_GUEST
obj-$(CONFIG_INTEL_TDX_GUEST) += tdx-kvm.o
#endifIs there truly no dependency between
CONFIG_KVM_GUESTandCONFIG_INTEL_TDX_GUEST?
We want to re-use TDX code with other hypervisors/guests as well. So we can't create direct dependency with CONFIG_KVM_GUEST in Kconfig.
Quoted 7 lines
If there isn't, then the way we do it is adding another (invisible)
Kconfigvariable to express the dependency for tdx-kvm.o:config
INTEL_TDX_GUEST_KVMbool depends on KVM_GUEST && INTEL_TDX_GUEST
Currently it will only be used for KVM hypercall code. Will it to be overkill to create a new config over #ifdefs for this use case ? But, if this is the preferred approach, I will go with this suggestion.
Quoted 1 line
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 10/32] x86/tdx: Wire up KVM hypercalls
Quoted 10 lines · Sathyanarayanan
Quoted 6 lines
If there isn't, then the way we do it is adding another (invisible)
Kconfigvariable to express the dependency for tdx-kvm.o:config
INTEL_TDX_GUEST_KVM
bool
depends onKVM_GUEST&&INTEL_TDX_GUESTCurrently it will only be used for KVM hypercall code. Will it to be overkill to create a new config over #ifdefs for this use case ? But, if this is the preferred approach, I will go with this suggestion.
You'll see this done lots of different (valid) ways over the kernel. (#ifdef'd #including C files is not one of them.)
*My* preference is to use Kconfig in the way I described. It keeps makefiles and #ifdef's clean and obvious, relegating the logic to Kconfig.
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
In reply to Dave HansenQuoted 11 lines · Dave Hansen
include/linux/protected_guest.h line 11
Quoted 8 lines · Sathyanarayanan
include/linux/protected_guest.h new file +8
-0+#define PROTECTED_GUEST_BITMAP_LEN 128 + +/* Protected Guest vendor types */ +#define GUEST_TYPE_TDX (1) +#define GUEST_TYPE_SEV (2) + +/* Protected Guest features */ +#define MEMORY_ENCRYPTION (20)I was assuming we'd reuse the
X86_FEATUREinfrastructure somehow. Is there a good reason not to?
This for generic code. Would be a gigantic lift and lots of refactoring to move that out.
Quoted 3 lines
That gives us all the compile-time optimization (via
en/disabled-features.h) and static branches for "free".
There's no user so far which is anywhere near performance critical, so that would be total overkil
BTW right now I'm not even sure we need the bitmap for anything, but I guess it doesn't hurt.
-Andi
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
Quoted 14 lines · Andi Kleen
Quoted 11 lines · Dave Hansen
include/linux/protected_guest.h line 11
Quoted 8 lines · Sathyanarayanan
include/linux/protected_guest.h new file +8
-0+#define PROTECTED_GUEST_BITMAP_LEN 128 + +/* Protected Guest vendor types */ +#define GUEST_TYPE_TDX (1) +#define GUEST_TYPE_SEV (2) + +/* Protected Guest features */ +#define MEMORY_ENCRYPTION (20)I was assuming we'd reuse the
X86_FEATUREinfrastructure somehow. Is there a good reason not to?
Quoted 2 lines
This for generic code. Would be a gigantic lift and lots of refactoring to move that out.
Ahh, forgot about that. The whole "x86/mm" subject threw me off.
Quoted 5 lines
Quoted 2 lines
That gives us all the compile-time optimization (via
en/disabled-features.h) and static branches for "free".There's no user so far which is anywhere near performance critical, so that would be total overkil
The *REALLY* nice thing is that it keeps you from having to create stub functions or #ifdefs and yet the compiler can still optimize the code to nothing.
Anyway, thanks for the clarification about it being in non-arch code.
Re: [RFC v2 08/32] x86/traps: Add #VE support for TDX guest
In reply to Dave HansenQuoted 10 lines · Dave Hansen
On 4/26/21 11:01 AM, Kuppuswamy Sathyanarayanan wrote:
...Quoted 5 lines
The #VE cannot be nested before TDGETVEINFO is called, if there is any reason for it to nest the TD would shut down. The TDX module guarantees that no NMIs (or #MC or similar) can happen in this window. After TDGETVEINFO the #VE handler can nest if needed, although we don’t expect it to happen normally.
I think this description really needs some work. Does "The #VE cannot be nested" mean that "hardware guarantees that #VE will not be generated", or "the #VE must not be nested"?
The next half sentence answers this question..
"if there is any reason for it to nest the TD would shut down."
So it cannot nest.
Quoted 4 lines
What does "the TD would shut down" mean? I think you mean that instead of delivering a nested #VE the hardware would actually exit to the host and TDX would prevent the guest from being reentered. Right?
Yes that's a shutdown. I Suppose we could add your sentence.
Quoted 2 lines
I find that description a bit unsatisfying. Could we make this a bit more concrete?
I don't see what could be added. If you have concrete suggestions please just propose something.
Quoted 1 line
By the way, what about *normal* interrupts?
Normal interrupts are blocked of course like in every other exception or interrupt entry.
Quoted 7 lines
Maybe we should talk about this in terms of *rules* that folks need to follow. Maybe:
NMIs and machine checks are suppressed. Before this point any #VE is fatal. After this point, NMIs and additional #VEs are permitted.
Okay that's fine for me.
-Andi
Re: [RFC v2 08/32] x86/traps: Add #VE support for TDX guest
Quoted 1 line · Andi Kleen
"if there is any reason for it to nest the TD would shut down."
The TDX EAS says:
Quoted 4 lines
If, when attempting to inject a #VE, the Intel TDX module discovers that the guest TD has not yet retrieved the information for a previous #VE (i.e., VE_INFO.VALID is not 0), the TDX module injects a #DF into the guest TD to indicate a #VE overrun.
How does that result in a shut down?
Re: [RFC v2 08/32] x86/traps: Add #VE support for TDX guest
In reply to Andi KleenQuoted 2 lines · Andi Kleen
I don't see what could be added. If you have concrete suggestions please just propose something.
Oh, boy, I love writing changelogs! I was hoping that the TDX folks would chip in to write their own changelogs, but oh well. You made my day!
--
Virtualization Exceptions (#VE) are delivered to TDX guests due to specific guest actions which may happen in either userspace or the kernel:
* Specific instructions (WBINVD, for example)
* Specific MSR accesses
* Specific CPUID leaf accesses
* Access to TD-shared memory, which includes MMIO
#VE exceptions are never generated on accesses to normal, TD-private memory.
The entry paths do not access TD-shared memory or use those specific MSRs, instructions, CPUID leaves. In addition, all interrupts including NMIs are blocked by the hardware starting with #VE delivery until TDGETVEINFO is called. This eliminates the chance of a #VE during the syscall gap or paranoid entry paths and simplifies #VE handling.
If a guest kernel action which would normally cause a #VE occurs in the interrupt-disabled region before TDGETVEINFO, a #DF is delivered to the guest.
Add basic infrastructure to handle any #VE which occurs in the kernel or userspace. Later patches will add handling for specific #VE scenarios.
Convert unhandled #VE's (everything, until later in this series) so that they appear just like a #GP by calling do_general_protection() directly.
--
Did I miss anything?
Re: [RFC v2 08/32] x86/traps: Add #VE support for TDX guest
In reply to Dave HansenQuoted 9 lines · Dave Hansen
Quoted 1 line · Andi Kleen
"if there is any reason for it to nest the TD would shut down."
The TDX EAS says:
Quoted 4 lines
If, when attempting to inject a #VE, the Intel TDX module discovers that the guest TD has not yet retrieved the information for a previous #VE (i.e., VE_INFO.VALID is not 0), the TDX module injects a #DF into the guest TD to indicate a #VE overrun.
How does that result in a shut down?
You're right. It's not a shutdown, but a panic. We'll need to fix the comment and replace 'shutdown' with 'panic'
-And
Sean Christopherson
maintainerRe: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
In reply to Andi KleenQuoted 14 lines · Andi Kleen
Quoted 11 lines · Dave Hansen
include/linux/protected_guest.h line 11
Quoted 8 lines · Sathyanarayanan
include/linux/protected_guest.h new file +8
-0+#define PROTECTED_GUEST_BITMAP_LEN 128 + +/* Protected Guest vendor types */ +#define GUEST_TYPE_TDX (1) +#define GUEST_TYPE_SEV (2) + +/* Protected Guest features */ +#define MEMORY_ENCRYPTION (20)I was assuming we'd reuse the
X86_FEATUREinfrastructure somehow. Is there a good reason not to?
Quoted 2 lines
This for generic code. Would be a gigantic lift and lots of refactoring to move that out.
What generic code needs access to SEV vs. TDX? force_dma_unencrypted() is called from generic code, but its implementation is x86 specific.
Quoted 5 lines
Quoted 2 lines
That gives us all the compile-time optimization (via
en/disabled-features.h) and static branches for "free".There's no user so far which is anywhere near performance critical, so that would be total overkil
SEV already has the sev_enable_key static key that it uses for unrolling string I/O, so there's at least one (debatable) case that wants to use static branches.
For SEV-ES and TDX, there's a better argument as using X86_FEATURE_* would unlock alternatives.
Quoted 6 lines
BTW right now I'm not even sure we need the bitmap for anything, but I guess it doesn't hurt.
-Andi
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
Quoted 2 lines · Sean Christopherson
What generic code needs access to SEV vs. TDX?
force_dma_unencrypted()is called from generic code, but its implementation is x86 specific.
When the hardening the drivers for TDX usage, we will have requirement to check for is_protected_guest() to add code specific to protected guests. Since this will be outside arch/x86, we need common framework for it.
Few examples are,
* ACPI sleep driver uses WBINVD (when doing cache flushes). We want to skip it for TDX.
* Forcing virtio to use dma API when running with untrusted host.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
Quoted 9 lines · Sathyanarayanan
Quoted 3 lines · Sean Christopherson
What generic code needs access to SEV vs. TDX?
force_dma_unencrypted()is called from generic code, but its implementation is x86 specific.When the hardening the drivers for TDX usage, we will have requirement to check for
is_protected_guest()to add code specific to protected guests. Since this will be outsidearch/x86, we need common framework for it.
Just remember, a "common framework" doesn't mean that it can't be backed by extremely arch-specific mechanisms.
For instance, there's a lot of pkey-specific code in mm/mprotect.c. It still gets optimized away on x86 with all the goodness of X86_FEATUREs.
Sean Christopherson
maintainerRe: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
Quoted 11 lines · Dave Hansen
Quoted 9 lines · Sathyanarayanan
Quoted 3 lines · Sean Christopherson
What generic code needs access to SEV vs. TDX?
force_dma_unencrypted()is called from generic code, but its implementation is x86 specific.When the hardening the drivers for TDX usage, we will have requirement to check for
is_protected_guest()to add code specific to protected guests. Since this will be outsidearch/x86, we need common framework for it.
Quoted 5 lines
Just remember, a "common framework" doesn't mean that it can't be backed by extremely arch-specific mechanisms.
For instance, there's a lot of pkey-specific code in
mm/mprotect.c. It still gets optimized away on x86 with all the goodness ofX86_FEATUREs.
Ya, exactly. Ideally, generic code shouldn't have to differentiate between SEV, SEV-ES, SEV-SNP, TDX, etc..., a vanilla "bool is_protected_guest(void)" should suffice. Under the hood, x86's implementation for is_protected_guest() can be boot_cpu_has() checks (if we want).
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
Quoted 9 lines · Sean Christopherson
Quoted 5 lines
Just remember, a "common framework" doesn't mean that it can't be backed by extremely arch-specific mechanisms.
For instance, there's a lot of pkey-specific code in
mm/mprotect.c. It still gets optimized away on x86 with all the goodness ofX86_FEATUREs.Ya, exactly. Ideally, generic code shouldn't have to differentiate between SEV, SEV-ES, SEV-SNP, TDX, etc..., a vanilla "bool
is_protected_guest(void)" should suffice. Under the hood, x86's implementation foris_protected_guest()can beboot_cpu_has()checks (if we want).
What about the use case of protected_guest_has(flag)? Do you want to call it with with X86_FEATURE_* flags outside arch/x86 code ?
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
Quoted 8 lines · Sathyanarayanan
Quoted 5 lines · Sean Christopherson
Quoted 5 lines
Just remember, a "common framework" doesn't mean that it can't be backed by extremely arch-specific mechanisms.
For instance, there's a lot of pkey-specific code in mm/mprotect.c. It still gets optimized away on x86 with all the goodness of
X86_FEATUREs.
Quoted 7 lines
Quoted 4 lines
Ya, exactly. Ideally, generic code shouldn't have to differentiate between SEV, SEV-ES, SEV-SNP, TDX, etc..., a vanilla "bool
is_protected_guest(void)" should suffice. Under the hood, x86's implementation foris_protected_guest()can beboot_cpu_has()checks (if we want).What about the use case of
protected_guest_has(flag)? Do you want to call it with with X86_FEATURE_* flags outsidearch/x86code ?
I don't think we need any flags in the generic code. Just a simple bool is enough.
-Andi
RFC 1/1 x86/paravirt: Move halt paravirt calls under CONFIG_PARAVIRT
In reply to Juergen GrossFrom: "Kirill A. Shutemov" <[email protected]>
CONFIG_PARAVIRT_XXL is mainly defined/used by XEN PV guests. For other VM guest types, features supported under CONFIG_PARAVIRT are self sufficient. CONFIG_PARAVIRT mainly provides support for TLB flush operations and time related operations.
For TDX guest as well, paravirt calls under CONFIG_PARVIRT meets most of its requirement except the need of HLT and SAFE_HLT paravirt calls, which is currently defined under COFNIG_PARAVIRT_XXL.
Since enabling CONFIG_PARAVIRT_XXL is too bloated for TDX guest like platforms, move HLT and SAFE_HLT paravirt calls under CONFIG_PARAVIRT.
Moving HLT and SAFE_HLT paravirt calls are not fatal and should not break any functionality for current users of CONFIG_PARAVIRT.
---
Changes since v1:
* Removed CONFIG_PARAVIRT_XL
* Moved HLT and SAFE_HLT under CONFIG_PARAVIRT
arch/x86/include/asm/irqflags.h | 40 +++++++++++++++------------ arch/x86/include/asm/paravirt.h | 20 +++++++------- arch/x86/include/asm/paravirt_types.h | 3 +- arch/x86/kernel/paravirt.c | 4 ++- 4 files changed, 36 insertions(+), 31 deletions(-)
arch/x86/include/asm/irqflags.h +22 -18
@@ -59,6 +59,28 @@ static inline __cpuidle void native_halt(void)#endif +#ifndef CONFIG_PARAVIRT +#ifndef __ASSEMBLY__ +/* +* Used in the idle loop; sti takes one instruction cycle +* to complete: +*/ +static inline __cpuidle void arch_safe_halt(void) +{ +native_safe_halt (); +} + +/* +* Used when interrupts are already enabled or to +* shutdown the processor: +*/ +static inline __cpuidle void halt(void) +{ +native_halt (); +} +#endif /* __ASSEMBLY__ */ +#endif /* CONFIG_PARAVIRT */ +#ifdef CONFIG_PARAVIRT_XXL #include <asm/paravirt.h> #else
@@ -80,24 +102,6 @@ static __always_inline void arch_local_irq_enable(void)native_irq_enable (); } -/* -* Used in the idle loop; sti takes one instruction cycle -* to complete: -*/ -static inline __cpuidle void arch_safe_halt(void) -{ -native_safe_halt (); -} - -/* -* Used when interrupts are already enabled or to -* shutdown the processor: -*/ -static inline __cpuidle void halt(void) -{ -native_halt (); -} -/* * For spinlocks, etc: */
arch/x86/include/asm/paravirt.h +10 -10
@@ -84,6 +84,16 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct *mm)PVOP_VCALL1 (mmu.exit_mmap, mm); } +static inline void arch_safe_halt (void ) +{ +PVOP_VCALL0 (irq.safe_halt); +} + +static inline void halt (void ) +{ +PVOP_VCALL0 (irq.halt); +} +#ifdef CONFIG_PARAVIRT_XXL static inline void load_sp0 (unsigned long sp0) {
@@ -145,16 +155,6 @@ static inline void __write_cr4(unsigned long x)PVOP_VCALL1 (cpu.write_cr4, x); } -static inline void arch_safe_halt (void ) -{ -PVOP_VCALL0 (irq.safe_halt); -} - -static inline void halt (void ) -{ -PVOP_VCALL0 (irq.halt); -} -static inline void wbinvd (void ) {PVOP_VCALL0 (cpu.wbinvd);
arch/x86/include/asm/paravirt_types.h +1 -2
@@ -177,10 +177,9 @@ struct pv_irq_ops {struct paravirt_callee_save save_fl;struct paravirt_callee_save irq_disable;struct paravirt_callee_save irq_enable; - +#endif void (*safe_halt)(void );void (*halt)(void ); -#endif } __no_randomize_layout;struct pv_mmu_ops {
arch/x86/kernel/paravirt.c +3 -1
@@ -322,9 +322,11 @@ struct paravirt_patch_template pv_ops = { .irq.save_fl = __PV_IS_CALLEE_SAVE(native_save_fl), .irq.irq_disable = __PV_IS_CALLEE_SAVE(native_irq_disable), .irq.irq_enable = __PV_IS_CALLEE_SAVE(native_irq_enable), +#endif /* CONFIG_PARAVIRT_XXL */ + +/* Irq HLT ops. */ .irq.safe_halt = native_safe_halt, .irq.halt = native_halt, -#endif /* CONFIG_PARAVIRT_XXL */ /* Mmu ops. */ .mmu.flush_tlb_user = native_flush_tlb_local,
Signature · 1 line
2.25.1
RFC 1/1 x86/traps: Add #VE support for TDX guest
In reply to Dave HansenFrom: "Kirill A. Shutemov" <[email protected]>
Virtualization Exceptions (#VE) are delivered to TDX guests due to specific guest actions which may happen in either user space or the kernel:
* Specific instructions (WBINVD, for example)
* Specific MSR accesses
* Specific CPUID leaf accesses
* Access to TD-shared memory, which includes MMIO
In the settings that Linux will run in, virtual exceptions are never generated on accesses to normal, TD-private memory that has been accepted.
The entry paths do not access TD-shared memory, MMIO regions or use those specific MSRs, instructions, CPUID leaves that might generate #VE. In addition, all interrupts including NMIs are blocked by the hardware starting with #VE delivery until TDGETVEINFO is called. This eliminates the chance of a #VE during the syscall gap or paranoid entry paths and simplifies #VE handling.
After TDGETVEINFO #VE could happen in theory (e.g. through an NMI), although we don't expect it to happen because we don't expect NMIs to trigger #VEs. Another case where they could happen is if the #VE exception panics, but in this case there are no guarantees on anything anyways.
If a guest kernel action which would normally cause a #VE occurs in the interrupt-disabled region before TDGETVEINFO, a #DF is delivered to the guest which will result in an oops (and should eventually be a panic, as we would like to set panic_on_oops to 1 for TDX guests).
Add basic infrastructure to handle any #VE which occurs in the kernel or userspace. Later patches will add handling for specific #VE scenarios.
Convert unhandled #VE's (everything, until later in this series) so that they appear just like a #GP by calling ve_raise_fault() directly. ve_raise_fault() is similar to #GP handler and is responsible for sending SIGSEGV to userspace and cpu die and notifying debuggers and other die chain users.
---
Changes since v1:
* Removed [RFC v2 07/32] x86/traps: Add do_general_protection() helper function.
* Instead of resuing #GP handler, defined a custom handler.
* Fixed commit log as per review comments.
arch/x86/include/asm/idtentry.h | 4 ++ arch/x86/include/asm/tdx.h | 20 ++++++++++ arch/x86/kernel/idt.c | 6 +++ arch/x86/kernel/tdx.c | 35 +++++++++++++++++ arch/x86/kernel/traps.c | 70 +++++++++++++++++++++++++++++++++ 5 files changed, 135 insertions(+)
arch/x86/include/asm/idtentry.h +4 -0
@@ -619,6 +619,10 @@ DECLARE_IDTENTRY_XENCB(X86_TRAP_OTHER, exc_xen_hypervisor_callback);DECLARE_IDTENTRY_RAW (X86_TRAP_OTHER , exc_xen_unknown_trap);#endif +#ifdef CONFIG_INTEL_TDX_GUEST +DECLARE_IDTENTRY (X86_TRAP_VE , exc_virtualization_exception); +#endif +/* Device interrupts common/spurious */ DECLARE_IDTENTRY_IRQ (X86_TRAP_OTHER , common_interrupt);#ifdef CONFIG_X86_LOCAL_APIC
arch/x86/include/asm/tdx.h +20 -0
@@ -11,6 +11,7 @@#include <linux/types.h> #define TDINFO 1 +#define TDGETVEINFO 3struct tdx_module_output {u64 rcx;
@@ -29,6 +30,25 @@ struct tdx_hypercall_output {u64 r15; }; +/* +* Used by #VE exception handler to gather the #VE exception +* info from the TDX module. This is software only structure +* and not related to TDX module/VMM. +*/ +struct ve_info { +u64 exit_reason; +u64 exit_qual; +u64 gla; +u64 gpa; +u32 instr_len; +u32 instr_info; +}; + +unsigned long tdg_get_ve_info (struct ve_info *ve); + +int tdg_handle_virtualization_exception (struct pt_regs *regs, +struct ve_info *ve); +/* Common API to check TDX support in decompression and common kernel code. */ bool is_tdx_guest (void );
arch/x86/kernel/idt.c +6 -0
@@ -64,6 +64,9 @@ static const __initconst struct idt_data early_idts[] = {*/ INTG(X86_TRAP_PF, asm_exc_page_fault),#endif +#ifdef CONFIG_INTEL_TDX_GUEST + INTG(X86_TRAP_VE, asm_exc_virtualization_exception), +#endif };/*
@@ -87,6 +90,9 @@ static const __initconst struct idt_data def_idts[] = {INTG (X86_TRAP_MF , asm_exc_coprocessor_error),INTG (X86_TRAP_AC , asm_exc_alignment_check),INTG (X86_TRAP_XF , asm_exc_simd_coprocessor_error), +#ifdef CONFIG_INTEL_TDX_GUEST +INTG (X86_TRAP_VE , asm_exc_virtualization_exception), +#endif #ifdef CONFIG_X86_32 TSKG(X86_TRAP_DF, GDT_ENTRY_DOUBLEFAULT_TSS),
arch/x86/kernel/tdx.c +35 -0
@@ -85,6 +85,41 @@ static void tdg_get_info(void) td_info.attributes = out.rdx; } +unsigned long tdg_get_ve_info (struct ve_info *ve) +{ +u64 ret; +struct tdx_module_output out = {0 }; + +/* +* NMIs and machine checks are suppressed. Before this point any +* #VE is fatal. After this point (TDGETVEINFO call), NMIs and +* additional #VEs are permitted (but we don't expect them to +* happen unless you panic). +*/ + ret =__tdx_module_call (TDGETVEINFO ,0 ,0 ,0 ,0 , &out); + + ve->exit_reason = out.rcx; + ve->exit_qual = out.rdx; + ve->gla = out.r8; + ve->gpa = out.r9; + ve->instr_len = out.r10 &UINT_MAX ; + ve->instr_info = out.r10 >>32 ; + +return ret; +} + +int tdg_handle_virtualization_exception (struct pt_regs *regs, +struct ve_info *ve) +{ +/* +* TODO: Add handler support for various #VE exit +* reasons. It will be added by other patches in +* the series. +*/ +pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason); +return -EFAULT ; +} +void __init tdx_early_init (void ) {if (!cpuid_has_tdx_guest ())
arch/x86/kernel/traps.c +70 -0
82 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC 1/1 x86/tdx: Wire up KVM hypercalls
In reply to Dave HansenFrom: "Kirill A. Shutemov" <[email protected]>
KVM hypercalls use the "vmcall" or "vmmcall" instructions. Although the ABI is similar, those instructions no longer function for TDX guests. Make vendor specififc TDVMCALLs instead of VMCALL.
---
Changes since RFC v2:
* Introduced INTEL_TDX_GUEST_KVM config for TDX+KVM related changes.
* Removed "C" include file.
* Fixed commit log as per Dave's comments.
arch/x86/Kconfig | 6 +++++ arch/x86/include/asm/kvm_para.h | 21 +++++++++++++++ arch/x86/include/asm/tdx.h | 41 ++++++++++++++++++++++++++++ arch/x86/kernel/Makefile | 1 + arch/x86/kernel/tdcall.S | 20 ++++++++++++++ arch/x86/kernel/tdx-kvm.c | 48 +++++++++++++++++++++++++++++++++ 6 files changed, 137 insertions(+) create mode 100644 arch/x86/kernel/tdx-kvm.c
arch/x86/Kconfig +6 -0
@@ -886,6 +886,12 @@ config INTEL_TDX_GUEST run in a CPU mode that protects the confidentiality of TD memory contents and the TD’s CPU state from other software, including VMM. +config INTEL_TDX_GUEST_KVM + def_bool y + depends on KVM_GUEST && INTEL_TDX_GUEST + help + This option enables KVM specific hypercalls in TDX guest. + endif #HYPERVISOR_GUEST source "arch/x86/Kconfig.cpu"
arch/x86/include/asm/kvm_para.h +21 -0
@@ -6,6 +6,7 @@#include <asm/alternative.h> #include <linux/interrupt.h> #include <uapi/asm/kvm_para.h> +#include <asm/tdx.h> extern void kvmclock_init (void );
@@ -34,6 +35,10 @@ static inline bool kvm_check_and_clear_guest_paused(void)static inline long kvm_hypercall0 (unsigned int nr) {long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall0 (nr); + asm volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr)
@@ -44,6 +49,10 @@ static inline long kvm_hypercall0(unsigned int nr)static inline long kvm_hypercall1 (unsigned int nr,unsigned long p1) {long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall1 (nr, p1); + asm volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1)
@@ -55,6 +64,10 @@ static inline long kvm_hypercall2(unsigned int nr, unsigned long p1, unsigned long p2) {long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall2 (nr, p1, p2); + asm volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2)
@@ -66,6 +79,10 @@ static inline long kvm_hypercall3(unsigned int nr, unsigned long p1, unsigned long p2, unsigned long p3) { long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall3 (nr, p1, p2, p3); + asm volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2), "d"(p3)
@@ -78,6 +95,10 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned long p1, unsigned long p4) {long ret; + +if (is_tdx_guest ()) +return tdx_kvm_hypercall4 (nr, p1, p2, p3, p4); + asm volatile(KVM_HYPERCALL : "=a"(ret) : "a"(nr), "b"(p1), "c"(p2), "d"(p3), "S"(p4)
arch/x86/include/asm/tdx.h +41 -0
45 lines not on this page · raw message
arch/x86/kernel/Makefile +1 -0
@@ -128,6 +128,7 @@ obj-$(CONFIG_X86_PMEM_LEGACY_DEVICE) += pmem.o obj-$(CONFIG_JAILHOUSE_GUEST) += jailhouse.o obj-$(CONFIG_INTEL_TDX_GUEST) += tdcall.o tdx.o +obj-$(CONFIG_INTEL_TDX_GUEST_KVM) += tdx-kvm.o obj-$(CONFIG_EISA) += eisa.o obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
arch/x86/kernel/tdcall.S +20 -0
@@ -25,6 +25,8 @@ TDG_R12 | TDG_R13 | \ TDG_R14 | TDG_R15 ) +#define TDVMCALL_VENDOR_KVM 0x4d564b2e584454 /* "TDX.KVM" */ + /* * TDX guests use the TDCALL instruction to make requests to the * TDX module and hypercalls to the VMM. It is supported in
@@ -213,3 +215,21 @@ SYM_FUNC_START(__tdx_hypercall) call do_tdx_hypercall retq SYM_FUNC_END(__tdx_hypercall) + +#ifdef CONFIG_INTEL_TDX_GUEST_KVM +/* + * Helper function for KVM vendor TDVMCALLs. This assembly wrapper + * lets us reuse do_tdvmcall() for KVM-specific hypercalls ( + * TDVMCALL_VENDOR_KVM). + */ +SYM_FUNC_START(__tdx_hypercall_vendor_kvm) + /* + * R10 is not part of the function call ABI, but it is a part + * of the TDVMCALL ABI. So set it before making call to the + * do_tdx_hypercall(). + */ + movq $TDVMCALL_VENDOR_KVM, %r10 + call do_tdx_hypercall + retq +SYM_FUNC_END(__tdx_hypercall_vendor_kvm) +#endif /* CONFIG_INTEL_TDX_GUEST_KVM */
arch/x86/kernel/tdx-kvm.c new file +48 -0
48 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC 1/1 x86/tdx: Handle in-kernel MMIO
In reply to Dave HansenFrom: "Kirill A. Shutemov" <[email protected]>
In traditional VMs, MMIO tends to be implemented by giving a guest access to a mapping which will cause a VMEXIT on access. That's not possible in TDX guest. So use #VE to implement MMIO support. In TDX guest, MMIO triggers #VE with EPT_VIOLATION exit reason.
For now we only handle a subset of instructions that the kernel uses for MMIO operations. User-space access triggers SIGBUS.
Also, reasons for supporting #VE based MMIO in TDX guest are,
* MMIO is widely used and we'll have more drivers in the future.
* We don't want to annotate every TDX specific MMIO readl/writel etc.
* If we didn't annotate we would need to add an alternative to every MMIO access in the kernel (even though 99.9% will never be used on TDX) which would be a complete waste and incredible binary bloat for nothing.
---
Changes since RFC v2:
* Fixed commit log as per Dave's review.
arch/x86/kernel/tdx.c | 100 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+)
arch/x86/kernel/tdx.c +100 -0
@@ -5,6 +5,8 @@#include <asm/tdx.h> #include <asm/vmx.h> +#include <asm/insn.h> +#include <linux/sched/signal.h> /* force_sig_fault() */ #include <linux/cpu.h> #include <linux/protected_guest.h>
@@ -209,6 +211,101 @@ static void tdg_handle_io(struct pt_regs *regs, u32 exit_qual) } } +static unsigned long tdg_mmio (int size,bool write,unsigned long addr, +unsigned long val) +{ +return tdx_hypercall_out_r11 (EXIT_REASON_EPT_VIOLATION , size, + write, addr, val); +} + +static inline void *get_reg_ptr (struct pt_regs *regs,struct insn *insn) +{ +static const int regoff[] = { + offsetof(struct pt_regs , ax), + offsetof(struct pt_regs , cx), + offsetof(struct pt_regs , dx), + offsetof(struct pt_regs , bx), + offsetof(struct pt_regs , sp), + offsetof(struct pt_regs , bp), + offsetof(struct pt_regs , si), + offsetof(struct pt_regs , di), + offsetof(struct pt_regs , r8), + offsetof(struct pt_regs , r9), + offsetof(struct pt_regs , r10), + offsetof(struct pt_regs , r11), + offsetof(struct pt_regs , r12), + offsetof(struct pt_regs , r13), + offsetof(struct pt_regs , r14), + offsetof(struct pt_regs , r15), + }; +int regno; + + regno =X86_MODRM_REG (insn->modrm.value); +if (X86_REX_R (insn->rex_prefix.value)) + regno +=8 ; + +return (void *)regs + regoff[regno]; +} + +static int tdg_handle_mmio (struct pt_regs *regs,struct ve_info *ve) +{ +int size; +bool write; +unsigned long *reg; +struct insn insn; +unsigned long val =0 ; + +/* +* User mode would mean the kernel exposed a device directly +* to ring3, which shouldn't happen except for things like +* DPDK. +*/ +if (user_mode (regs)) { +pr_err ("Unexpected user-mode MMIO access.\n" ); +force_sig_fault (SIGBUS ,BUS_ADRERR , (void __user *) ve->gla); +return 0 ; + } + +kernel_insn_init (&insn, (void *) regs->ip,MAX_INSN_SIZE ); +insn_get_length (&insn); +insn_get_opcode (&insn); + + write = ve->exit_qual &0x2 ; + + size = insn.opnd_bytes; +switch (insn.opcode.bytes[0 ]) { +/* MOV r/m8 r8 */ +case 0x88 : +/* MOV r8 r/m8 */ +case 0x8A : +/* MOV r/m8 imm8 */ +case 0xC6 : + size =1 ; +break ; + } + +if (inat_has_immediate (insn.attr)) { +BUG_ON (!write); + val = insn.immediate.value; +tdg_mmio (size, write, ve->gpa, val); +return insn.length; + } + +BUG_ON (!inat_has_modrm (insn.attr)); + + reg =get_reg_ptr (regs, &insn); + +if (write) { +memcpy (&val, reg, size); +tdg_mmio (size, write, ve->gpa, val); + }else { + val =tdg_mmio (size, write, ve->gpa, val); +memset (reg,0 , size); +memcpy (reg, &val, size); + } +return insn.length; +} +unsigned long tdg_get_ve_info (struct ve_info *ve) {u64 ret;
@@ -258,6 +355,9 @@ int tdg_handle_virtualization_exception(struct pt_regs *regs,case EXIT_REASON_IO_INSTRUCTION :tdg_handle_io (regs, ve->exit_qual);break ; +case EXIT_REASON_EPT_VIOLATION : + ve->instr_len =tdg_handle_mmio (regs, ve); +break ;default :pr_warn ("Unexpected #VE: %lld\n" , ve->exit_reason);return -EFAULT ;
Signature · 1 line
2.25.1
RFC 1/1 x86/boot: Add a trampoline for APs booting in 64-bit mode
In reply to Dan WilliamsFrom: Sean Christopherson <[email protected]>
Add a trampoline for booting APs in 64-bit mode via a software handoff with BIOS, and use the new trampoline for the ACPI MP wake protocol used by TDX. You can find MADT MP wake protocol details in ACPI specification r6.4, sec 5.2.12.19.
Extend the real mode IDT pointer by four bytes to support LIDT in 64-bit mode. For the GDT pointer, create a new entry as the existing storage for the pointer occupies the zero entry in the GDT itself.
---
Changes since RFC v2:
* Removed X86_CR0_NE and EFER related changes from this changes and moved it to patch titled "x86/boot: Avoid #VE during boot for TDX platforms"
* Fixed commit log as per Dan's suggestion.
* Added inline get_trampoline_start_ip() to set start_ip.
arch/x86/boot/compressed/pgtable.h | 2 +- arch/x86/include/asm/realmode.h | 10 +++++++ arch/x86/kernel/smpboot.c | 2 +- arch/x86/realmode/rm/header.S | 1 + arch/x86/realmode/rm/trampoline_64.S | 38 ++++++++++++++++++++++++ arch/x86/realmode/rm/trampoline_common.S | 7 ++++- 6 files changed, 57 insertions(+), 3 deletions(-)
arch/x86/boot/compressed/pgtable.h +1 -1
@@ -6,7 +6,7 @@#define TRAMPOLINE_32BIT_PGTABLE_OFFSET 0#define TRAMPOLINE_32BIT_CODE_OFFSET PAGE_SIZE -#define TRAMPOLINE_32BIT_CODE_SIZE 0x70 +#define TRAMPOLINE_32BIT_CODE_SIZE 0x80#define TRAMPOLINE_32BIT_STACK_END TRAMPOLINE_32BIT_SIZE
arch/x86/include/asm/realmode.h +10 -0
@@ -25,6 +25,7 @@ struct real_mode_header {u32 sev_es_trampoline_start;#endif #ifdef CONFIG_X86_64 +u32 trampoline_start64;u32 trampoline_pgd;#endif /* ACPI S3 wakeup */
@@ -88,6 +89,15 @@ static inline void set_real_mode_mem(phys_addr_t mem) real_mode_header = (struct real_mode_header *) __va(mem); } +static inline unsigned long get_trampoline_start_ip (void ) +{ +#ifdef CONFIG_X86_64 +if (is_tdx_guest ()) +return real_mode_header->trampoline_start64; +#endif +return real_mode_header->trampoline_start; +} +void reserve_real_mode (void );#endif /* __ASSEMBLY__ */
arch/x86/kernel/smpboot.c +1 -1
@@ -1031,7 +1031,7 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle, int *cpu0_nmi_registered) {/* start_ip had better be page-aligned! */ -unsigned long start_ip = real_mode_header->trampoline_start; +unsigned long start_ip =get_trampoline_start_ip ();unsigned long boot_error =0 ;unsigned long timeout;
arch/x86/realmode/rm/header.S +1 -0
@@ -24,6 +24,7 @@ SYM_DATA_START(real_mode_header) .long pa_sev_es_trampoline_start #endif #ifdef CONFIG_X86_64 + .long pa_trampoline_start64 .long pa_trampoline_pgd; #endif /* ACPI S3 wakeup */
arch/x86/realmode/rm/trampoline_64.S +38 -0
@@ -161,6 +161,19 @@ SYM_CODE_START(startup_32) ljmpl $__KERNEL_CS, $pa_startup_64 SYM_CODE_END(startup_32) +SYM_CODE_START(pa_trampoline_compat) + /* + * In compatibility mode. Prep ESP and DX for startup_32, then disable + * paging and complete the switch to legacy 32-bit mode. + */ + movl $rm_stack_end, %esp + movw $__KERNEL_DS, %dx + + movl $(X86_CR0_NE | X86_CR0_PE), %eax + movl %eax, %cr0 + ljmpl $__KERNEL32_CS, $pa_startup_32 +SYM_CODE_END(pa_trampoline_compat) + .section ".text64","ax" .code64 .balign 4
@@ -169,6 +182,20 @@ SYM_CODE_START(startup_64) jmpq *tr_start(%rip) SYM_CODE_END(startup_64) +SYM_CODE_START(trampoline_start64) + /* + * APs start here on a direct transfer from 64-bit BIOS with identity + * mapped page tables. Load the kernel's GDT in order to gear down to + * 32-bit mode (to handle 4-level vs. 5-level paging), and to (re)load + * segment registers. Load the zero IDT so any fault triggers a + * shutdown instead of jumping back into BIOS. + */ + lidt tr_idt(%rip) + lgdt tr_gdt64(%rip) + + ljmpl *tr_compat(%rip) +SYM_CODE_END(trampoline_start64) + .section ".rodata","a" # Duplicate the global descriptor table # so the kernel can live anywhere
@@ -182,6 +209,17 @@ SYM_DATA_START(tr_gdt) .quad 0x00cf93000000ffff # __KERNEL_DS SYM_DATA_END_LABEL(tr_gdt, SYM_L_LOCAL, tr_gdt_end) +SYM_DATA_START(tr_gdt64) + .short tr_gdt_end - tr_gdt - 1 # gdt limit + .long pa_tr_gdt + .long 0 +SYM_DATA_END(tr_gdt64) + +SYM_DATA_START(tr_compat) + .long pa_trampoline_compat + .short __KERNEL32_CS +SYM_DATA_END(tr_compat) + .bss .balign PAGE_SIZE SYM_DATA(trampoline_pgd, .space PAGE_SIZE)
arch/x86/realmode/rm/trampoline_common.S +6 -1
@@ -1,4 +1,9 @@ /* SPDX-License-Identifier: GPL-2.0 */ .section ".rodata","a" .balign 16 -SYM_DATA_LOCAL(tr_idt, .fill 1, 6, 0) + +/* .fill cannot be used for size > 8. So use short and quad */ +SYM_DATA_START_LOCAL(tr_idt) + .short 0 + .quad 0 +SYM_DATA_END(tr_idt)
Signature · 1 line
2.25.1
RFC 1/1 x86/boot: Avoid #VE during boot for TDX platforms
In reply to Dan WilliamsFrom: Sean Christopherson <[email protected]>
Avoid operations which will inject #VE during boot process, which is obviously fatal for TDX platforms.
Details are,
1. TDX module injects #VE if a TDX guest attempts to write EFER.
Boot code updates EFER in following cases:
* When enabling Long Mode configuration, EFER.LME bit will be set. Since TDX forces EFER.LME=1, we can skip updating it again. Check for EFER.LME before updating it and skip it if it is already set.
* EFER is also updated to enable support for features like System call and No Execute page setting. In TDX, these features are set up by the TDX module. So check whether it is already enabled, and skip enabling it again.
2. TDX module also injects a #VE if the guest attempts to clear CR0.NE. Ensure CR0.NE is set when loading CR0 during compressed boot. The Setting CR0.NE should be a nop on all CPUs that support 64-bit mode.
3. The TDX-Module (effectively part of the hypervisor) requires CR4.MCE to be set at all times and injects a #VE if the guest attempts to clear CR4.MCE. So, preserve CR4.MCE instead of clearing it during boot to avoid #VE.
---
Changes since RFC v2:
* Merged Avoid #VE related changes together.
* [RFC v2 22/32] x86/boot: Avoid #VE during compressed boot for TDX platforms
* [RFC v2 23/32] x86/boot: Avoid unnecessary #VE during boot process.
* Fixed commit log as per review comments.
arch/x86/boot/compressed/head_64.S | 10 +++++++--- arch/x86/kernel/head_64.S | 13 +++++++++++-- arch/x86/realmode/rm/trampoline_64.S | 11 +++++++++-- 3 files changed, 27 insertions(+), 7 deletions(-)
arch/x86/boot/compressed/head_64.S +7 -3
@@ -616,12 +616,16 @@ SYM_CODE_START(trampoline_32bit_src) movl $MSR_EFER, %ecx rdmsr btsl $_EFER_LME, %eax + jc 1f wrmsr - popl %edx +1: popl %edx popl %ecx /* Enable PAE and LA57 (if required) paging modes */ - movl $X86_CR4_PAE, %eax + movl %cr4, %eax + /* Clearing CR4.MCE will #VE on TDX guests. Leave it alone. */ + andl $X86_CR4_MCE, %eax + orl $X86_CR4_PAE, %eax testl %edx, %edx jz 1f orl $X86_CR4_LA57, %eax
@@ -636,7 +640,7 @@ SYM_CODE_START(trampoline_32bit_src) pushl %eax /* Enable paging again */ - movl $(X86_CR0_PG | X86_CR0_PE), %eax + movl $(X86_CR0_PG | X86_CR0_NE | X86_CR0_PE), %eax movl %eax, %cr0 lret
arch/x86/kernel/head_64.S +11 -2
@@ -141,7 +141,10 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) 1: /* Enable PAE mode, PGE and LA57 */ - movl $(X86_CR4_PAE | X86_CR4_PGE), %ecx + movq %cr4, %rcx + /* Clearing CR4.MCE will #VE on TDX guests. Leave it alone. */ + andl $X86_CR4_MCE, %ecx + orl $(X86_CR4_PAE | X86_CR4_PGE), %ecx #ifdef CONFIG_X86_5LEVEL testl $1, __pgtable_l5_enabled(%rip) jz 1f
@@ -229,13 +232,19 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) /* Setup EFER (Extended Feature Enable Register) */ movl $MSR_EFER, %ecx rdmsr + movl %eax, %edx btsl $_EFER_SCE, %eax /* Enable System Call */ btl $20,%edi /* No Execute supported? */ jnc 1f btsl $_EFER_NX, %eax btsq $_PAGE_BIT_NX,early_pmd_flags(%rip) -1: wrmsr /* Make changes effective */ + /* Skip the WRMSR if the current value matches the desired value. */ +1: cmpl %edx, %eax + je 1f + xor %edx, %edx + wrmsr /* Make changes effective */ +1: /* Setup cr0 */ movl $CR0_STATE, %eax /* Make changes effective */
arch/x86/realmode/rm/trampoline_64.S +9 -2
@@ -143,13 +143,20 @@ SYM_CODE_START(startup_32) movl %eax, %cr3 # Set up EFER + movl $MSR_EFER, %ecx + rdmsr + cmp pa_tr_efer, %eax + jne .Lwrite_efer + cmp pa_tr_efer + 4, %edx + je .Ldone_efer +.Lwrite_efer: movl pa_tr_efer, %eax movl pa_tr_efer + 4, %edx - movl $MSR_EFER, %ecx wrmsr +.Ldone_efer: # Enable paging and in turn activate Long Mode - movl $(X86_CR0_PG | X86_CR0_WP | X86_CR0_PE), %eax + movl $(X86_CR0_PG | X86_CR0_WP | X86_CR0_NE | X86_CR0_PE), %eax movl %eax, %cr0 /*
Signature · 1 line
2.25.1
RFC 1/1 x86/tdx: Make DMA pages shared
In reply to Kuppuswamy SathyanarayananFrom: "Kirill A. Shutemov" <[email protected]>
Intel TDX doesn't allow VMM to access guest memory. Any memory that is required for communication with VMM must be shared explicitly by setting the bit in page table entry. And, after setting the shared bit, the conversion must be completed with MapGPA TDVMALL. The call informs VMM about the conversion and makes it remove the GPA from the S-EPT mapping. The shared memory is similar to unencrypted memory in AMD SME/SEV terminology but the underlying process of sharing/un-sharing the memory is different for Intel TDX guest platform.
SEV assumes that I/O devices can only do DMA to "decrypted" physical addresses without the C-bit set. In order for the CPU to interact with this memory, the CPU needs a decrypted mapping. To add this support, AMD SME code forces force_dma_unencrypted() to return true for platforms that support AMD SEV feature. It will be used for DMA memory allocation API to trigger set_memory_decrypted() for platforms that support AMD SEV feature.
TDX is similar. TDX architecturally prevents access to private guest memory by anything other than the guest itself. This means that any DMA buffers must be shared.
So create a new file mem_encrypt_tdx.c to hold TDX specific memory initialization code, and re-define force_dma_unencrypted() for TDX guest and make it return true to get DMA pages mapped as shared.
__set_memory_enc_dec() is now aware about TDX and sets Shared bit accordingly following with relevant TDVMCALL.
Also, Do TDACCEPTPAGE on every 4k page after mapping the GPA range when converting memory to private. If the VMM uses a common pool for private and shared memory, it will likely do TDAUGPAGE in response to MAP_GPA (or on the first access to the private GPA), in which case TDX-Module will hold the page in a non-present "pending" state until it is explicitly accepted.
BUG() if TDACCEPTPAGE fails (except the above case), as the guest is completely hosed if it can't access memory.
---
Changes since RFC v2:
* Since the common code between AMD-SEV and TDX is very minimal, defining a new config (X86_MEM_ENCRYPT_COMMON) for common code is not very useful. So createed a seperate file for Intel TDX specific memory initialization (similar to AMD SEV).
* Removed patch titled "x86/mm: Move force_dma_unencrypted() to common code" from this series. And merged required changes in this patch.
arch/x86/Kconfig | 1 + arch/x86/include/asm/tdx.h | 3 +++ arch/x86/kernel/tdx.c | 26 ++++++++++++++++++- arch/x86/mm/Makefile | 1 + arch/x86/mm/mem_encrypt_tdx.c | 19 ++++++++++++++ arch/x86/mm/pat/set_memory.c | 48 +++++++++++++++++++++++++++++------ 6 files changed, 89 insertions(+), 9 deletions(-) create mode 100644 arch/x86/mm/mem_encrypt_tdx.c
arch/x86/Kconfig +1 -0
@@ -879,6 +879,7 @@ config INTEL_TDX_GUEST select X86_X2APIC select SECURITY_LOCKDOWN_LSM select ARCH_HAS_PROTECTED_GUEST + select ARCH_HAS_FORCE_DMA_UNENCRYPTED select DYNAMIC_PHYSICAL_MASK help Provide support for running in a trusted domain on Intel processors
arch/x86/include/asm/tdx.h +3 -0
@@ -19,6 +19,9 @@ enum tdx_map_type {#define TDINFO 1#define TDGETVEINFO 3 +#define TDACCEPTPAGE 6 + +#define TDX_PAGE_ALREADY_ACCEPTED 0x8000000000000001struct tdx_module_output {u64 rcx;
arch/x86/kernel/tdx.c +25 -1
@@ -100,7 +100,8 @@ static void tdg_get_info(void) physical_mask &= ~tdg_shared_mask (); } -int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +static int __tdg_map_gpa (phys_addr_t gpa,int numpages, +enum tdx_map_type map_type) {u64 ret;
@@ -111,6 +112,29 @@ int tdg_map_gpa(phys_addr_t gpa, int numpages, enum tdx_map_type map_type)return ret ? -EIO :0 ; } +static void tdg_accept_page (phys_addr_t gpa) +{ +u64 ret; + + ret =__tdx_module_call (TDACCEPTPAGE , gpa,0 ,0 ,0 ,NULL ); + +BUG_ON (ret && ret !=TDX_PAGE_ALREADY_ACCEPTED ); +} + +int tdg_map_gpa (phys_addr_t gpa,int numpages,enum tdx_map_type map_type) +{ +int ret, i; + + ret =__tdg_map_gpa (gpa, numpages, map_type); +if (ret || map_type ==TDX_MAP_SHARED ) +return ret; + +for (i =0 ; i < numpages; i++) +tdg_accept_page (gpa + i*PAGE_SIZE ); + +return 0 ; +} + static __cpuidle void tdg_halt(void) {u64 ret;
arch/x86/mm/Makefile +1 -0
@@ -55,3 +55,4 @@ obj-$(CONFIG_PAGE_TABLE_ISOLATION) += pti.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_identity.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_boot.o +obj-$(CONFIG_INTEL_TDX_GUEST) += mem_encrypt_tdx.o
arch/x86/mm/mem_encrypt_tdx.c new file +19 -0
@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Intel TDX Memory Encryption Support +* +* Copyright (C) 2020 Intel Corporation +* +* Author: Kuppuswamy Sathyanarayanan <[email protected]> +*/ + +#include <linux/mm.h> +#include <linux/dma-mapping.h> + +#include <asm/tdx.h> + +/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ +bool force_dma_unencrypted (struct device *dev) +{ +return is_tdx_guest (); +}
arch/x86/mm/pat/set_memory.c +40 -8
93 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [RFC v2 26/32] x86/mm: Move force_dma_unencrypted() to common code
In reply to Dave HansenQuoted 7 lines · Dave Hansen
Because the code is already separate. You're actually going to some trouble to move the SEV-specific code and then combine it with the TDX-specific code.
Anyway, please just give it a shot. Should take all of ten minutes. If it doesn't work out in practice, fine. You'll have a good paragraph for the changelog.
After reviewing the code again, I have noticed that we don't really have much common code between AMD and TDX. So I don't see any justification for creating this common layer. So, I have decided to drop this patch and move Intel TDX specific memory encryption init code to patch titled "[RFC v2 30/32] x86/tdx: Make DMA pages shared". This model is similar to how AMD-SEV does the initialization.
I have sent the modified patch as reply to patch titled "[RFC v2 30/32] x86/tdx: Make DMA pages shared". Please check and let me know your comments.
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2-fix 1/1] x86/boot: Add a trampoline for APs booting in 64-bit mode
In reply to Kuppuswamy SathyanarayananI notice that you have [RFC v2-fix 1/1] as the prefix for this patch. b4 recently gained support for partial series re-rolls [1], but I think you would need to bump the version number [RFC PATCH v3 21/32] and maintain the patch numbering. In this case with changes moving between patches, and those other patches being squashed any chance of automated reconstruction of this series is likely lost.
Just wanted to note that for future reference in case you were hoping to avoid resending full series in the future. For now, some more comments below:
[1]: https://lore.kernel.org/tools/[email protected]/
Quoted 12 lines · Kuppuswamy Sathyanarayanan
From: Sean Christopherson <[email protected]>
Add a trampoline for booting APs in 64-bit mode via a software handoff with BIOS, and use the new trampoline for the ACPI MP wake protocol used by TDX. You can find MADT MP wake protocol details in ACPI specification r6.4, sec 5.2.12.19.
Extend the real mode IDT pointer by four bytes to support LIDT in 64-bit mode. For the GDT pointer, create a new entry as the existing storage for the pointer occupies the zero entry in the GDT itself.
Quoted 8 lines
Reported-by Kai Huang <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Reviewed-by Andi Kleen <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>---
Changes since RFC v2:
* RemovedX86_CR0_NEand EFER related changes from this changes
This was only partially done, see below...
Quoted 4 lines
and moved it to patch titled "x86/boot: Avoid #VE during boot for TDX platforms"
* Fixed commit log as per Dan's suggestion.
* Added inlineget_trampoline_start_ip()to setstart_ip.
You also added a comment to tr_idt, but didn't mention it here, so I went to double check. Please take care to document all changes to the patch from the previous review.
Quoted 35 lines
arch/x86/boot/compressed/pgtable.h | 2 +- arch/x86/include/asm/realmode.h | 10 +++++++ arch/x86/kernel/smpboot.c | 2 +- arch/x86/realmode/rm/header.S | 1 + arch/x86/realmode/rm/trampoline_64.S | 38 ++++++++++++++++++++++++ arch/x86/realmode/rm/trampoline_common.S | 7 ++++- 6 files changed, 57 insertions(+), 3 deletions(-)arch/x86/boot/compressed/pgtable.h +1
-1@@ -6,7 +6,7 @@#define TRAMPOLINE_32BIT_PGTABLE_OFFSET 0#define TRAMPOLINE_32BIT_CODE_OFFSET PAGE_SIZE -#define TRAMPOLINE_32BIT_CODE_SIZE 0x70 +#define TRAMPOLINE_32BIT_CODE_SIZE 0x80#define TRAMPOLINE_32BIT_STACK_END TRAMPOLINE_32BIT_SIZEarch/x86/include/asm/realmode.h +1
-0@@ -25,6 +25,7 @@ struct real_mode_header {u32 sev_es_trampoline_start;#endif #ifdef CONFIG_X86_64 +u32 trampoline_start64;u32 trampoline_pgd;#endif /* ACPI S3 wakeup */
arch/x86/include/asm/realmode.h static inline void set_real_mode_mem(phys_addr_t mem) · line 92
Quoted 4 lines
arch/x86/include/asm/realmode.h +1
-0@@ -88,6 +89,15 @@ static inline void set_real_mode_mem(phys_addr_t mem) real_mode_header = (struct real_mode_header *)__va (mem); } +static inline unsigned long get_trampoline_start_ip (void )
I'd prefer this helper take a 'struct real_mode_header *rmh' as an argument rather than assume a global variable.
Quoted 43 lines
arch/x86/include/asm/realmode.h +8
-0+{ +#ifdef CONFIG_X86_64 +if (is_tdx_guest ()) +return real_mode_header->trampoline_start64; +#endif +return real_mode_header->trampoline_start; +} +void reserve_real_mode (void );#endif /* __ASSEMBLY__ */ arch/x86/kernel/smpboot.c +1
-1@@ -1031,7 +1031,7 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle, int *cpu0_nmi_registered) {/* start_ip had better be page-aligned! */ -unsigned long start_ip = real_mode_header->trampoline_start; +unsigned long start_ip =get_trampoline_start_ip ();unsigned long boot_error =0 ;unsigned long timeout;arch/x86/realmode/rm/header.S +1
-0@@ -24,6 +24,7 @@ SYM_DATA_START(real_mode_header) .long pa_sev_es_trampoline_start #endif #ifdef CONFIG_X86_64 + .long pa_trampoline_start64 .long pa_trampoline_pgd; #endif /* ACPI S3 wakeup */arch/x86/realmode/rm/trampoline_64.S +0
-0@@ -161,6 +161,19 @@ SYM_CODE_START(startup_32) ljmpl $__KERNEL_CS, $pa_startup_64 SYM_CODE_END(startup_32)
arch/x86/realmode/rm/trampoline_64.S SYM_CODE_START(startup_32) · line 172
Quoted 10 lines
arch/x86/realmode/rm/trampoline_64.S +9
-0+SYM_CODE_START(pa_trampoline_compat) + /* + * In compatibility mode. Prep ESP and DX for startup_32, then disable + * paging and complete the switch to legacy 32-bit mode. + */ + movl $rm_stack_end, %esp + movw $__KERNEL_DS, %dx + + movl $(X86_CR0_NE | X86_CR0_PE), %eax
Before this patch the startup path did not touch X86_CR0_NE. I assume it was added opportunistically for the TDX case? If it is to stay in this patch it deserves a code comment / mention in the changelog, or it needs to move to the other patch that fixes up the CR0 setup for TDX.
Quoted 51 lines
arch/x86/realmode/rm/trampoline_64.S +29
-0+ movl %eax, %cr0 + ljmpl $__KERNEL32_CS, $pa_startup_32 +SYM_CODE_END(pa_trampoline_compat) + .section ".text64","ax" .code64 .balign 4@@ -169,6 +182,20 @@ SYM_CODE_START(startup_64) jmpq *tr_start(%rip) SYM_CODE_END(startup_64) +SYM_CODE_START(trampoline_start64) + /* + * APs start here on a direct transfer from 64-bit BIOS with identity + * mapped page tables. Load the kernel's GDT in order to gear down to + * 32-bit mode (to handle 4-level vs. 5-level paging), and to (re)load + * segment registers. Load the zero IDT so any fault triggers a + * shutdown instead of jumping back into BIOS. + */ + lidt tr_idt(%rip) + lgdt tr_gdt64(%rip) + + ljmpl *tr_compat(%rip) +SYM_CODE_END(trampoline_start64) + .section ".rodata","a" # Duplicate the global descriptor table # so the kernel can live anywhere@@ -182,6 +209,17 @@ SYM_DATA_START(tr_gdt) .quad 0x00cf93000000ffff # __KERNEL_DS SYM_DATA_END_LABEL(tr_gdt, SYM_L_LOCAL, tr_gdt_end) +SYM_DATA_START(tr_gdt64) + .short tr_gdt_end - tr_gdt - 1 # gdt limit + .long pa_tr_gdt + .long 0 +SYM_DATA_END(tr_gdt64) + +SYM_DATA_START(tr_compat) + .long pa_trampoline_compat + .short __KERNEL32_CS +SYM_DATA_END(tr_compat) + .bss .balign PAGE_SIZE SYM_DATA(trampoline_pgd, .space PAGE_SIZE)arch/x86/realmode/rm/trampoline_common.S +0
-0
arch/x86/realmode/rm/trampoline_common.S line 5
Quoted 6 lines
arch/x86/realmode/rm/trampoline_common.S +2
-1@@ -1,4 +1,9 @@ /* SPDX-License-Identifier: GPL-2.0 */ .section ".rodata","a" .balign 16 -SYM_DATA_LOCAL(tr_idt, .fill 1, 6, 0) + +/* .fill cannot be used for size > 8. So use short and quad */
If there is to be a comment here it should be to clarify why @tr_idt is 10 bytes, not necessarily a quirk of the assembler.
arch/x86/realmode/rm/trampoline_common.S line 6
Quoted 1 line
arch/x86/realmode/rm/trampoline_common.S +1
-0+SYM_DATA_START_LOCAL(tr_idt)
The .fill restriction is only for @size, not @repeat. So, what's wrong with SYM_DATA_LOCAL(tr_idt, .fill 2, 5, 0)?
Re: [RFC v2-fix 1/1] x86/boot: Add a trampoline for APs booting in 64-bit mode
Quoted 6 lines · Dan Williams
I notice that you have [RFC v2-fix 1/1] as the prefix for this patch. b4 recently gained support for partial series re-rolls [1], but I think you would need to bump the version number [RFC PATCH v3 21/32] and maintain the patch numbering. In this case with changes moving between patches, and those other patches being squashed any chance of automated reconstruction of this series is likely lost.
Ok. I will make sure to bump the version in next partial re-roll.
If I am fixing this patch as per your comments, do I need bump the patch version for it as well?
Quoted 4 lines
Just wanted to note that for future reference in case you were hoping to avoid resending full series in the future. For now, some more comments below:
Thanks.
Quoted 28 lines
[1]: https://lore.kernel.org/tools/[email protected]/
Quoted 12 lines · Kuppuswamy Sathyanarayanan
From: Sean Christopherson <[email protected]>
Add a trampoline for booting APs in 64-bit mode via a software handoff with BIOS, and use the new trampoline for the ACPI MP wake protocol used by TDX. You can find MADT MP wake protocol details in ACPI specification r6.4, sec 5.2.12.19.
Extend the real mode IDT pointer by four bytes to support LIDT in 64-bit mode. For the GDT pointer, create a new entry as the existing storage for the pointer occupies the zero entry in the GDT itself.
Quoted 8 lines
Reported-by Kai Huang <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Reviewed-by Andi Kleen <[email protected]>Signed-off-by Kuppuswamy Sathyanarayanan <[email protected]>---
Changes since RFC v2:
* RemovedX86_CR0_NEand EFER related changes from this changesThis was only partially done, see below...
Quoted 8 lines
Quoted 4 lines
and moved it to patch titled "x86/boot: Avoid #VE during boot for TDX platforms"
* Fixed commit log as per Dan's suggestion.
* Added inlineget_trampoline_start_ip()to setstart_ip.You also added a comment to
tr_idt, but didn't mention it here, so I went to double check. Please take care to document all changes to the patch from the previous review.
Ok. I will make sure change log is current.
Quoted 39 lines
Quoted 39 lines
arch/x86/boot/compressed/pgtable.h | 2 +- arch/x86/include/asm/realmode.h | 10 +++++++ arch/x86/kernel/smpboot.c | 2 +- arch/x86/realmode/rm/header.S | 1 + arch/x86/realmode/rm/trampoline_64.S | 38 ++++++++++++++++++++++++ arch/x86/realmode/rm/trampoline_common.S | 7 ++++- 6 files changed, 57 insertions(+), 3 deletions(-)arch/x86/boot/compressed/pgtable.h +1
-1@@ -6,7 +6,7 @@#define TRAMPOLINE_32BIT_PGTABLE_OFFSET 0#define TRAMPOLINE_32BIT_CODE_OFFSET PAGE_SIZE -#define TRAMPOLINE_32BIT_CODE_SIZE 0x70 +#define TRAMPOLINE_32BIT_CODE_SIZE 0x80#define TRAMPOLINE_32BIT_STACK_END TRAMPOLINE_32BIT_SIZEarch/x86/include/asm/realmode.h +1
-0@@ -25,6 +25,7 @@ struct real_mode_header {u32 sev_es_trampoline_start;#endif #ifdef CONFIG_X86_64 +u32 trampoline_start64;u32 trampoline_pgd;#endif /* ACPI S3 wakeup */ @@ -88,6 +89,15 @@ static inline void set_real_mode_mem(phys_addr_t mem) real_mode_header = (struct real_mode_header *)__va (mem); }
Quoted 4 lines
arch/x86/include/asm/realmode.h static inline void set_real_mode_mem(phys_addr_t mem) · line 92
Quoted 1 line
arch/x86/include/asm/realmode.h +1
-0+static inline unsigned long get_trampoline_start_ip (void )I'd prefer this helper take a 'struct
real_mode_header*rmh' as an argument rather than assume a global variable.
I am fine with it. But existing inline functions also directly read/writes the real_mode_header. So I just followed the same format.
I will fix this in next version.
Quoted 45 lines
Quoted 45 lines
arch/x86/include/asm/realmode.h +8
-0+{ +#ifdef CONFIG_X86_64 +if (is_tdx_guest ()) +return real_mode_header->trampoline_start64; +#endif +return real_mode_header->trampoline_start; +} +void reserve_real_mode (void );#endif /* __ASSEMBLY__ */ arch/x86/kernel/smpboot.c +1
-1@@ -1031,7 +1031,7 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle, int *cpu0_nmi_registered) {/* start_ip had better be page-aligned! */ -unsigned long start_ip = real_mode_header->trampoline_start; +unsigned long start_ip =get_trampoline_start_ip ();unsigned long boot_error =0 ;unsigned long timeout;arch/x86/realmode/rm/header.S +1
-0@@ -24,6 +24,7 @@ SYM_DATA_START(real_mode_header) .long pa_sev_es_trampoline_start #endif #ifdef CONFIG_X86_64 + .long pa_trampoline_start64 .long pa_trampoline_pgd; #endif /* ACPI S3 wakeup */arch/x86/realmode/rm/trampoline_64.S +0
-0@@ -161,6 +161,19 @@ SYM_CODE_START(startup_32) ljmpl $__KERNEL_CS, $pa_startup_64 SYM_CODE_END(startup_32)
Quoted 15 lines
arch/x86/realmode/rm/trampoline_64.S SYM_CODE_START(startup_32) · line 172
Quoted 9 lines
arch/x86/realmode/rm/trampoline_64.S +9
-0+SYM_CODE_START(pa_trampoline_compat) + /* + * In compatibility mode. Prep ESP and DX for startup_32, then disable + * paging and complete the switch to legacy 32-bit mode. + */ + movl $rm_stack_end, %esp + movw $__KERNEL_DS, %dx + + movl $(X86_CR0_NE | X86_CR0_PE), %eaxBefore this patch the startup path did not touch
X86_CR0_NE. I assume it was added opportunistically for the TDX case? If it is to stay in this patch it deserves a code comment / mention in the changelog, or it needs to move to the other patch that fixes up the CR0 setup for TDX.
I will move X86_CR0_NE related update to the patch that has other X86_CR0_NE related updates.
Quoted 54 lines
Quoted 54 lines
arch/x86/realmode/rm/trampoline_64.S +29
-0+ movl %eax, %cr0 + ljmpl $__KERNEL32_CS, $pa_startup_32 +SYM_CODE_END(pa_trampoline_compat) + .section ".text64","ax" .code64 .balign 4@@ -169,6 +182,20 @@ SYM_CODE_START(startup_64) jmpq *tr_start(%rip) SYM_CODE_END(startup_64) +SYM_CODE_START(trampoline_start64) + /* + * APs start here on a direct transfer from 64-bit BIOS with identity + * mapped page tables. Load the kernel's GDT in order to gear down to + * 32-bit mode (to handle 4-level vs. 5-level paging), and to (re)load + * segment registers. Load the zero IDT so any fault triggers a + * shutdown instead of jumping back into BIOS. + */ + lidt tr_idt(%rip) + lgdt tr_gdt64(%rip) + + ljmpl *tr_compat(%rip) +SYM_CODE_END(trampoline_start64) + .section ".rodata","a" # Duplicate the global descriptor table # so the kernel can live anywhere@@ -182,6 +209,17 @@ SYM_DATA_START(tr_gdt) .quad 0x00cf93000000ffff # __KERNEL_DS SYM_DATA_END_LABEL(tr_gdt, SYM_L_LOCAL, tr_gdt_end) +SYM_DATA_START(tr_gdt64) + .short tr_gdt_end - tr_gdt - 1 # gdt limit + .long pa_tr_gdt + .long 0 +SYM_DATA_END(tr_gdt64) + +SYM_DATA_START(tr_compat) + .long pa_trampoline_compat + .short __KERNEL32_CS +SYM_DATA_END(tr_compat) + .bss .balign PAGE_SIZE SYM_DATA(trampoline_pgd, .space PAGE_SIZE)arch/x86/realmode/rm/trampoline_common.S +0
-0@@ -1,4 +1,9 @@ /* SPDX-License-Identifier: GPL-2.0 */
Quoted 8 lines
arch/x86/realmode/rm/trampoline_common.S line 5
Quoted 5 lines
arch/x86/realmode/rm/trampoline_common.S +2
-1.section ".rodata","a" .balign 16 -SYM_DATA_LOCAL(tr_idt, .fill 1, 6, 0) + +/* .fill cannot be used for size > 8. So use short and quad */If there is to be a comment here it should be to clarify why @tr_idt is 10 bytes, not necessarily a quirk of the assembler.
Got it. I will fix the comment or remove it.
Quoted 5 lines
arch/x86/realmode/rm/trampoline_common.S line 6
Quoted 2 lines
arch/x86/realmode/rm/trampoline_common.S +1
-0+SYM_DATA_START_LOCAL(tr_idt)The .fill restriction is only for @size, not @repeat. So, what's wrong with
SYM_DATA_LOCAL(tr_idt, .fill 2, 5, 0)?
Any reason to prefer above change over previous code ?
SYM_DATA_START_LOCAL(tr_idt)
.short 0
.quad 0SYM_DATA_END(tr_idt)
Quoted 1 line
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Re: [RFC v2-fix 1/1] x86/boot: Add a trampoline for APs booting in 64-bit mode
Quoted 11 lines · Sathyanarayanan
Quoted 6 lines · Dan Williams
I notice that you have [RFC v2-fix 1/1] as the prefix for this patch. b4 recently gained support for partial series re-rolls [1], but I think you would need to bump the version number [RFC PATCH v3 21/32] and maintain the patch numbering. In this case with changes moving between patches, and those other patches being squashed any chance of automated reconstruction of this series is likely lost.
Quoted 4 lines
Ok. I will make sure to bump the version in next partial re-roll.
If I am fixing this patch as per your comments, do I need bump the patch version for it as well?
I don't think it matters too much in this case as I don't think I can use b4 to assemble this series. So just for future reference on other patch sets. That said, I wouldn't mind a link to your work-in-progress branch to see all the changes together in one place.
[..]
Quoted 5 lines
Quoted 2 lines
I'd prefer this helper take a 'struct
real_mode_header*rmh' as an argument rather than assume a global variable.I am fine with it. But existing inline functions also directly read/writes the
real_mode_header. So I just followed the same format.
I notice the SEV-ES code passes an @rmh variable around for this purpose.
[..]
Quoted 6 lines
Quoted 2 lines
If there is to be a comment here it should be to clarify why @tr_idt is 10 bytes, not necessarily a quirk of the assembler.
Got it. I will fix the comment or remove it.
Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
+SYM_DATA_START_LOCAL(tr_idt)
The .fill restriction is only for @size, not @repeat. So, what's wrong with
SYM_DATA_LOCAL(tr_idt, .fill 2, 5, 0)?Any reason to prefer above change over previous code ?
What I'm really after is capturing why this size needs to be adjusted for future reference. Maybe it's plainly obvious to someone who has worked with this code, but it was not immediately obvious to me.
Quoted 5 lines
SYM_DATA_START_LOCAL(tr_idt)
.short 0
.quad 0SYM_DATA_END(tr_idt)
This format implies that tr_idt is reserving space for 2 distinct data structure attributes of those sizes, can you just put those names here as comments? Otherwise the .fill format is more compact.