KVM: X86: TDX support
Subsystems (8)
v3 cover KVM: X86: TDX support
From: Isaku Yamahata <[email protected]>
Changes from v2:
- update based on patch review
- support TDP MMU
- drop non-essential fetures (ftrace etc.) to reduce patch size
TODO:
- integrate vm type patch
- integrate unmapping user space mapping
---
* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform. [1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.
* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.
- 01-13: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 14-30: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 31-38: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 39-54: refactoring KVM
- 55: main patch to add "basic" support for building/running TDX.
- 56-57: TDP MMU support
- 58: support TDX hypercall, GetQuote and SetupEventNotifyInterrupt, that
requires qemu help
- 59: Documentation
* Missing features
Those major features are intentionally missing from this patch series to keep this patch series small. They are addressed as independent patch series.
- qemu gdb stub support
- Large page support
- guest PMU support
- and more
Changes from v1:
- rebase to v5.13
- drop load/initialization of TDX module
- catch up the update of related specifications.
- rework on C-wrapper function to invoke seamcall
- various code clean up
[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAS
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf
[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVF
Chao Gao (1):
KVM: x86: Add a helper function to restore 4 host MSRs on exit to user
space
Isaku Yamahata (9):
x86/mktme: move out MKTME related constatnts/macro to msr-index.h x86/mtrr: mask out keyid bits from variable mtrr mask register
KVM: TDX: Define TDX architectural definitions
KVM: TDX: add a helper function for kvm to call seamcall
KVM: TDX: Add helper functions to print TDX SEAMCALL error
KVM: Add per-VM flag to mark read-only memory as unsupported
KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
KVM: TDX: exit to user space on GET_QUOTE,
SETUP_EVENT_NOTIFY_INTERRUPT
Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)
Kai Huang (3):
KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level
routes
KVM: TDX: Protect private mapping related SEAMCALLs with spinlock KVM, x86/mmu: Support TDX private mapping for TDP MMU
Rick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bits
Sean Christopherson (44):
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Split core of hypercall emulation to helper function
KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO
KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
KVM: Add max_vcpus field in common 'struct kvm'
KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
KVM: x86: Introduce "protected guest" concept and block disallowed
ioctls
KVM: x86: Add per-VM flag to disable direct IRQ injection
KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE
KVM: x86: Add flag to mark TSC as immutable (for TDX)
KVM: Add per-VM flag to disable dirty logging of memslots for TDs
KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()
KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched
behavior
KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: x86: Add guest_supported_xss placholder
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Ignore bits 63 and 62 when checking for "present" SPTEs
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()
KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
KVM: VMX: Move NMI/exception handler to common helper
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Define EPT Violation architectural bits
KVM: VMX: Define VMCS encodings for shared EPT pointer
KVM: VMX: Add 'main.c' to wrap VMX and TDX
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: VMX: Move register caching logic to common code
KVM: TDX: Define TDCALL exit reason
KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h
KVM: VMX: MOVE GDT and IDT accessors to common code
KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX
code
KVM: TDX: Add "basic" support for building and running Trust Domains
Xiaoyao Li (1):
KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
Documentation/virt/kvm/api.rst | 9 +- Documentation/virt/kvm/intel-tdx.rst | 359 ++++ arch/arm64/include/asm/kvm_host.h | 3 - arch/arm64/kvm/arm.c | 7 +- arch/arm64/kvm/vgic/vgic-init.c | 6 +- arch/x86/events/intel/ds.c | 1 + arch/x86/include/asm/kvm-x86-ops.h | 11 + arch/x86/include/asm/kvm_host.h | 63 +- arch/x86/include/asm/msr-index.h | 16 + arch/x86/include/asm/vmx.h | 6 + arch/x86/include/uapi/asm/kvm.h | 60 + arch/x86/include/uapi/asm/vmx.h | 7 +- arch/x86/kernel/cpu/intel.c | 14 - arch/x86/kernel/cpu/mtrr/mtrr.c | 9 + arch/x86/kvm/Makefile | 6 +- arch/x86/kvm/ioapic.c | 4 + arch/x86/kvm/irq_comm.c | 13 +- arch/x86/kvm/lapic.c | 7 +- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/mmu.h | 29 +- arch/x86/kvm/mmu/mmu.c | 667 ++++++- arch/x86/kvm/mmu/mmu_internal.h | 12 + arch/x86/kvm/mmu/paging_tmpl.h | 32 +- arch/x86/kvm/mmu/spte.c | 15 +- arch/x86/kvm/mmu/spte.h | 51 +- arch/x86/kvm/mmu/tdp_iter.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 544 +++++- arch/x86/kvm/mmu/tdp_mmu.h | 15 +- arch/x86/kvm/svm/svm.c | 13 +- arch/x86/kvm/vmx/common.h | 178 ++ arch/x86/kvm/vmx/main.c | 1152 ++++++++++++ arch/x86/kvm/vmx/posted_intr.c | 6 + arch/x86/kvm/vmx/seamcall.h | 116 ++ arch/x86/kvm/vmx/tdx.c | 2437 +++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 290 +++ arch/x86/kvm/vmx/tdx_arch.h | 239 +++ arch/x86/kvm/vmx/tdx_errno.h | 111 ++ arch/x86/kvm/vmx/tdx_error.c | 53 + arch/x86/kvm/vmx/tdx_ops.h | 224 +++ arch/x86/kvm/vmx/tdx_stubs.c | 50 + arch/x86/kvm/vmx/vmenter.S | 146 ++ arch/x86/kvm/vmx/vmx.c | 689 ++----- arch/x86/kvm/vmx/x86_ops.h | 203 ++ arch/x86/kvm/x86.c | 276 ++- include/linux/kvm_host.h | 5 + include/uapi/linux/kvm.h | 59 + tools/arch/x86/include/uapi/asm/kvm.h | 55 + tools/include/uapi/linux/kvm.h | 2 + virt/kvm/kvm_main.c | 34 +- 49 files changed, 7469 insertions(+), 839 deletions(-) create mode 100644 Documentation/virt/kvm/intel-tdx.rst create mode 100644 arch/x86/kvm/vmx/common.h create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/seamcall.h create mode 100644 arch/x86/kvm/vmx/tdx.c create mode 100644 arch/x86/kvm/vmx/tdx.h create mode 100644 arch/x86/kvm/vmx/tdx_arch.h create mode 100644 arch/x86/kvm/vmx/tdx_errno.h create mode 100644 arch/x86/kvm/vmx/tdx_error.c create mode 100644 arch/x86/kvm/vmx/tdx_ops.h create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c create mode 100644 arch/x86/kvm/vmx/x86_ops.h
Signature · 1 line
2.25.1
RFC PATCH v3 2/59 x86/mtrr: mask out keyid bits from variable mtrr mask register
From: Isaku Yamahata <[email protected]>
This is a preparation for TDX support. TDX repurposes high bits of physcial address to private key ID similarly to MKTME. IA32_TME_ACTIVATE.MK_TME_KEYID_BITS has same meaning for both TDX disabled/enable for compatibility.
MTRR calculates mask based on available physical address bits. MKTME repurpose high bit of physical address to key id for key id. CPUID MAX_PA remains same and the bits stolen for key id is controlled IA32_TME_ACTIVATE MSR bit 35:32. Because Key ID bits shouldn't affects memory cachability, MTRR mask should exclude bits repourposed for Key ID. It's OS responsibility to maintain cache coherency. detect_tme @ arch/x86/kernel/cpu/intel.c detects tme and destract it from total usable physical bits. This patch adds same logic needed for MTRR.
--- arch/x86/kernel/cpu/mtrr/mtrr.c | 9 +++++++++ 1 file changed, 9 insertions(+)
arch/x86/kernel/cpu/mtrr/mtrr.c +9 -0
@@ -713,6 +713,15 @@ void __init mtrr_bp_init(void) boot_cpu_data.x86_stepping == 0x4)) phys_addr =36 ; +if (boot_cpu_has (X86_FEATURE_TME )) { +u64 tme_activate; + +rdmsrl (MSR_IA32_TME_ACTIVATE , tme_activate); +if (TME_ACTIVATE_LOCKED (tme_activate) && +TME_ACTIVATE_ENABLED (tme_activate)) { + phys_addr -=TME_ACTIVATE_KEYID_BITS (tme_activate); + } + } size_or_mask =SIZE_OR_MASK_BITS (phys_addr); size_and_mask = ~size_or_mask &0xfffff00000ULL ; } else if (boot_cpu_data.x86_vendor == X86_VENDOR_CENTAUR &&
Signature · 1 line
2.25.1
RFC PATCH v3 3/59 KVM: TDX: Define TDX architectural definitions
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Define architectural definitions for KVM to issue TDX SEAMCALLs.
Structures and values that are architecturally defined in the TDX module specifications the chapter of ABI Reference
--- arch/x86/kvm/vmx/tdx_arch.h | 219 ++++++++++++++++++++++++++++++++++++ 1 file changed, 219 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_arch.h
arch/x86/kvm/vmx/tdx_arch.h new file +219 -0
@@ -0,0 +1,219 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_ARCH_H +#define __KVM_X86_TDX_ARCH_H + +#include <linux/types.h> + +/* +* TDX SEAMCALL API function leaves +*/ +#define TDH_VP_ENTER 0 +#define TDH_MNG_ADDCX 1 +#define TDH_MEM_PAGE_ADD 2 +#define TDH_MEM_SEPT_ADD 3 +#define TDH_VP_ADDCX 4 +#define TDH_MEM_PAGE_AUG 6 +#define TDH_MEM_RANGE_BLOCK 7 +#define TDH_MNG_KEY_CONFIG 8 +#define TDH_MNG_CREATE 9 +#define TDH_VP_CREATE 10 +#define TDH_MNG_RD 11 +#define TDH_MEM_RD 12 +#define TDH_MNG_WR 13 +#define TDH_MEM_WR 14 +#define TDH_MEM_PAGE_DEMOTE 15 +#define TDH_MR_EXTEND 16 +#define TDH_MR_FINALIZE 17 +#define TDH_VP_FLUSH 18 +#define TDH_MNG_VPFLUSHDONE 19 +#define TDH_MNG_KEY_FREEID 20 +#define TDH_MNG_INIT 21 +#define TDH_VP_INIT 22 +#define TDH_MEM_PAGE_PROMOTE 23 +#define TDH_PHYMEM_PAGE_RDMD 24 +#define TDH_MEM_SEPT_RD 25 +#define TDH_VP_RD 26 +#define TDH_MNG_KEY_RECLAIMID 27 +#define TDH_PHYMEM_PAGE_RECLAIM 28 +#define TDH_MEM_PAGE_REMOVE 29 +#define TDH_MEM_SEPT_REMOVE 30 +#define TDH_MEM_TRACK 38 +#define TDH_MEM_RANGE_UNBLOCK 39 +#define TDH_PHYMEM_CACHE_WB 40 +#define TDH_PHYMEM_PAGE_WBINVD 41 +#define TDH_MEM_SEPT_WR 42 +#define TDH_VP_WR 43 +#define TDH_SYS_LP_SHUTDOWN 44 + +#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO 0x10000 +#define TDG_VP_VMCALL_MAP_GPA 0x10001 +#define TDG_VP_VMCALL_GET_QUOTE 0x10002 +#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003 +#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004 + +/* TDX control structure (TDR/TDCS/TDVPS) field access codes */ +#define TDX_CLASS_SHIFT 56 +#define TDX_FIELD_MASK GENMASK_ULL(31, 0) + +#define BUILD_TDX_FIELD (class, field) \ + (((u64)(class) << TDX_CLASS_SHIFT) | ((u64)(field) & TDX_FIELD_MASK)) + +/* @field is the VMCS field encoding */ +#define TDVPS_VMCS (field) BUILD_TDX_FIELD(0, (field)) + +/* +* @offset is the offset (in bytes) from the beginning of the architectural +* virtual APIC page. +*/ +#define TDVPS_APIC (offset) BUILD_TDX_FIELD(1, (offset)) + +/* @gpr is the index of a general purpose register, e.g. eax=0 */ +#define TDVPS_GPR (gpr) BUILD_TDX_FIELD(16, (gpr)) + +#define TDVPS_DR (dr) BUILD_TDX_FIELD(17, (0 + (dr))) + +enum tdx_guest_other_state { +TD_VCPU_XCR0 =32 , +TD_VCPU_IWK_ENCKEY0 =64 , +TD_VCPU_IWK_ENCKEY1 , +TD_VCPU_IWK_ENCKEY2 , +TD_VCPU_IWK_ENCKEY3 , +TD_VCPU_IWK_INTKEY0 =68 , +TD_VCPU_IWK_INTKEY1 , +TD_VCPU_IWK_FLAGS =70 , +}; + +/* @field is any of enum tdx_guest_other_state */ +#define TDVPS_STATE (field) BUILD_TDX_FIELD(17, (field)) + +/* @msr is the MSR index */ +#define TDVPS_MSR (msr) BUILD_TDX_FIELD(19, (msr)) + +/* Management class fields */ +enum tdx_guest_management { +TD_VCPU_PEND_NMI =11 , +}; + +/* @field is any of enum tdx_guest_management */ +#define TDVPS_MANAGEMENT (field) BUILD_TDX_FIELD(32, (field)) + +enum tdx_tdcs_execution_control { +TD_TDCS_EXEC_TSC_OFFSET =10 , +}; + +/* @field is any of enum tdx_tdcs_execution_control */ +#define TDCS_EXEC (field) BUILD_TDX_FIELD(17, (field)) + +/* +* Hard code those values for simplicity and efficiency. They are constant for +* the TDX 1.0. TDH.SYS.INIT enumerates those values. On kvm initialization, +* do sanity checks so that they are correct values. +* +* TODO: If they are bumped in future, increase those value. +* (or make them full runtime values) +*/ +#define TDX_NR_TDCX_PAGES 4 +#define TDX_NR_TDVPX_PAGES 5 +#define TDX_MAX_NR_CPUID_CONFIGS 6 + +#define TDX_EXTENDMR_CHUNKSIZE 256 + +struct tdx_cpuid_value { +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; +} __packed; + +struct tdx_cpuid_config { +u32 leaf; +u32 sub_leaf; +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; +} __packed; + +struct tdsysinfo_struct { +/* TDX-SEAM Module Info */ +u32 attributes; +u32 vendor_id; +u32 build_date; +u16 build_num; +u16 minor_version; +u16 major_version; +u8 reserved0[14 ]; +/* Memory Info */ +u16 max_tdmrs; +u16 max_reserved_per_tdmr; +u16 pamt_entry_size; +u8 reserved1[10 ]; +/* Control Struct Info */ +u16 tdcs_base_size; +u8 reserved2[2 ]; +u16 tdvps_base_size; +u8 tdvps_xfam_dependent_size; +u8 reserved3[9 ]; +/* TD Capabilities */ +u64 attributes_fixed0; +u64 attributes_fixed1; +u64 xfam_fixed0; +u64 xfam_fixed1; +u8 reserved4[32 ]; +u32 num_cpuid_config; +union { +struct tdx_cpuid_config cpuid_configs[0 ]; +u8 reserved5[892 ]; + }; +} __packed__aligned(1024) ; + +#define TDX_TD_ATTRIBUTE_DEBUG BIT_ULL(0) +#define TDX_TD_ATTRIBUTE_PKS BIT_ULL(30) +#define TDX_TD_ATTRIBUTE_KL BIT_ULL(31) +#define TDX_TD_ATTRIBUTE_PERFMON BIT_ULL(63) + +#define TDX_TD_XFAM_LBR BIT_ULL(15) +#define TDX_TD_XFAM_AMX (BIT_ULL(17) | BIT_ULL(18)) + +/* +* TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B. +*/ +struct td_params { +u64 attributes; +u64 xfam; +u32 max_vcpus; +u32 reserved0; + +u64 eptp_controls; +u64 exec_controls; +u16 tsc_frequency; +u8 reserved1[38 ]; + +u64 mrconfigid[6 ]; +u64 mrowner[6 ]; +u64 mrownerconfig[6 ]; +u64 reserved2[4 ]; + +union { +struct tdx_cpuid_value cpuid_values[0 ]; +u8 reserved3[768 ]; + }; +} __packed__aligned(1024) ; + +/* Guest uses MAX_PA for GPAW when set. */ +#define TDX_EXEC_CONTROL_MAX_GPAW BIT_ULL(0) + +/* +* TDX requires the frequency to be defined in units of 25MHz, which is the +* frequency of the core crystal clock on TDX-capable platforms, i.e. TDX-SEAM +* can only program frequencies that are multiples of 25MHz. The frequency +* must be between 1ghz and 10ghz (inclusive). +*/ +#define TDX_TSC_KHZ_TO_25MHZ (tsc_in_khz) ((tsc_in_khz) / (25 * 1000)) +#define TDX_TSC_25MHZ_TO_KHZ (tsc_in_25mhz) ((tsc_in_25mhz) * (25 * 1000)) +#define TDX_MIN_TSC_FREQUENCY_KHZ (100 * 1000) +#define TDX_MAX_TSC_FREQUENCY_KHZ (10 * 1000 * 1000) + +#endif /* __KVM_X86_TDX_ARCH_H */
Signature · 1 line
2.25.1
RFC PATCH v3 1/59 x86/mktme: move out MKTME related constatnts/macro to msr-index.h
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Later the constants/macro will be used by MTRR logic to reduce physical bits to calculate MTRR mask.
--- arch/x86/include/asm/msr-index.h | 16 ++++++++++++++++ arch/x86/kernel/cpu/intel.c | 14 -------------- 2 files changed, 16 insertions(+), 14 deletions(-)
arch/x86/include/asm/msr-index.h +16 -0
@@ -905,6 +905,22 @@/* Geode defined MSRs */ #define MSR_GEODE_BUSCONT_CONF0 0x00001900 +/* MKTME MSRs */ +#define MSR_IA32_TME_ACTIVATE 0x00000982 + +/* Helpers to access TME_ACTIVATE MSR */ +#define TME_ACTIVATE_LOCKED (x) (x & 0x1) +#define TME_ACTIVATE_ENABLED (x) (x & 0x2) + +#define TME_ACTIVATE_POLICY (x) ((x >> 4) & 0xf)/* Bits 7:4 */ +#define TME_ACTIVATE_KEYID_BITS (x) ((x >> 32) & 0xf)/* Bits 35:32 */ + +#define TME_ACTIVATE_POLICY_AES_XTS_128 0 + +#define TME_ACTIVATE_CRYPTO_ALGS (x) ((x >> 48) & 0xffff)/* Bits 63:48 */ +#define TME_ACTIVATE_CRYPTO_AES_XTS_128 1 + +/* Intel VT MSRs */ #define MSR_IA32_VMX_BASIC 0x00000480#define MSR_IA32_VMX_PINBASED_CTLS 0x00000481
arch/x86/kernel/cpu/intel.c +0 -14
@@ -492,20 +492,6 @@ static void srat_detect_node(struct cpuinfo_x86 *c)#endif } -#define MSR_IA32_TME_ACTIVATE 0x982 - -/* Helpers to access TME_ACTIVATE MSR */ -#define TME_ACTIVATE_LOCKED (x) (x & 0x1) -#define TME_ACTIVATE_ENABLED (x) (x & 0x2) - -#define TME_ACTIVATE_POLICY (x) ((x >> 4) & 0xf)/* Bits 7:4 */ -#define TME_ACTIVATE_POLICY_AES_XTS_128 0 - -#define TME_ACTIVATE_KEYID_BITS (x) ((x >> 32) & 0xf)/* Bits 35:32 */ - -#define TME_ACTIVATE_CRYPTO_ALGS (x) ((x >> 48) & 0xffff)/* Bits 63:48 */ -#define TME_ACTIVATE_CRYPTO_AES_XTS_128 1 -/* Values for mktme_status (SW only construct) */ #define MKTME_ENABLED 0#define MKTME_DISABLED 1
Signature · 1 line
2.25.1
RFC PATCH v3 5/59 KVM: TDX: add a helper function for kvm to call seamcall
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add a helper function for kvm to call seamcall and a helper macro to check its return value. The later patches will use them.
--- arch/x86/kvm/vmx/seamcall.h | 113 ++++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100644 arch/x86/kvm/vmx/seamcall.h
arch/x86/kvm/vmx/seamcall.h new file +113 -0
@@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_VMX_SEAMCALL_H +#define __KVM_VMX_SEAMCALL_H + +#include <asm/asm.h> + +#ifdef CONFIG_INTEL_TDX_HOST + +#ifdef __ASSEMBLER__ + +.macro seamcall + .byte 0x66, 0x0f, 0x01, 0xcf +.endm + +#else + +/* +* TDX extended return: +* Some of The "TDX module" SEAMCALLs return extended values (which are function +* leaf specific) in registers in addition to the completion status code in +%rax. +*/ +struct tdx_ex_ret { +union { +struct { +u64 rcx; +u64 rdx; +u64 r8; +u64 r9; +u64 r10; +u64 r11; + } regs; +/* TDH_MNG_INIT returns CPUID info on error. */ +struct { +u32 leaf; +u32 subleaf; + } mng_init; +/* Functions that walk SEPT */ +struct { +u64 septe; +struct { +u64 level :3 ; +u64 sept_reserved_0 :5 ; +u64 state :8 ; +u64 sept_reserved_1 :48 ; + }; + } sept_walk; +/* TDH_MNG_{RD,WR} return the field value. */ +struct { +u64 field_val; + } mng_rdwr; +/* TDH_MEM_{RD,WR} return the error info and value. */ +struct { +u64 ext_err_info_1; +u64 ext_err_info_2; +u64 mem_val; + } mem_rdwr; +/* TDH_PHYMEM_PAGE_RDMD and TDH_PHYMEM_PAGE_RECLAIM return page metadata. */ +struct { +u64 page_type; +u64 owner; +u64 page_size; + } phymem_page_md; + }; +}; + +static inline u64 seamcall (u64 op,u64 rcx,u64 rdx,u64 r8,u64 r9,u64 r10, +struct tdx_ex_ret *ex) +{ + registerunsigned long r8_in asm("r8" ); + registerunsigned long r9_in asm("r9" ); + registerunsigned long r10_in asm("r10" ); + registerunsigned long r8_out asm("r8" ); + registerunsigned long r9_out asm("r9" ); + registerunsigned long r10_out asm("r10" ); + registerunsigned long r11_out asm("r11" ); +struct tdx_ex_ret dummy; +u64 ret; + +if (!ex) +/* The following inline assembly requires non-NULL ex. */ + ex = &dummy; + +/* +* Because the TDX module is known to be already initialized, seamcall +* instruction should always succeed without exceptions. Don't check +* the instruction error with CF=1 for the availability of the TDX +* module. +*/ + r8_in = r8; + r9_in = r9; + r10_in = r10; + asmvolatile ( +".byte 0x66, 0x0f, 0x01, 0xcf\n\t" /* seamcall instruction */ + : ASM_CALL_CONSTRAINT, "=a"(ret), +"=c" (ex->regs.rcx),"=d" (ex->regs.rdx), +"=r" (r8_out),"=r" (r9_out),"=r" (r10_out),"=r" (r11_out) + :"a" (op),"c" (rcx),"d" (rdx), +"r" (r8_in),"r" (r9_in),"r" (r10_in) + :"cc" ,"memory" ); + ex->regs.r8 = r8_out; + ex->regs.r9 = r9_out; + ex->regs.r10 = r10_out; + ex->regs.r11 = r11_out; + +return ret; +} + +#endif /* !__ASSEMBLER__ */ + +#endif /* CONFIG_INTEL_TDX_HOST */ + +#endif /* __KVM_VMX_SEAMCALL_H */
Signature · 1 line
2.25.1
RFC PATCH v3 6/59 KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
In reply to [email protected]From: Sean Christopherson <[email protected]>
TDX SEAMCALL interface is defined in the TDX module specification. Define C wrapper functions for SEAMCALLs which the later patches will use.
--- arch/x86/kvm/vmx/tdx_ops.h | 210 +++++++++++++++++++++++++++++++++++++ 1 file changed, 210 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_ops.h
arch/x86/kvm/vmx/tdx_ops.h new file +210 -0
@@ -0,0 +1,210 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_OPS_H +#define __KVM_X86_TDX_OPS_H + +#include <linux/compiler.h> + +#include <asm/asm.h> +#include <asm/kvm_host.h> + +#include "seamcall.h" + +#ifdef CONFIG_INTEL_TDX_HOST + +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) +{ +return seamcall (TDH_MNG_ADDCX , addr, tdr,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_page_add (hpa_t tdr,gpa_t gpa,hpa_t hpa,hpa_t source, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_PAGE_ADD , gpa, tdr, hpa, source,0 , ex); +} + +static inline u64 tdh_mem_sept_add (hpa_t tdr,gpa_t gpa,int level,hpa_t page, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_SEPT_ADD , gpa | level, tdr, page,0 ,0 , ex); +} + +static inline u64 tdh_vp_addcx (hpa_t tdvpr,hpa_t addr) +{ +return seamcall (TDH_VP_ADDCX , addr, tdvpr,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_page_aug (hpa_t tdr,gpa_t gpa,hpa_t hpa, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_PAGE_AUG , gpa, tdr, hpa,0 ,0 , ex); +} + +static inline u64 tdh_mem_range_block (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_RANGE_BLOCK , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mng_key_config (hpa_t tdr) +{ +return seamcall (TDH_MNG_KEY_CONFIG , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_create (hpa_t tdr,int hkid) +{ +return seamcall (TDH_MNG_CREATE , tdr, hkid,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_vp_create (hpa_t tdr,hpa_t tdvpr) +{ +return seamcall (TDH_VP_CREATE , tdvpr, tdr,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_rd (hpa_t tdr,u64 field,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MNG_RD , tdr, field,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mng_wr (hpa_t tdr,u64 field,u64 val,u64 mask, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MNG_WR , tdr, field, val, mask,0 , ex); +} + +static inline u64 tdh_mem_rd (hpa_t addr,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_RD , addr,0 ,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mem_wr (hpa_t addr,u64 val,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_WR , addr, val,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mem_page_demote (hpa_t tdr,gpa_t gpa,int level,hpa_t page, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_PAGE_DEMOTE , gpa | level, tdr, page,0 ,0 , ex); +} + +static inline u64 tdh_mr_extend (hpa_t tdr,gpa_t gpa,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MR_EXTEND , gpa, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mr_finalize (hpa_t tdr) +{ +return seamcall (TDH_MR_FINALIZE , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_vp_flush (hpa_t tdvpr) +{ +return seamcall (TDH_VP_FLUSH , tdvpr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_vpflushdone (hpa_t tdr) +{ +return seamcall (TDH_MNG_VPFLUSHDONE , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_key_freeid (hpa_t tdr) +{ +return seamcall (TDH_MNG_KEY_FREEID , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_init (hpa_t tdr,hpa_t td_params,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MNG_INIT , tdr, td_params,0 ,0 ,0 , ex); +} + +static inline u64 tdh_vp_init (hpa_t tdvpr,u64 rcx) +{ +return seamcall (TDH_VP_INIT , tdvpr, rcx,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_page_promote (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_PAGE_PROMOTE , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_phymem_page_rdmd (hpa_t page,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_PHYMEM_PAGE_RDMD , page,0 ,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mem_sept_rd (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_SEPT_RD , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_vp_rd (hpa_t tdvpr,u64 field,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_VP_RD , tdvpr, field,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mng_key_reclaimid (hpa_t tdr) +{ +return seamcall (TDH_MNG_KEY_RECLAIMID , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_phymem_page_reclaim (hpa_t page,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_PHYMEM_PAGE_RECLAIM , page,0 ,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mem_page_remove (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_PAGE_REMOVE , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mem_sept_remove (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_SEPT_REMOVE , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_sys_lp_shutdown (void ) +{ +return seamcall (TDH_SYS_LP_SHUTDOWN ,0 ,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_track (hpa_t tdr) +{ +return seamcall (TDH_MEM_TRACK , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_range_unblock (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_RANGE_UNBLOCK , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_phymem_cache_wb (bool resume) +{ +return seamcall (TDH_PHYMEM_CACHE_WB , resume ?1 :0 ,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_phymem_page_wbinvd (hpa_t page) +{ +return seamcall (TDH_PHYMEM_PAGE_WBINVD , page,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_sept_wr (hpa_t tdr,gpa_t gpa,int level,u64 val, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_SEPT_WR , gpa | level, tdr, val,0 ,0 , ex); +} + +static inline u64 tdh_vp_wr (hpa_t tdvpr,u64 field,u64 val,u64 mask, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_VP_WR , tdvpr, field, val, mask,0 , ex); +} +#endif /* CONFIG_INTEL_TDX_HOST */ + +#endif /* __KVM_X86_TDX_OPS_H */
Signature · 1 line
2.25.1
RFC PATCH v3 7/59 KVM: TDX: Add helper functions to print TDX SEAMCALL error
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add helper functions to print out errors from the TDX module in an uniform way.
--- arch/x86/kvm/Makefile | 1 + arch/x86/kvm/vmx/seamcall.h | 3 ++ arch/x86/kvm/vmx/tdx_error.c | 53 ++++++++++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_error.c
arch/x86/kvm/Makefile +1 -0
@@ -29,6 +29,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx_error.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o
arch/x86/kvm/vmx/seamcall.h +3 -0
@@ -106,6 +106,9 @@ static inline u64 seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9, u64 r10,return ret; } +void pr_tdx_ex_ret_info (u64 op,u64 error_code,const struct tdx_ex_ret *ex_ret); +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_ex_ret *ex_ret); +#endif /* !__ASSEMBLER__ */ #endif /* CONFIG_INTEL_TDX_HOST */
arch/x86/kvm/vmx/tdx_error.c new file +53 -0
53 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v3 8/59 KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
In reply to [email protected]From: Sean Christopherson <[email protected]>
Later kvm_intel.ko will use it.
--- virt/kvm/kvm_main.c | 1 + 1 file changed, 1 insertion(+)
virt/kvm/kvm_main.c +1 -0
@@ -4994,6 +4994,7 @@ int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr, r =__kvm_io_bus_read (vcpu, bus, &range, val);return r <0 ? r :0 ; } +EXPORT_SYMBOL_GPL (kvm_io_bus_read);/* Caller must hold slots_lock. */ int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
Signature · 1 line
2.25.1
RFC PATCH v3 9/59 KVM: Enable hardware before doing arch VM initialization
In reply to [email protected]From: Sean Christopherson <[email protected]>
Swap the order of hardware_enable_all() and kvm_arch_init_vm() to accommodate Intel's TDX, which needs VMX to be enabled during VM init in order to make SEAMCALLs.
This also provides consistent ordering between kvm_create_vm() and kvm_destroy_vm() with respect to calling kvm_arch_destroy_vm() and hardware_disable_all().
--- virt/kvm/kvm_main.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
virt/kvm/kvm_main.c +8 -8
@@ -1065,7 +1065,7 @@ static struct kvm *kvm_create_vm(unsigned long type)struct kvm_memslots *slots =kvm_alloc_memslots ();if (!slots) - goto out_err_no_arch_destroy_vm; + goto out_err_no_disable;/* Generations must be different for each address space. */ slots->generation = i;rcu_assign_pointer (kvm->memslots[i], slots);
@@ -1075,19 +1075,19 @@ static struct kvm *kvm_create_vm(unsigned long type)rcu_assign_pointer (kvm->buses[i],kzalloc (sizeof (struct kvm_io_bus ),GFP_KERNEL_ACCOUNT ));if (!kvm->buses[i]) - goto out_err_no_arch_destroy_vm; + goto out_err_no_disable; } kvm->max_halt_poll_ns = halt_poll_ns; - r =kvm_arch_init_vm (kvm, type); -if (r) - goto out_err_no_arch_destroy_vm; - r =hardware_enable_all ();if (r) goto out_err_no_disable; + r =kvm_arch_init_vm (kvm, type); +if (r) + goto out_err_no_arch_destroy_vm; +#ifdef CONFIG_HAVE_KVM_IRQFD INIT_HLIST_HEAD (&kvm->irq_ack_notifier_list);#endif
@@ -1115,10 +1115,10 @@ static struct kvm *kvm_create_vm(unsigned long type)mmu_notifier_unregister (&kvm->mmu_notifier, current->mm);#endif out_err_no_mmu_notifier: -hardware_disable_all (); -out_err_no_disable:kvm_arch_destroy_vm (kvm); out_err_no_arch_destroy_vm: +hardware_disable_all (); +out_err_no_disable:WARN_ON_ONCE (!refcount_dec_and_test (&kvm->users_count));for (i =0 ; i <KVM_NR_BUSES ; i++)kfree (kvm_get_bus (kvm, i));
Signature · 1 line
2.25.1
RFC PATCH v3 10/59 KVM: x86: Split core of hypercall emulation to helper function
In reply to [email protected]From: Sean Christopherson <[email protected]>
By necessity, TDX will use a different register ABI for hypercalls. Break out the core functionality so that it may be reused for TDX.
--- arch/x86/include/asm/kvm_host.h | 4 +++ arch/x86/kvm/x86.c | 55 ++++++++++++++++++++------------- 2 files changed, 38 insertions(+), 21 deletions(-)
arch/x86/include/asm/kvm_host.h +4 -0
@@ -1778,6 +1778,10 @@ void kvm_request_apicv_update(struct kvm *kvm, bool activate,void __kvm_request_apicv_update (struct kvm *kvm,bool activate,unsigned long bit); +unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit);int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,
arch/x86/kvm/x86.c +34 -21
77 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v3 11/59 KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO
In reply to [email protected]From: Sean Christopherson <[email protected]>
Later kvm_intel.ko will use it.
--- arch/x86/kvm/x86.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kvm/x86.c +1 -0
@@ -12584,6 +12584,7 @@ EXPORT_SYMBOL_GPL(kvm_sev_es_string_io);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_entry);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_exit); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_mmio);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_fast_mmio);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_inj_virq);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_page_fault);
Signature · 1 line
2.25.1
RFC PATCH v3 14/59 KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a capability to effectively allow userspace to query what VM types are supported by KVM.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/include/uapi/asm/kvm.h | 4 ++++ arch/x86/kvm/svm/svm.c | 6 ++++++ arch/x86/kvm/vmx/vmx.c | 6 ++++++ arch/x86/kvm/x86.c | 9 ++++++++- include/uapi/linux/kvm.h | 2 ++ tools/arch/x86/include/uapi/asm/kvm.h | 4 ++++ tools/include/uapi/linux/kvm.h | 2 ++ 9 files changed, 35 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -18,6 +18,7 @@ KVM_X86_OP_NULL(hardware_unsetup) KVM_X86_OP_NULL(cpu_has_accelerated_tpr) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) +KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) KVM_X86_OP_NULL(vm_destroy) KVM_X86_OP(vcpu_create)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1037,6 +1037,7 @@ struct kvm_x86_msr_filter {#define APICV_INHIBIT_REASON_BLOCKIRQ 6struct kvm_arch { +unsigned long vm_type;unsigned long n_used_mmu_pages;unsigned long n_requested_mmu_pages;unsigned long n_max_mmu_pages;
@@ -1311,6 +1312,7 @@ struct kvm_x86_ops {bool (*has_emulated_msr)(struct kvm *kvm,u32 index);void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); +bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);
arch/x86/include/uapi/asm/kvm.h +4 -0
@@ -508,4 +508,8 @@ struct kvm_pmu_event_filter {#define KVM_VCPU_TSC_CTRL 0/* control group for the timestamp counter (TSC) */ #define KVM_VCPU_TSC_OFFSET 0/* attribute for the TSC offset */ +#define KVM_X86_LEGACY_VM 0 +#define KVM_X86_SEV_ES_VM 1 +#define KVM_X86_TDX_VM 2 +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/svm/svm.c +6 -0
@@ -4562,6 +4562,11 @@ static void svm_vm_destroy(struct kvm *kvm)sev_vm_destroy (kvm); } +static bool svm_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_LEGACY_VM ; +} +static int svm_vm_init (struct kvm *kvm) {if (!pause_filter_count || !pause_filter_thresh)
@@ -4589,6 +4594,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_free = svm_free_vcpu, .vcpu_reset = svm_vcpu_reset, + .is_vm_type_supported = svm_is_vm_type_supported, .vm_size = sizeof(struct kvm_svm), .vm_init = svm_vm_init, .vm_destroy = svm_vm_destroy,
arch/x86/kvm/vmx/vmx.c +6 -0
@@ -6851,6 +6851,11 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu)return err; } +static bool vmx_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_LEGACY_VM ; +} +#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
@@ -7505,6 +7510,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .cpu_has_accelerated_tpr = report_flexpriority, .has_emulated_msr = vmx_has_emulated_msr, + .is_vm_type_supported = vmx_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vmx_vm_init,
arch/x86/kvm/x86.c +8 -1
@@ -4225,6 +4225,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)else r =0 ;break ; +case KVM_CAP_VM_TYPES : + r =BIT (KVM_X86_LEGACY_VM ); +if (static_call (kvm_x86_is_vm_type_supported)(KVM_X86_TDX_VM )) + r |=BIT (KVM_X86_TDX_VM ); +break ;default :break ; }
@@ -11320,9 +11325,11 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)int ret;unsigned long flags; -if (type) +if (!static_call (kvm_x86_is_vm_type_supported)(type))return -EINVAL ; + kvm->arch.vm_type = type; + ret =kvm_page_track_init (kvm);if (ret)return ret;
include/uapi/linux/kvm.h +2 -0
@@ -1132,6 +1132,8 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_ARM_MTE 205#define KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM 206 +#define KVM_CAP_VM_TYPES 1000 +#ifdef KVM_CAP_IRQ_ROUTING struct kvm_irq_routing_irqchip {
tools/arch/x86/include/uapi/asm/kvm.h +4 -0
@@ -504,4 +504,8 @@ struct kvm_pmu_event_filter {#define KVM_PMU_EVENT_ALLOW 0#define KVM_PMU_EVENT_DENY 1 +#define KVM_X86_LEGACY_VM 0 +#define KVM_X86_SEV_ES_VM 1 +#define KVM_X86_TDX_VM 2 +#endif /* _ASM_X86_KVM_H */
tools/include/uapi/linux/kvm.h +2 -0
@@ -1113,6 +1113,8 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204#define KVM_CAP_ARM_MTE 205 +#define KVM_CAP_VM_TYPES 1000 +#ifdef KVM_CAP_IRQ_ROUTING struct kvm_irq_routing_irqchip {
Signature · 1 line
2.25.1
RFC PATCH v3 12/59 KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
In reply to [email protected]From: Sean Christopherson <[email protected]>
Zap only leaf SPTEs when deleting/moving a memslot by default, and add a module param to allow reverting to the old behavior of zapping all SPTEs at all levels and memslots when any memslot is updated.
--- arch/x86/kvm/mmu/mmu.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +20 -1
@@ -91,6 +91,9 @@ __MODULE_PARM_TYPE(nx_huge_pages_recovery_period_ms, "uint");static bool __read_mostly force_flush_and_sync_on_reuse;module_param_named (flush_on_reuse, force_flush_and_sync_on_reuse, bool,0644 ); +static bool __read_mostly memslot_update_zap_all; +module_param (memslot_update_zap_all, bool,0444 ); +/* * When setting this variable to true it enables Two-Dimensional-Paging * where the hardware walks 2 page tables:
@@ -5681,11 +5684,27 @@ static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm)return unlikely (!list_empty_careful (&kvm->arch.zapped_obsolete_pages)); } +static void kvm_mmu_zap_memslot (struct kvm *kvm,struct kvm_memory_slot *slot) +{ +/* +* Zapping non-leaf SPTEs, a.k.a. not-last SPTEs, isn't required, worst +* case scenario we'll have unused shadow pages lying around until they +* are recycled due to age or when the VM is destroyed. +*/ +write_lock (&kvm->mmu_lock); +slot_handle_level (kvm, slot, kvm_zap_rmapp,PG_LEVEL_4K , +KVM_MAX_HUGEPAGE_LEVEL , true); +write_unlock (&kvm->mmu_lock); +} +static void kvm_mmu_invalidate_zap_pages_in_memslot (struct kvm *kvm,struct kvm_memory_slot *slot,struct kvm_page_track_notifier_node *node) { -kvm_mmu_zap_all_fast (kvm); +if (memslot_update_zap_all) +kvm_mmu_zap_all_fast (kvm); +else +kvm_mmu_zap_memslot (kvm, slot); }void kvm_mmu_init_vm (struct kvm *kvm)
Signature · 1 line
2.25.1
RFC PATCH v3 18/59 KVM: x86: Add flag to mark TSC as immutable (for TDX)
In reply to [email protected]From: Sean Christopherson <[email protected]>
The TSC for TDX1 guests is fixed at TD creation time. Add tsc_immutable to reflect that the TSC of the guest cannot be changed in any way, and use it to short circuit all paths that lead to one of the myriad TSC adjustment flows.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 35 +++++++++++++++++++++++++-------- 2 files changed, 28 insertions(+), 8 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1122,6 +1122,7 @@ struct kvm_arch {int audit_point;#endif +bool tsc_immutable;bool backwards_tsc_observed;bool boot_vcpu_runs_old_kvmclock;u32 bsp_vcpu_id;
arch/x86/kvm/x86.c +27 -8
@@ -2247,7 +2247,9 @@ static int set_tsc_khz(struct kvm_vcpu *vcpu, u32 user_tsc_khz, bool scale)u64 ratio;/* Guest TSC same frequency as host TSC? */ -if (!scale) { +if (!scale || vcpu->kvm->arch.tsc_immutable) { +if (scale) +pr_warn_ratelimited ("Guest TSC immutable, scaling not supported\n" );kvm_vcpu_write_tsc_multiplier (vcpu, kvm_default_tsc_scaling_ratio);return 0 ; }
@@ -2534,6 +2536,9 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)bool matched = false;bool synchronizing = false; +if (WARN_ON_ONCE (vcpu->kvm->arch.tsc_immutable)) +return ; +raw_spin_lock_irqsave (&kvm->arch.tsc_write_lock, flags); offset =kvm_compute_l1_tsc_offset (vcpu, data); ns =get_kvmclock_base_ns ();
@@ -2960,6 +2965,10 @@ static int kvm_guest_time_update(struct kvm_vcpu *v)u8 pvclock_flags;bool use_master_clock; +/* Unable to update guest time if the TSC is immutable. */ +if (ka->tsc_immutable) +return 0 ; + kernel_ns =0 ; host_tsc =0 ;
@@ -4372,7 +4381,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)if (tsc_delta <0 )mark_tsc_unstable ("KVM discovered backwards TSC" ); -if (kvm_check_tsc_unstable ()) { +if (kvm_check_tsc_unstable () && + !vcpu->kvm->arch.tsc_immutable) {u64 offset =kvm_compute_l1_tsc_offset (vcpu, vcpu->arch.last_guest_tsc);kvm_vcpu_write_tsc_offset (vcpu, offset);
@@ -4386,7 +4396,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)* On a host with synchronized TSC, there is no need to update * kvmclock on vcpu->cpu migration */ -if (!vcpu->kvm->arch.use_master_clock || vcpu->cpu ==-1 ) +if ((!vcpu->kvm->arch.use_master_clock || vcpu->cpu ==-1 ) && + !vcpu->kvm->arch.tsc_immutable)kvm_make_request (KVM_REQ_GLOBAL_CLOCK_UPDATE , vcpu);if (vcpu->cpu != cpu)kvm_make_request (KVM_REQ_MIGRATE_TIMER , vcpu);
@@ -5352,10 +5363,11 @@ long kvm_arch_vcpu_ioctl(struct file *filp,break ; }case KVM_SET_TSC_KHZ : { -u32 user_tsc_khz; +u32 user_tsc_khz = (u32 )arg; r = -EINVAL ; - user_tsc_khz = (u32 )arg; +if (vcpu->kvm->arch.tsc_immutable) + goto out;if (kvm_has_tsc_control && user_tsc_khz >= kvm_max_guest_tsc_khz)
@@ -10994,9 +11006,12 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)if (mutex_lock_killable (&vcpu->mutex))return ; -vcpu_load (vcpu); -kvm_synchronize_tsc (vcpu,0 ); -vcpu_put (vcpu); + +if (!kvm->arch.tsc_immutable) { +vcpu_load (vcpu); +kvm_synchronize_tsc (vcpu,0 ); +vcpu_put (vcpu); + }/* poll control enabled by default */ vcpu->arch.msr_kvm_poll_control =1 ;
@@ -11253,6 +11268,10 @@ int kvm_arch_hardware_enable(void)if (backwards_tsc) {u64 delta_cyc = max_tsc - local_tsc;list_for_each_entry (kvm, &vm_list, vm_list) { +if (vcpu->kvm->arch.tsc_immutable) { +pr_warn_ratelimited ("Backwards TSC observed and guest with immutable TSC active\n" ); +continue ; + } kvm->arch.backwards_tsc_observed = true;kvm_for_each_vcpu (i, vcpu, kvm) { vcpu->arch.tsc_offset_adjustment += delta_cyc;
Signature · 1 line
2.25.1
RFC PATCH v3 4/59 KVM: TDX: Add TDX "architectural" error codes
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add error codes for the TDX SEAMCALLs both for TDX VMM side and TDX guest side.
TDX SEAMCALL uses bits 31:0 to return more information, so these error codes will only exactly match RAX[63:32]. Error codes for TDG.VP.VMCALL is defined by TDX Guest-Host-Communication interface spec.
--- arch/x86/kvm/vmx/tdx_errno.h | 111 +++++++++++++++++++++++++++++++++++ 1 file changed, 111 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_errno.h
arch/x86/kvm/vmx/tdx_errno.h new file +111 -0
@@ -0,0 +1,111 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural status code for SEAMCALL */ + +#ifndef __KVM_X86_TDX_ERRNO_H +#define __KVM_X86_TDX_ERRNO_H + +#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL + +/* +* TDX SEAMCALL Status Codes (returned in RAX) +*/ +#define TDX_SUCCESS 0x0000000000000000ULL +#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL +#define TDX_NON_RECOVERABLE_TD 0x4000000200000000ULL +#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL +#define TDX_INTERRUPTED_RESTARTABLE 0x8000000400000000ULL +#define TDX_NON_RECOVERABLE_TD_FATAL 0x4000000500000000ULL +#define TDX_INVALID_RESUMPTION 0xC000000600000000ULL +#define TDX_NON_RECOVERABLE_TD_NO_APIC 0xC000000700000000ULL +#define TDX_OPERAND_INVALID 0xC000010000000000ULL +#define TDX_OPERAND_ADDR_RANGE_ERROR 0xC000010100000000ULL +#define TDX_OPERAND_BUSY 0x8000020000000000ULL +#define TDX_PREVIOUS_TLB_EPOCH_BUSY 0x8000020100000000ULL +#define TDX_SYS_BUSY 0x8000020200000000ULL +#define TDX_PAGE_METADATA_INCORRECT 0xC000030000000000ULL +#define TDX_PAGE_ALREADY_FREE 0x0000030100000000ULL +#define TDX_PAGE_NOT_OWNED_BY_TD 0xC000030200000000ULL +#define TDX_PAGE_NOT_FREE 0xC000030300000000ULL +#define TDX_TD_ASSOCIATED_PAGES_EXIST 0xC000040000000000ULL +#define TDX_SYSINIT_NOT_PENDING 0xC000050000000000ULL +#define TDX_SYSINIT_NOT_DONE 0xC000050100000000ULL +#define TDX_SYSINITLP_NOT_DONE 0xC000050200000000ULL +#define TDX_SYSINITLP_DONE 0xC000050300000000ULL +#define TDX_SYS_NOT_READY 0xC000050500000000ULL +#define TDX_SYS_SHUTDOWN 0xC000050600000000ULL +#define TDX_SYSCONFIG_NOT_DONE 0xC000050700000000ULL +#define TDX_TD_NOT_INITIALIZED 0xC000060000000000ULL +#define TDX_TD_INITIALIZED 0xC000060100000000ULL +#define TDX_TD_NOT_FINALIZED 0xC000060200000000ULL +#define TDX_TD_FINALIZED 0xC000060300000000ULL +#define TDX_TD_FATAL 0xC000060400000000ULL +#define TDX_TD_NON_DEBUG 0xC000060500000000ULL +#define TDX_LIFECYCLE_STATE_INCORRECT 0xC000060700000000ULL +#define TDX_TDCX_NUM_INCORRECT 0xC000061000000000ULL +#define TDX_VCPU_STATE_INCORRECT 0xC000070000000000ULL +#define TDX_VCPU_ASSOCIATED 0x8000070100000000ULL +#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL +#define TDX_TDVPX_NUM_INCORRECT 0xC000070300000000ULL +#define TDX_NO_VALID_VE_INFO 0xC000070400000000ULL +#define TDX_MAX_VCPUS_EXCEEDED 0xC000070500000000ULL +#define TDX_TSC_ROLLBACK 0xC000070600000000ULL +#define TDX_FIELD_NOT_WRITABLE 0xC000072000000000ULL +#define TDX_FIELD_NOT_READABLE 0xC000072100000000ULL +#define TDX_TD_VMCS_FIELD_NOT_INITIALIZED 0xC000073000000000ULL +#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL +#define TDX_TD_KEYS_NOT_CONFIGURED 0x8000081000000000ULL +#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL +#define TDX_KEY_CONFIGURED 0x0000081500000000ULL +#define TDX_WBCACHE_NOT_COMPLETE 0x8000081700000000ULL +#define TDX_HKID_NOT_FREE 0xC000082000000000ULL +#define TDX_NO_HKID_READY_TO_WBCACHE 0x0000082100000000ULL +#define TDX_WBCACHE_RESUME_ERROR 0xC000082300000000ULL +#define TDX_FLUSHVP_NOT_DONE 0x8000082400000000ULL +#define TDX_NUM_ACTIVATED_HKIDS_NOT_SUPPORTED 0xC000082500000000ULL +#define TDX_INCORRECT_CPUID_VALUE 0xC000090000000000ULL +#define TDX_BOOT_NT4_SET 0xC000090100000000ULL +#define TDX_INCONSISTENT_CPUID_FIELD 0xC000090200000000ULL +#define TDX_CPUID_LEAF_1F_FORMAT_UNRECOGNIZED 0xC000090400000000ULL +#define TDX_INVALID_WBINVD_SCOPE 0xC000090500000000ULL +#define TDX_INVALID_PKG_ID 0xC000090600000000ULL +#define TDX_CPUID_LEAF_NOT_SUPPORTED 0xC000090800000000ULL +#define TDX_SMRR_NOT_LOCKED 0xC000091000000000ULL +#define TDX_INVALID_SMRR_CONFIGURATION 0xC000091100000000ULL +#define TDX_SMRR_OVERLAPS_CMR 0xC000091200000000ULL +#define TDX_SMRR_LOCK_NOT_SUPPORTED 0xC000091300000000ULL +#define TDX_SMRR_NOT_SUPPORTED 0xC000091400000000ULL +#define TDX_INCONSISTENT_MSR 0xC000092000000000ULL +#define TDX_INCORRECT_MSR_VALUE 0xC000092100000000ULL +#define TDX_SEAMREPORT_NOT_AVAILABLE 0xC000093000000000ULL +#define TDX_PERF_COUNTERS_ARE_PEBS_ENABLED 0x8000094000000000ULL +#define TDX_INVALID_TDMR 0xC0000A0000000000ULL +#define TDX_NON_ORDERED_TDMR 0xC0000A0100000000ULL +#define TDX_TDMR_OUTSIDE_CMRS 0xC0000A0200000000ULL +#define TDX_TDMR_ALREADY_INITIALIZED 0x00000A0300000000ULL +#define TDX_INVALID_PAMT 0xC0000A1000000000ULL +#define TDX_PAMT_OUTSIDE_CMRS 0xC0000A1100000000ULL +#define TDX_PAMT_OVERLAP 0xC0000A1200000000ULL +#define TDX_INVALID_RESERVED_IN_TDMR 0xC0000A2000000000ULL +#define TDX_NON_ORDERED_RESERVED_IN_TDMR 0xC0000A2100000000ULL +#define TDX_CMR_LIST_INVALID 0xC0000A2200000000ULL +#define TDX_EPT_WALK_FAILED 0xC0000B0000000000ULL +#define TDX_EPT_ENTRY_FREE 0xC0000B0100000000ULL +#define TDX_EPT_ENTRY_NOT_FREE 0xC0000B0200000000ULL +#define TDX_EPT_ENTRY_NOT_PRESENT 0xC0000B0300000000ULL +#define TDX_EPT_ENTRY_NOT_LEAF 0xC0000B0400000000ULL +#define TDX_EPT_ENTRY_LEAF 0xC0000B0500000000ULL +#define TDX_GPA_RANGE_NOT_BLOCKED 0xC0000B0600000000ULL +#define TDX_GPA_RANGE_ALREADY_BLOCKED 0x00000B0700000000ULL +#define TDX_TLB_TRACKING_NOT_DONE 0xC0000B0800000000ULL +#define TDX_EPT_INVALID_PROMOTE_CONDITIONS 0xC0000B0900000000ULL +#define TDX_PAGE_ALREADY_ACCEPTED 0x00000B0A00000000ULL +#define TDX_PAGE_SIZE_MISMATCH 0xC0000B0B00000000ULL + +/* +* TDG.VP.VMCALL Status Codes (returned in R10) +*/ +#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL +#define TDG_VP_VMCALL_INVALID_OPERAND 0x8000000000000000ULL +#define TDG_VP_VMCALL_TDREPORT_FAILED 0x8000000000000001ULL + +#endif /* __KVM_X86_TDX_ERRNO_H */
Signature · 1 line
2.25.1
RFC PATCH v3 21/59 KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level routes
In reply to [email protected]From: Kai Huang <[email protected]>
Add a flag to let TDX disallow the in-kernel I/O APIC, level triggered routes for a userspace I/O APIC, and anything else that relies on being able to intercept EOIs. TDX-SEAM does not allow intercepting EOI.
Note, technically KVM could partially emulate the I/O APIC by allowing only edge triggered interrupts, but that adds a lot of complexity for basically zero benefit. Ideally KVM wouldn't even allow I/O APIC route reservation, but disabling that is a train wreck for Qemu.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/ioapic.c | 4 ++++ arch/x86/kvm/irq_comm.c | 9 +++++++-- arch/x86/kvm/lapic.c | 3 ++- arch/x86/kvm/x86.c | 6 ++++++ 5 files changed, 20 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1132,6 +1132,7 @@ struct kvm_arch {enum kvm_irqchip_mode irqchip_mode;u8 nr_reserved_ioapic_pins; +bool eoi_intercept_unsupported;bool disabled_lapic_found;
arch/x86/kvm/ioapic.c +4 -0
@@ -311,6 +311,10 @@ void kvm_arch_post_irq_ack_notifier_list_update(struct kvm *kvm) {if (!ioapic_in_kernel (kvm))return ; + +if (WARN_ON_ONCE (kvm->arch.eoi_intercept_unsupported)) +return ; +kvm_make_scan_ioapic_request (kvm); }
arch/x86/kvm/irq_comm.c +7 -2
@@ -123,7 +123,12 @@ EXPORT_SYMBOL_GPL(kvm_set_msi_irq);static inline bool kvm_msi_route_invalid (struct kvm *kvm,struct kvm_kernel_irq_routing_entry *e) { -return kvm->arch.x2apic_format && (e->msi.address_hi &0xff ); +struct msi_msg msg = { .address_lo = e->msi.address_lo, + .address_hi = e->msi.address_hi, + .data = e->msi.data }; +return (kvm->arch.eoi_intercept_unsupported && + msg.arch_data.is_level) || + (kvm->arch.x2apic_format && (msg.address_hi &0xff )); } int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e,
@@ -385,7 +390,7 @@ int kvm_setup_empty_irq_routing(struct kvm *kvm)void kvm_arch_post_irq_routing_update (struct kvm *kvm) { -if (!irqchip_split (kvm)) +if (!irqchip_split (kvm) || kvm->arch.eoi_intercept_unsupported)return ;kvm_make_scan_ioapic_request (kvm); }
arch/x86/kvm/lapic.c +2 -1
@@ -281,7 +281,8 @@ void kvm_recalculate_apic_map(struct kvm *kvm)if (old)call_rcu (&old->rcu, kvm_apic_map_free); -kvm_make_scan_ioapic_request (kvm); +if (!kvm->arch.eoi_intercept_unsupported) +kvm_make_scan_ioapic_request (kvm); }static inline void apic_set_spiv (struct kvm_lapic *apic,u32 val)
arch/x86/kvm/x86.c +6 -0
@@ -6110,6 +6110,9 @@ long kvm_arch_vm_ioctl(struct file *filp, goto create_irqchip_unlock; r = -EINVAL ; +if (kvm->arch.eoi_intercept_unsupported) + goto create_irqchip_unlock; +if (kvm->created_vcpus) goto create_irqchip_unlock;
@@ -6140,6 +6143,9 @@ long kvm_arch_vm_ioctl(struct file *filp, u.pit_config.flags =KVM_PIT_SPEAKER_DUMMY ; goto create_pit;case KVM_CREATE_PIT2 : + r = -EINVAL ; +if (kvm->arch.eoi_intercept_unsupported) + goto out; r = -EFAULT ;if (copy_from_user (&u.pit_config, argp,sizeof (struct kvm_pit_config )))
Signature · 1 line
2.25.1
RFC PATCH v3 27/59 KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched behavior
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a flag, KVM_DEBUGREG_AUTO_SWITCHED_GUEST, to skip saving/restoring DRs irrespective of any other flags. TDX-SEAM unconditionally saves and restores guest DRs and reset to architectural INIT state on TD exit. So, KVM needs to save host DRs before TD enter without restoring guest DRs and restore host DRs after TD exit.
Opportunistically convert the KVM_DEBUGREG_* definitions to use BIT().
--- arch/x86/include/asm/kvm_host.h | 10 ++++++++-- arch/x86/kvm/x86.c | 3 ++- 2 files changed, 10 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +8 -2
@@ -526,8 +526,14 @@ struct kvm_pmu {struct kvm_pmu_ops ;enum { -KVM_DEBUGREG_BP_ENABLED =1 , -KVM_DEBUGREG_WONT_EXIT =2 , +KVM_DEBUGREG_BP_ENABLED =BIT (0 ), +KVM_DEBUGREG_WONT_EXIT =BIT (1 ), +KVM_DEBUGREG_RELOAD =BIT (2 ), +/* +* Guest debug registers are saved/restored by hardware on exit from +* or enter guest. KVM needn't switch them. +*/ +KVM_DEBUGREG_AUTO_SWITCH =BIT (3 ), };struct kvm_mtrr_range {
arch/x86/kvm/x86.c +2 -1
@@ -9910,7 +9910,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (test_thread_flag (TIF_NEED_FPU_LOAD ))switch_fpu_return (); -if (unlikely (vcpu->arch.switch_db_regs)) { +if (unlikely (vcpu->arch.switch_db_regs & ~KVM_DEBUGREG_AUTO_SWITCH )) {set_debugreg (0 ,7 );set_debugreg (vcpu->arch.eff_db[0 ],0 );set_debugreg (vcpu->arch.eff_db[1 ],1 );
@@ -9950,6 +9950,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)*/ if (unlikely (vcpu->arch.switch_db_regs &KVM_DEBUGREG_WONT_EXIT )) {WARN_ON (vcpu->guest_debug &KVM_GUESTDBG_USE_HW_BP ); +WARN_ON (vcpu->arch.switch_db_regs &KVM_DEBUGREG_AUTO_SWITCH );static_call (kvm_x86_sync_dirty_debug_regs)(vcpu);kvm_update_dr0123 (vcpu);kvm_update_dr7 (vcpu);
Signature · 1 line
2.25.1
RFC PATCH v3 26/59 KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a second kvm_x86_ops hook in kvm_arch_vm_destroy() to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.
Note, this knowingly creates a discrepancy in nomenclature for SVM as svm_vm_teardown() invokes avic_vm_destroy() and sev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm/svm.c | 5 +++-- arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 3 ++- 5 files changed, 14 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -20,6 +20,7 @@ KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) +KVM_X86_OP_NULL(vm_teardown) KVM_X86_OP_NULL(vm_destroy) KVM_X86_OP(vcpu_create) KVM_X86_OP(vcpu_free)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1321,6 +1321,7 @@ struct kvm_x86_ops {bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm); +void (*vm_teardown)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);/* Create, but do not attach this VCPU */
arch/x86/kvm/svm/svm.c +3 -2
@@ -4556,7 +4556,7 @@ static void svm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)sev_vcpu_deliver_sipi_vector (vcpu, vector); } -static void svm_vm_destroy (struct kvm *kvm) +static void svm_vm_teardown (struct kvm *kvm) {avic_vm_destroy (kvm);sev_vm_destroy (kvm);
@@ -4597,7 +4597,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .is_vm_type_supported = svm_is_vm_type_supported, .vm_size = sizeof(struct kvm_svm), .vm_init = svm_vm_init, - .vm_destroy = svm_vm_destroy, + .vm_teardown = svm_vm_teardown, + .vm_destroy = NULL, .prepare_guest_switch = svm_prepare_guest_switch, .vcpu_load = svm_vcpu_load,
arch/x86/kvm/vmx/vmx.c +7 -0
@@ -6890,6 +6890,11 @@ static int vmx_vm_init(struct kvm *kvm)return 0 ; } +static void vmx_vm_destroy (struct kvm *kvm) +{ + +} +static int __init vmx_check_processor_compat (void ) {struct vmcs_config vmcs_conf;
@@ -7513,6 +7518,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .is_vm_type_supported = vmx_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vmx_vm_init, + .vm_teardown = NULL, + .vm_destroy = vmx_vm_destroy, .vcpu_create = vmx_create_vcpu, .vcpu_free = vmx_free_vcpu,
arch/x86/kvm/x86.c +2 -1
@@ -11613,7 +11613,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)__x86_set_memory_region (kvm,TSS_PRIVATE_MEMSLOT ,0 ,0 );mutex_unlock (&kvm->slots_lock); } -static_call_cond (kvm_x86_vm_destroy)(kvm); +static_call_cond (kvm_x86_vm_teardown)(kvm);kvm_free_msr_filter (srcu_dereference_check (kvm->arch.msr_filter, &kvm->srcu,1 ));kvm_pic_destroy (kvm);kvm_ioapic_destroy (kvm);
@@ -11624,6 +11624,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)kvm_page_track_cleanup (kvm);kvm_xen_destroy_vm (kvm);kvm_hv_destroy_vm (kvm); +static_call_cond (kvm_x86_vm_destroy)(kvm); }static void memslot_rmap_free (struct kvm_memory_slot *slot)
Signature · 1 line
2.25.1
RFC PATCH v3 29/59 KVM: x86: Add option to force LAPIC expiration wait
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add an option to skip the IRR check in kvm_wait_lapic_expire(). This will be used by TDX to wait if there is an outstanding notification for a TD, i.e. a virtual interrupt is being triggered via posted interrupt processing. KVM TDX doesn't emulate PI processing, i.e. there will never be a bit set in IRR/ISR, so the default behavior for APICv of querying the IRR doesn't work as intended.
--- arch/x86/kvm/lapic.c | 4 ++-- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-)
arch/x86/kvm/lapic.c +2 -2
@@ -1625,12 +1625,12 @@ static void __kvm_wait_lapic_expire(struct kvm_vcpu *vcpu)__wait_lapic_expire (vcpu, tsc_deadline - guest_tsc); } -void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu) +void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu,bool force_wait) {if (lapic_in_kernel (vcpu) && vcpu->arch.apic->lapic_timer.expired_tscdeadline && vcpu->arch.apic->lapic_timer.timer_advance_ns && -lapic_timer_int_injected (vcpu)) + (force_wait ||lapic_timer_int_injected (vcpu)))__kvm_wait_lapic_expire (vcpu); }EXPORT_SYMBOL_GPL (kvm_wait_lapic_expire);
arch/x86/kvm/lapic.h +1 -1
@@ -233,7 +233,7 @@ static inline int kvm_lapic_latched_init(struct kvm_vcpu *vcpu)bool kvm_apic_pending_eoi (struct kvm_vcpu *vcpu,int vector); -void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu); +void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu,bool force_wait);void kvm_bitmap_or_dest_vcpus (struct kvm *kvm,struct kvm_lapic_irq *irq,unsigned long *vcpu_bitmap);
arch/x86/kvm/svm/svm.c +1 -1
@@ -3885,7 +3885,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)clgi ();kvm_load_guest_xsave_state (vcpu); -kvm_wait_lapic_expire (vcpu); +kvm_wait_lapic_expire (vcpu, false);/* * If this vCPU has touched SPEC_CTRL, restore the guest's value if
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -6647,7 +6647,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)if (enable_preemption_timer)vmx_update_hv_timer (vcpu); -kvm_wait_lapic_expire (vcpu); +kvm_wait_lapic_expire (vcpu, false);/* * If this vCPU has touched SPEC_CTRL, restore the guest's value if
Signature · 1 line
2.25.1
RFC PATCH v3 37/59 KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
In reply to [email protected]From: Sean Christopherson <[email protected]>
Introduce a helper to directly (pun intended) fault-in a TDP page without having to go through the full page fault path. This allows TDX to get the resulting pfn and also allows the RET_PF_* enums to stay in mmu.c where they belong.
--- arch/x86/kvm/mmu.h | 3 +++ arch/x86/kvm/mmu/mmu.c | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+)
arch/x86/kvm/mmu.h +3 -0
@@ -202,6 +202,9 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,return vcpu->arch.mmu->page_fault (vcpu, &fault); } +kvm_pfn_t kvm_mmu_map_tdp_page (struct kvm_vcpu *vcpu,gpa_t gpa, +u32 error_code,int max_level); +/* * Currently, we have two sorts of write-protection, a) the first one * write-protects guest page to sync the guest modification, b) another one is
arch/x86/kvm/mmu/mmu.c +38 -0
@@ -4432,6 +4432,44 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)return direct_page_fault (vcpu, fault); } +kvm_pfn_t kvm_mmu_map_tdp_page (struct kvm_vcpu *vcpu,gpa_t gpa, +u32 error_code,int max_level) +{ +int r; +struct kvm_page_fault fault = (struct kvm_page_fault ) { + .addr = gpa, + .error_code = error_code, + .exec = error_code &PFERR_FETCH_MASK , + .write = error_code &PFERR_WRITE_MASK , + .present = error_code &PFERR_PRESENT_MASK , + .rsvd = error_code &PFERR_RSVD_MASK , + .user = error_code &PFERR_USER_MASK , + .prefetch = false, + .is_tdp = true, + .nx_huge_page_workaround_enabled =is_nx_huge_page_enabled (), + }; + +if (mmu_topup_memory_caches (vcpu, false)) +return KVM_PFN_ERR_FAULT ; + +/* +* Loop on the page fault path to handle the case where an mmu_notifier +* invalidation triggers RET_PF_RETRY. In the normal page fault path, +* KVM needs to resume the guest in case the invalidation changed any +* of the page fault properties, i.e. the gpa or error code. For this +* path, the gpa and error code are fixed by the caller, and the caller +* expects failure if and only if the page fault can't be fixed. +*/ +do { + fault.max_level = max_level; + fault.req_level =PG_LEVEL_4K ; + fault.goal_level =PG_LEVEL_4K ; + r =direct_page_fault (vcpu, &fault); + }while (r ==RET_PF_RETRY && !is_error_noslot_pfn (fault.pfn)); +return fault.pfn; +} +EXPORT_SYMBOL_GPL (kvm_mmu_map_tdp_page); +static void nonpaging_init_context (struct kvm_mmu *context) { context->page_fault = nonpaging_page_fault;
Signature · 1 line
2.25.1
RFC PATCH v3 36/59 KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add kvm_x86_ops hooks to set/clear private SPTEs, i.e. SEPT entries, and to link/free private shadow pages, i.e. non-leaf SEPT pages.
Because SEAMCALLs are bloody expensive, and because KVM's MMU is already complex enough, TDX's SEPT will mirror KVM's shadow pages instead of replacing them outright. This costs extra memory, but is simpler and far more performant.
Add a separate list for tracking active private shadow pages. Zapping and freeing SEPT entries is subject to very different rules than normal pages/memory, and need to be preserved (along with their shadow page counterparts) when KVM gets trigger happy, e.g. zaps everything during a memslot update.
Zap any aliases of a GPA when mapping in a guest that supports guest private GPAs. This is necessary to avoid integrity failures with TDX due to pointing shared and private GPAs at the same HPA.
Do not prefetch private pages (this should probably be a property of the VM).
TDX needs one more bits in spte for SPTE_PRIVATE_ZAPPED. Steal 1 bit from MMIO generation as 62 bit and re-purpose it for non-present SPTE, SPTE_PRIVATE_ZAPPED.
--- arch/x86/include/asm/kvm-x86-ops.h | 7 + arch/x86/include/asm/kvm_host.h | 32 +- arch/x86/kvm/mmu.h | 16 +- arch/x86/kvm/mmu/mmu.c | 478 +++++++++++++++++++++++------ arch/x86/kvm/mmu/mmu_internal.h | 13 +- arch/x86/kvm/mmu/paging_tmpl.h | 11 +- arch/x86/kvm/mmu/spte.h | 16 +- arch/x86/kvm/x86.c | 10 +- virt/kvm/kvm_main.c | 1 + 9 files changed, 463 insertions(+), 121 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +7 -0
@@ -20,6 +20,7 @@ KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) +KVM_X86_OP_NULL(mmu_prezap) KVM_X86_OP_NULL(vm_teardown) KVM_X86_OP_NULL(vm_destroy) KVM_X86_OP(vcpu_create)
@@ -89,6 +90,12 @@ KVM_X86_OP(set_tss_addr) KVM_X86_OP(set_identity_map_addr) KVM_X86_OP(get_mt_mask) KVM_X86_OP(load_mmu_pgd) +KVM_X86_OP(set_private_spte) +KVM_X86_OP(drop_private_spte) +KVM_X86_OP(zap_private_spte) +KVM_X86_OP(unzap_private_spte) +KVM_X86_OP(link_private_sp) +KVM_X86_OP(free_private_sp) KVM_X86_OP_NULL(has_wbinvd_exit) KVM_X86_OP(get_l2_tsc_offset) KVM_X86_OP(get_l2_tsc_multiplier)
arch/x86/include/asm/kvm_host.h +29 -3
@@ -320,7 +320,8 @@ union kvm_mmu_page_role { unsigned smap_andnot_wp:1; unsigned ad_disabled:1; unsigned guest_mode:1; - unsigned :6; + unsigned private:1; + unsigned :5;/* * This is left at the top of the word so that
@@ -427,6 +428,7 @@ struct kvm_mmu { struct kvm_mmu_page *sp);void (*invlpg)(struct kvm_vcpu *vcpu,gva_t gva,hpa_t root_hpa);hpa_t root_hpa; +hpa_t private_root_hpa;gpa_t root_pgd;union kvm_mmu_role mmu_role;u8 root_level;
@@ -463,6 +465,8 @@ struct kvm_mmu {struct rsvd_bits_validate guest_rsvd_check; +bool no_prefetch; +u64 pdptrs[4 ];/* pae */ };
@@ -685,6 +689,7 @@ struct kvm_vcpu_arch {struct kvm_mmu_memory_cache mmu_shadow_page_cache;struct kvm_mmu_memory_cache mmu_gfn_array_cache;struct kvm_mmu_memory_cache mmu_page_header_cache; +struct kvm_mmu_memory_cache mmu_private_sp_cache;/* * QEMU userspace and the guest each have their own FPU state.
@@ -1052,6 +1057,7 @@ struct kvm_arch {u8 mmu_valid_gen;struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES ];struct list_head active_mmu_pages; +struct list_head private_mmu_pages;struct list_head zapped_obsolete_pages;struct list_head lpage_disallowed_mmu_pages;struct kvm_page_track_notifier_node mmu_sp_tracker;
@@ -1236,6 +1242,8 @@ struct kvm_arch {hpa_t hv_root_tdp;spinlock_t hv_root_tdp_lock;#endif + +gfn_t gfn_shared_mask; };struct kvm_vm_stat {
@@ -1328,6 +1336,7 @@ struct kvm_x86_ops {bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm); +void (*mmu_prezap)(struct kvm *kvm);void (*vm_teardown)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);
@@ -1423,6 +1432,17 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level); +void (*set_private_spte)(struct kvm *kvm,gfn_t gfn,enum pg_level level, +kvm_pfn_t pfn); +void (*drop_private_spte)(struct kvm *kvm,gfn_t gfn,enum pg_level level, +kvm_pfn_t pfn); +void (*zap_private_spte)(struct kvm *kvm,gfn_t gfn,enum pg_level level); +void (*unzap_private_spte)(struct kvm *kvm,gfn_t gfn,enum pg_level level); +int (*link_private_sp)(struct kvm *kvm,gfn_t gfn,enum pg_level level, +void *private_sp); +int (*free_private_sp)(struct kvm *kvm,gfn_t gfn,enum pg_level level, +void *private_sp); +bool (*has_wbinvd_exit)(void );u64 (*get_l2_tsc_offset)(struct kvm_vcpu *vcpu);
@@ -1602,7 +1622,8 @@ void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm, const struct kvm_memory_slot *memslot);void kvm_mmu_slot_leaf_clear_dirty (struct kvm *kvm,const struct kvm_memory_slot *memslot); -void kvm_mmu_zap_all (struct kvm *kvm); +void kvm_mmu_zap_all_active (struct kvm *kvm); +void kvm_mmu_zap_all_private (struct kvm *kvm);void kvm_mmu_invalidate_mmio_sptes (struct kvm *kvm,u64 gen);unsigned long kvm_mmu_calculate_default_mmu_pages (struct kvm *kvm);void kvm_mmu_change_mmu_pages (struct kvm *kvm,unsigned long kvm_nr_mmu_pages);
@@ -1765,7 +1786,9 @@ static inline int __kvm_irq_line_state(unsigned long *irq_state,#define KVM_MMU_ROOT_CURRENT BIT(0)#define KVM_MMU_ROOT_PREVIOUS (i) BIT(1+i) -#define KVM_MMU_ROOTS_ALL (~0UL) +#define KVM_MMU_ROOT_PRIVATE BIT(1+KVM_MMU_NUM_PREV_ROOTS) +#define KVM_MMU_ROOTS_ALL ((u32)(~KVM_MMU_ROOT_PRIVATE)) +#define KVM_MMU_ROOTS_ALL_INC_PRIVATE (KVM_MMU_ROOTS_ALL | KVM_MMU_ROOT_PRIVATE)int kvm_pic_set_irq (struct kvm_pic *pic,int irq,int irq_source_id,int level);void kvm_pic_clear_all (struct kvm_pic *pic,int irq_source_id);
@@ -1973,4 +1996,7 @@ int memslot_rmap_alloc(struct kvm_memory_slot *slot, unsigned long npages);#define KVM_CLOCK_VALID_FLAGS \ (KVM_CLOCK_TSC_STABLE | KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC) +bool kvm_is_private_gfn (struct kvm *kvm,gfn_t gfn); +bool kvm_vcpu_is_private_gfn (struct kvm_vcpu *vcpu,gfn_t gfn); +#endif /* _ASM_X86_KVM_HOST_H */
arch/x86/kvm/mmu.h +5 -11
@@ -355,13 +355,7 @@ static inline void kvm_update_page_stats(struct kvm *kvm, int level, int count)static inline gfn_t kvm_gfn_stolen_mask (struct kvm *kvm) { -/* Currently there are no stolen bits in KVM */ -return 0 ; -} - -static inline gfn_t vcpu_gfn_stolen_mask (struct kvm_vcpu *vcpu) -{ -return kvm_gfn_stolen_mask (vcpu->kvm); +return kvm->arch.gfn_shared_mask; }static inline gpa_t kvm_gpa_stolen_mask (struct kvm *kvm)
@@ -369,14 +363,14 @@ static inline gpa_t kvm_gpa_stolen_mask(struct kvm *kvm) return kvm_gfn_stolen_mask(kvm) << PAGE_SHIFT; } -static inline gpa_t vcpu_gpa_stolen_mask (struct kvm_vcpu *vcpu) +static inline gpa_t kvm_gpa_unalias (struct kvm *kvm,gpa_t gpa) { -return kvm_gpa_stolen_mask (vcpu->kvm); +return gpa & ~kvm_gpa_stolen_mask (kvm); } -static inline gfn_t vcpu_gpa_to_gfn_unalias (struct kvm_vcpu *vcpu,gpa_t gpa) +static inline gfn_t kvm_gfn_unalias (struct kvm *kvm,gpa_t gpa) { -return (gpa >>PAGE_SHIFT ) & ~vcpu_gfn_stolen_mask (vcpu); +return kvm_gpa_unalias (kvm, gpa) >>PAGE_SHIFT ; }#endif
arch/x86/kvm/mmu/mmu.c +382 -96
833 lines not on this page · raw message
arch/x86/kvm/mmu/mmu_internal.h +12 -1
@@ -49,11 +49,12 @@ struct kvm_mmu_page {*/ union kvm_mmu_page_role role;gfn_t gfn; -gfn_t gfn_stolen_bits;u64 *spt;/* hold the gfn of each spte inside spt */ gfn_t *gfns; +/* associated private shadow page, e.g. SEPT page */ +void *private_sp;/* Currently serving as active root */ union {int root_count;
@@ -105,6 +106,16 @@ static inline int kvm_mmu_page_as_id(struct kvm_mmu_page *sp)return kvm_mmu_role_as_id (sp->role); } +static inline bool is_private_sp (struct kvm_mmu_page *sp) +{ +return sp->role.private; +} + +static inline bool is_private_spte (u64 *sptep) +{ +return is_private_sp (sptep_to_sp (sptep)); +} +static inline bool kvm_vcpu_ad_need_write_protect (struct kvm_vcpu *vcpu) {/*
arch/x86/kvm/mmu/paging_tmpl.h +6 -5
@@ -25,7 +25,7 @@#define guest_walker guest_walker64#define FNAME (name) paging##64_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (vcpu, lvl) (~vcpu_gpa_stolen_mask(vcpu) & \ +#define PT_LVL_ADDR_MASK (vcpu, lvl) (~kvm_gpa_stolen_mask(vcpu->kvm) & \ PT64_LVL_ADDR_MASK(lvl))#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)
@@ -567,7 +567,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,gfn_t gfn;kvm_pfn_t pfn; -WARN_ON (gpte &vcpu_gpa_stolen_mask (vcpu)); +WARN_ON (gpte &kvm_gpa_stolen_mask (vcpu->kvm));if (FNAME (prefetch_invalid_gpte)(vcpu, sp, spte, gpte))return false;
@@ -670,7 +670,7 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,WARN_ON_ONCE (gw->gfn != base_gfn); direct_access = gw->pte_access; -WARN_ON (fault->addr &vcpu_gpa_stolen_mask (vcpu)); +WARN_ON (fault->addr &kvm_gpa_stolen_mask (vcpu->kvm)); top_level = vcpu->arch.mmu->root_level;if (top_level ==PT32E_ROOT_LEVEL )
@@ -700,7 +700,7 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, table_gfn = gw->table_gfn[it.level -2 ]; access = gw->pt_access[it.level -2 ]; sp =kvm_mmu_get_page (vcpu, table_gfn, fault->addr, - it.level-1 , false, access); + it.level-1 , false, access, false);/* * We must synchronize the pagetable before linking it * because the guest doesn't need to flush tlb when
@@ -757,7 +757,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,if (!is_shadow_present_pte (*it.sptep)) { sp =kvm_mmu_get_page (vcpu, base_gfn, fault->addr, - it.level -1 , true, direct_access); + it.level -1 , true, direct_access, + false);link_shadow_page (vcpu, it.sptep, sp);if (fault->huge_page_disallowed && fault->req_level >= it.level)
arch/x86/kvm/mmu/spte.h +12 -4
@@ -14,6 +14,9 @@*/ #define SPTE_MMU_PRESENT_MASK BIT_ULL(11) +/* Masks that used to track metadata for not-present SPTEs. */ +#define SPTE_PRIVATE_ZAPPED BIT_ULL(62) +/* * TDP SPTES (more specifically, EPT SPTEs) may not have A/D bits, and may also * be restricted to using write-protection (for L2 when CPU dirty logging, i.e.
@@ -95,11 +98,11 @@ static_assert(!(EPT_SPTE_MMU_WRITABLE & SHADOW_ACC_TRACK_SAVED_MASK));#undef SHADOW_ACC_TRACK_SAVED_MASK/* -* Due to limited space in PTEs, the MMIO generation is a 19 bit subset of +* Due to limited space in PTEs, the MMIO generation is a 18 bit subset of * the memslots generation and is derived as follows: * * Bits 0-7 of the MMIO generation are propagated to spte bits 3-10 -* Bits 8-18 of the MMIO generation are propagated to spte bits 52-62 +* Bits 8-17 of the MMIO generation are propagated to spte bits 52-61 * * The KVM_MEMSLOT_GEN_UPDATE_IN_PROGRESS flag is intentionally not included in * the MMIO generation number, as doing so would require stealing a bit from
@@ -113,7 +116,7 @@ static_assert(!(EPT_SPTE_MMU_WRITABLE & SHADOW_ACC_TRACK_SAVED_MASK));#define MMIO_SPTE_GEN_LOW_END 10#define MMIO_SPTE_GEN_HIGH_START 52 -#define MMIO_SPTE_GEN_HIGH_END 62 +#define MMIO_SPTE_GEN_HIGH_END 61#define MMIO_SPTE_GEN_LOW_MASK GENMASK_ULL(MMIO_SPTE_GEN_LOW_END, \ MMIO_SPTE_GEN_LOW_START)
@@ -126,7 +129,7 @@ static_assert(!(SPTE_MMU_PRESENT_MASK &#define MMIO_SPTE_GEN_HIGH_BITS (MMIO_SPTE_GEN_HIGH_END - MMIO_SPTE_GEN_HIGH_START + 1)/* remember to adjust the comment above as well if you change these */ -static_assert (MMIO_SPTE_GEN_LOW_BITS ==8 &&MMIO_SPTE_GEN_HIGH_BITS ==11 ); +static_assert (MMIO_SPTE_GEN_LOW_BITS ==8 &&MMIO_SPTE_GEN_HIGH_BITS ==10 );#define MMIO_SPTE_GEN_LOW_SHIFT (MMIO_SPTE_GEN_LOW_START - 0)#define MMIO_SPTE_GEN_HIGH_SHIFT (MMIO_SPTE_GEN_HIGH_START - MMIO_SPTE_GEN_LOW_BITS)
@@ -267,6 +270,11 @@ static inline bool is_access_track_spte(u64 spte)return !spte_ad_enabled (spte) && (spte & shadow_acc_track_mask) ==0 ; } +static inline bool is_zapped_private_pte (u64 pte) +{ +return !!(pte &SPTE_PRIVATE_ZAPPED ); +} +static inline bool is_large_pte (u64 pte) {return pte &PT_PAGE_SIZE_MASK ;
arch/x86/kvm/x86.c +9 -1
@@ -11436,6 +11436,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)INIT_HLIST_HEAD (&kvm->arch.mask_notifier_list);INIT_LIST_HEAD (&kvm->arch.active_mmu_pages); +INIT_LIST_HEAD (&kvm->arch.private_mmu_pages);INIT_LIST_HEAD (&kvm->arch.zapped_obsolete_pages);INIT_LIST_HEAD (&kvm->arch.lpage_disallowed_mmu_pages);INIT_LIST_HEAD (&kvm->arch.assigned_dev_head);
@@ -11872,7 +11873,14 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,void kvm_arch_flush_shadow_all (struct kvm *kvm) { -kvm_mmu_zap_all (kvm); +/* +* kvm_mmu_zap_all_active() zaps both private and shared page tables. +* Before tearing down private page tables, TDX requires TD has started +* to be destroyed (i.e. keyID must have been reclaimed, etc). Invoke +* kvm_x86_mmu_prezap() for this. +*/ +static_call_cond (kvm_x86_mmu_prezap)(kvm); +kvm_mmu_zap_all_active (kvm); } void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
virt/kvm/kvm_main.c +1 -0
@@ -190,6 +190,7 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn)return true; } +EXPORT_SYMBOL_GPL (kvm_is_reserved_pfn);/* * Switches to specified vcpu, until a matching vcpu_put()
Signature · 1 line
2.25.1
RFC PATCH v3 38/59 KVM: x86/mmu: Allow per-VM override of the TDP max page level
In reply to [email protected]From: Sean Christopherson <[email protected]>
TODO: This is tentative patch. Support large page and delete this patch.
Allow TDX to effectively disable large pages, as SEPT will initially support only 4k pages.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu.h | 2 +- arch/x86/kvm/mmu/mmu.c | 2 ++ 3 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1054,6 +1054,7 @@ struct kvm_arch {unsigned long n_requested_mmu_pages;unsigned long n_max_mmu_pages;unsigned int indirect_shadow_pages; +int tdp_max_page_level;u8 mmu_valid_gen;struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES ];struct list_head active_mmu_pages;
arch/x86/kvm/mmu.h +1 -1
@@ -191,7 +191,7 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .is_tdp = likely(vcpu->arch.mmu->page_fault == kvm_tdp_page_fault), .nx_huge_page_workaround_enabled = is_nx_huge_page_enabled(), - .max_level = KVM_MAX_HUGEPAGE_LEVEL, + .max_level = vcpu->kvm->arch.tdp_max_page_level, .req_level = PG_LEVEL_4K, .goal_level = PG_LEVEL_4K, };
arch/x86/kvm/mmu/mmu.c +2 -0
@@ -6102,6 +6102,8 @@ void kvm_mmu_init_vm(struct kvm *kvm) node->track_write = kvm_mmu_pte_write; node->track_flush_slot = kvm_mmu_invalidate_zap_pages_in_memslot;kvm_page_track_register_notifier (kvm, node); + + kvm->arch.tdp_max_page_level =KVM_MAX_HUGEPAGE_LEVEL ; }void kvm_mmu_uninit_vm (struct kvm *kvm)
Signature · 1 line
2.25.1
RFC PATCH v3 33/59 KVM: x86/mmu: Ignore bits 63 and 62 when checking for "present" SPTEs
In reply to [email protected]From: Sean Christopherson <[email protected]>
Ignore bits 63 and 62 when checking for present SPTEs to allow setting said bits in not-present SPTEs. TDX will set bit 63 in "zero" SPTEs to suppress #VEs (TDX-SEAM unconditionally enables EPT Violation #VE), and will use bit 62 to track zapped private SPTEs.
--- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/spte.h | 13 +++++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu/paging_tmpl.h +1 -1
@@ -1102,7 +1102,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)gpa_t pte_gpa;gfn_t gfn; -if (!sp->spt[i]) +if (!__is_shadow_present_pte (sp->spt[i]))continue ; pte_gpa = first_pte_gpa + i *sizeof (pt_element_t);
arch/x86/kvm/mmu/spte.h +13 -0
@@ -208,6 +208,19 @@ static inline bool is_mmio_spte(u64 spte)likely (shadow_mmio_value); } +static inline bool __is_shadow_present_pte (u64 pte) +{ +/* +* Ignore bits 63 and 62 so that they can be set in SPTEs that are well +* and truly not present. We can't use the sane/obvious approach of +* querying bits 2:0 (RWX or P) because EPT without A/D bits will clear +* RWX of a "present" SPTE to do access tracking. Tracking updates can +* be done out of mmu_lock, so even the flushing logic needs to treat +* such SPTEs as present. +*/ +return !!(pte <<2 ); +} +static inline bool is_shadow_present_pte (u64 pte) {return !!(pte &SPTE_MMU_PRESENT_MASK );
Signature · 1 line
2.25.1
RFC PATCH v3 39/59 KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
In reply to [email protected]From: Sean Christopherson <[email protected]>
Pass intr_info to the NMI and INTR handlers instead of pulling it from vcpu_vmx in preparation for sharing the bulk of the handlers with TDX.
--- arch/x86/kvm/vmx/vmx.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
arch/x86/kvm/vmx/vmx.c +7 -8
@@ -6302,25 +6302,24 @@ static void handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu,kvm_after_interrupt (vcpu); } -static void handle_exception_nmi_irqoff (struct vcpu_vmx *vmx) +static void handle_exception_nmi_irqoff (struct kvm_vcpu *vcpu,u32 intr_info) {const unsigned long nmi_entry = (unsigned long )asm_exc_nmi_noist; -u32 intr_info =vmx_get_intr_info (&vmx->vcpu);/* if exit due to PF check for async PF */ if (is_page_fault (intr_info)) - vmx->vcpu.arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags (); + vcpu->arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags ();/* Handle machine checks before interrupts are enabled */ else if (is_machine_check (intr_info))kvm_machine_check ();/* We need to handle NMIs before interrupts are enabled */ else if (is_nmi (intr_info)) -handle_interrupt_nmi_irqoff (&vmx->vcpu, nmi_entry); +handle_interrupt_nmi_irqoff (vcpu, nmi_entry); } -static void handle_external_interrupt_irqoff (struct kvm_vcpu *vcpu) +static void handle_external_interrupt_irqoff (struct kvm_vcpu *vcpu, +u32 intr_info) { -u32 intr_info =vmx_get_intr_info (vcpu);unsigned int vector = intr_info &INTR_INFO_VECTOR_MASK ;gate_desc *desc = (gate_desc *)host_idt_base + vector;
@@ -6339,9 +6338,9 @@ static void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu)return ;if (vmx->exit_reason.basic ==EXIT_REASON_EXTERNAL_INTERRUPT ) -handle_external_interrupt_irqoff (vcpu); +handle_external_interrupt_irqoff (vcpu,vmx_get_intr_info (vcpu));else if (vmx->exit_reason.basic ==EXIT_REASON_EXCEPTION_NMI ) -handle_exception_nmi_irqoff (vmx); +handle_exception_nmi_irqoff (vcpu,vmx_get_intr_info (vcpu)); }/*
Signature · 1 line
2.25.1
RFC PATCH v3 45/59 KVM: VMX: Move setting of EPT MMU masks to common VT-x code
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/vmx/main.c | 5 +++++ arch/x86/kvm/vmx/vmx.c | 4 ---- 2 files changed, 5 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/main.c +5 -0
@@ -4,6 +4,7 @@#include "x86_ops.h" #include "vmx.h" #include "nested.h" +#include "mmu.h" #include "pmu.h" static int __init vt_cpu_has_kvm_support (void )
@@ -35,6 +36,10 @@ static __init int vt_hardware_setup(void)if (ret)return ret; +if (enable_ept) +kvm_mmu_set_ept_masks (enable_ept_ad_bits, +cpu_has_vmx_ept_execute_only ()); +return 0 ; }
arch/x86/kvm/vmx/vmx.c +0 -4
@@ -7546,10 +7546,6 @@ __init int hardware_setup(void)set_bit (0 , vmx_vpid_bitmap);/* 0 is reserved for host */ -if (enable_ept) -kvm_mmu_set_ept_masks (enable_ept_ad_bits, -cpu_has_vmx_ept_execute_only ()); -if (!enable_ept) ept_lpage_level =0 ;else if (cpu_has_vmx_ept_1g_page ())
Signature · 1 line
2.25.1
RFC PATCH v3 41/59 KVM: VMX: Split out guts of EPT violation to common/exposed function
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/vmx/common.h | 29 +++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 33 +++++---------------------------- 2 files changed, 34 insertions(+), 28 deletions(-)
arch/x86/kvm/vmx/common.h +29 -0
@@ -5,8 +5,11 @@#include <linux/kvm_host.h> #include <asm/traps.h> +#include <asm/vmx.h> +#include "mmu.h" #include "vmcs.h" +#include "vmx.h" #include "x86.h" extern unsigned long vmx_host_idt_base;
@@ -49,4 +52,30 @@ static inline void vmx_handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, vmx_handle_interrupt_nmi_irqoff(vcpu, gate_offset(desc)); } +static inline int __vmx_handle_ept_violation (struct kvm_vcpu *vcpu,gpa_t gpa, +unsigned long exit_qualification) +{ +u64 error_code; + +/* Is it a read fault? */ + error_code = (exit_qualification &EPT_VIOLATION_ACC_READ ) + ?PFERR_USER_MASK :0 ; +/* Is it a write fault? */ + error_code |= (exit_qualification &EPT_VIOLATION_ACC_WRITE ) + ?PFERR_WRITE_MASK :0 ; +/* Is it a fetch fault? */ + error_code |= (exit_qualification &EPT_VIOLATION_ACC_INSTR ) + ?PFERR_FETCH_MASK :0 ; +/* ept page table entry is present? */ + error_code |= (exit_qualification & + (EPT_VIOLATION_READABLE |EPT_VIOLATION_WRITABLE | +EPT_VIOLATION_EXECUTABLE )) + ?PFERR_PRESENT_MASK :0 ; + + error_code |= (exit_qualification &EPT_VIOLATION_GVA_TRANSLATED ) !=0 ? +PFERR_GUEST_FINAL_MASK :PFERR_GUEST_PAGE_MASK ; + +return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); +} +#endif /* __KVM_X86_VMX_COMMON_H */
arch/x86/kvm/vmx/vmx.c +5 -28
@@ -5230,11 +5230,10 @@ static int handle_task_switch(struct kvm_vcpu *vcpu)static int handle_ept_violation (struct kvm_vcpu *vcpu) { -unsigned long exit_qualification; -gpa_t gpa; -u64 error_code; +unsigned long exit_qualification =vmx_get_exit_qual (vcpu); +gpa_t gpa =vmcs_read64 (GUEST_PHYSICAL_ADDRESS ); - exit_qualification =vmx_get_exit_qual (vcpu); +trace_kvm_page_fault (gpa, exit_qualification);/* * EPT violation happened while executing iret from NMI,
@@ -5243,31 +5242,9 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)* AAK134, BY25. */ if (!(to_vmx (vcpu)->idt_vectoring_info &VECTORING_INFO_VALID_MASK ) && - enable_vnmi && - (exit_qualification &INTR_INFO_UNBLOCK_NMI )) + enable_vnmi && (exit_qualification &INTR_INFO_UNBLOCK_NMI ))vmcs_set_bits (GUEST_INTERRUPTIBILITY_INFO ,GUEST_INTR_STATE_NMI ); - gpa =vmcs_read64 (GUEST_PHYSICAL_ADDRESS ); -trace_kvm_page_fault (gpa, exit_qualification); - -/* Is it a read fault? */ - error_code = (exit_qualification &EPT_VIOLATION_ACC_READ ) - ?PFERR_USER_MASK :0 ; -/* Is it a write fault? */ - error_code |= (exit_qualification &EPT_VIOLATION_ACC_WRITE ) - ?PFERR_WRITE_MASK :0 ; -/* Is it a fetch fault? */ - error_code |= (exit_qualification &EPT_VIOLATION_ACC_INSTR ) - ?PFERR_FETCH_MASK :0 ; -/* ept page table entry is present? */ - error_code |= (exit_qualification & - (EPT_VIOLATION_READABLE |EPT_VIOLATION_WRITABLE | -EPT_VIOLATION_EXECUTABLE )) - ?PFERR_PRESENT_MASK :0 ; - - error_code |= (exit_qualification &EPT_VIOLATION_GVA_TRANSLATED ) !=0 ? -PFERR_GUEST_FINAL_MASK :PFERR_GUEST_PAGE_MASK ; - vcpu->arch.exit_qualification = exit_qualification;/*
@@ -5281,7 +5258,7 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)if (unlikely (allow_smaller_maxphyaddr &&kvm_vcpu_is_illegal_gpa (vcpu, gpa)))return kvm_emulate_instruction (vcpu,0 ); -return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); +return __vmx_handle_ept_violation (vcpu, gpa, exit_qualification); }static int handle_ept_misconfig (struct kvm_vcpu *vcpu)
Signature · 1 line
2.25.1
RFC PATCH v3 31/59 KVM: x86: Add infrastructure for stolen GPA bits
In reply to [email protected]From: Rick Edgecombe <[email protected]>
Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA alising will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.
Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.
With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.
To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.
To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.
GFNs (without stolen bits) continue to be used to:
-Specify physical memory by userspace via memslots -Map GPAs to TDP PTEs via RMAP
-Specify dirty tracking and write protection -Look up MTRR types
-Inject async page faults
Since there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN, kvm_handle_hva() based operations will be applied to both aliases.
In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.
For TLB flushes that are address based, make sure to flush both aliases in the stolen bits case.
Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.
--- arch/x86/kvm/mmu.h | 27 +++++++++++ arch/x86/kvm/mmu/mmu.c | 82 +++++++++++++++++++++++---------- arch/x86/kvm/mmu/mmu_internal.h | 1 + arch/x86/kvm/mmu/paging_tmpl.h | 25 ++++++---- 4 files changed, 100 insertions(+), 35 deletions(-)
arch/x86/kvm/mmu.h +27 -0
@@ -351,4 +351,31 @@ static inline void kvm_update_page_stats(struct kvm *kvm, int level, int count) {atomic64_add (count, &kvm->stat.pages[level -1 ]); } + +static inline gfn_t kvm_gfn_stolen_mask (struct kvm *kvm) +{ +/* Currently there are no stolen bits in KVM */ +return 0 ; +} + +static inline gfn_t vcpu_gfn_stolen_mask (struct kvm_vcpu *vcpu) +{ +return kvm_gfn_stolen_mask (vcpu->kvm); +} + +static inline gpa_t kvm_gpa_stolen_mask (struct kvm *kvm) +{ +return kvm_gfn_stolen_mask (kvm) <<PAGE_SHIFT ; +} + +static inline gpa_t vcpu_gpa_stolen_mask (struct kvm_vcpu *vcpu) +{ +return kvm_gpa_stolen_mask (vcpu->kvm); +} + +static inline gfn_t vcpu_gpa_to_gfn_unalias (struct kvm_vcpu *vcpu,gpa_t gpa) +{ +return (gpa >>PAGE_SHIFT ) & ~vcpu_gfn_stolen_mask (vcpu); +} +#endif
arch/x86/kvm/mmu/mmu.c +57 -25
153 lines not on this page · raw message
arch/x86/kvm/mmu/mmu_internal.h +1 -0
@@ -49,6 +49,7 @@ struct kvm_mmu_page {*/ union kvm_mmu_page_role role;gfn_t gfn; +gfn_t gfn_stolen_bits;u64 *spt;/* hold the gfn of each spte inside spt */
arch/x86/kvm/mmu/paging_tmpl.h +15 -10
@@ -25,7 +25,8 @@#define guest_walker guest_walker64#define FNAME (name) paging##64_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) (~vcpu_gpa_stolen_mask(vcpu) & \ + PT64_LVL_ADDR_MASK(lvl))#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS
@@ -44,7 +45,7 @@#define guest_walker guest_walker32#define FNAME (name) paging##32_##name#define PT_BASE_ADDR_MASK PT32_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT32_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT32_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT32_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT32_INDEX(addr, level)#define PT_LEVEL_BITS PT32_LEVEL_BITS
@@ -58,7 +59,7 @@#define guest_walker guest_walkerEPT#define FNAME (name) ept_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT64_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS
@@ -75,7 +76,7 @@#define PT_GUEST_ACCESSED_MASK (1 << PT_GUEST_ACCESSED_SHIFT)#define gpte_to_gfn_lvl FNAME(gpte_to_gfn_lvl) -#define gpte_to_gfn (pte) gpte_to_gfn_lvl((pte), PG_LEVEL_4K) +#define gpte_to_gfn (vcpu, pte) gpte_to_gfn_lvl(vcpu, pte, PG_LEVEL_4K)/* * The guest_walker structure emulates the behavior of the hardware page
@@ -96,9 +97,9 @@ struct guest_walker {struct x86_exception fault; }; -static gfn_t gpte_to_gfn_lvl (pt_element_t gpte,int lvl) +static gfn_t gpte_to_gfn_lvl (struct kvm_vcpu *vcpu,pt_element_t gpte,int lvl) { -return (gpte &PT_LVL_ADDR_MASK (lvl)) >>PAGE_SHIFT ; +return (gpte &PT_LVL_ADDR_MASK (vcpu, lvl)) >>PAGE_SHIFT ; } static inline void FNAME(protect_clean_gpte)(struct kvm_mmu *mmu, unsigned *access,
@@ -395,7 +396,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, --walker->level; index =PT_INDEX (addr, walker->level); - table_gfn =gpte_to_gfn (pte); + table_gfn =gpte_to_gfn (vcpu, pte); offset = index *sizeof (pt_element_t); pte_gpa =gfn_to_gpa (table_gfn) + offset;
@@ -461,7 +462,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,if (unlikely (errcode)) goto error; - gfn =gpte_to_gfn_lvl (pte, walker->level); + gfn =gpte_to_gfn_lvl (vcpu, pte, walker->level); gfn += (addr &PT_LVL_OFFSET_MASK (walker->level)) >>PAGE_SHIFT ;if (PTTYPE ==32 && walker->level >PG_LEVEL_4K &&is_cpuid_PSE36 ())
@@ -566,12 +567,14 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,gfn_t gfn;kvm_pfn_t pfn; +WARN_ON (gpte &vcpu_gpa_stolen_mask (vcpu)); +if (FNAME (prefetch_invalid_gpte)(vcpu, sp, spte, gpte))return false;pgprintk ("%s: gpte %llx spte %p\n" , __func__, (u64 )gpte, spte); - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access &FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);
@@ -667,6 +670,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,WARN_ON_ONCE (gw->gfn != base_gfn); direct_access = gw->pte_access; +WARN_ON (fault->addr &vcpu_gpa_stolen_mask (vcpu)); + top_level = vcpu->arch.mmu->root_level;if (top_level ==PT32E_ROOT_LEVEL ) top_level =PT32_ROOT_LEVEL ;
@@ -1111,7 +1116,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)continue ; } - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access; pte_access &=FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);
Signature · 1 line
2.25.1
RFC PATCH v3 43/59 KVM: VMX: Define VMCS encodings for shared EPT pointer
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add the VMCS field encoding for the shared EPTP, which will be used by TDX to have separate EPT walks for private GPAs (existing EPTP) versus shared GPAs (new shared EPTP).
--- arch/x86/include/asm/vmx.h | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/include/asm/vmx.h +2 -0
@@ -235,6 +235,8 @@ enum vmcs_field {ENCLS_EXITING_BITMAP_HIGH =0x0000202F ,TSC_MULTIPLIER =0x00002032 ,TSC_MULTIPLIER_HIGH =0x00002033 , +SHARED_EPT_POINTER =0x0000203C , +SHARED_EPT_POINTER_HIGH =0x0000203D ,GUEST_PHYSICAL_ADDRESS =0x00002400 ,GUEST_PHYSICAL_ADDRESS_HIGH =0x00002401 , VMCS_LINK_POINTER = 0x00002800,
Signature · 1 line
2.25.1
RFC PATCH v3 40/59 KVM: VMX: Move NMI/exception handler to common helper
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/vmx/common.h | 52 +++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 52 ++++++--------------------------------- 2 files changed, 60 insertions(+), 44 deletions(-) create mode 100644 arch/x86/kvm/vmx/common.h
arch/x86/kvm/vmx/common.h new file +52 -0
@@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __KVM_X86_VMX_COMMON_H +#define __KVM_X86_VMX_COMMON_H + +#include <linux/kvm_host.h> + +#include <asm/traps.h> + +#include "vmcs.h" +#include "x86.h" + +extern unsigned long vmx_host_idt_base; +void vmx_do_interrupt_nmi_irqoff (unsigned long entry); + +static inline void vmx_handle_interrupt_nmi_irqoff (struct kvm_vcpu *vcpu, +unsigned long entry) +{ +kvm_before_interrupt (vcpu); +vmx_do_interrupt_nmi_irqoff (entry); +kvm_after_interrupt (vcpu); +} + +static inline void vmx_handle_exception_nmi_irqoff (struct kvm_vcpu *vcpu, +u32 intr_info) +{ +const unsigned long nmi_entry = (unsigned long )asm_exc_nmi_noist; + +/* if exit due to PF check for async PF */ +if (is_page_fault (intr_info)) + vcpu->arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags (); +/* Handle machine checks before interrupts are enabled */ +else if (is_machine_check (intr_info)) +kvm_machine_check (); +/* We need to handle NMIs before interrupts are enabled */ +else if (is_nmi (intr_info)) +vmx_handle_interrupt_nmi_irqoff (vcpu, nmi_entry); +} + +static inline void vmx_handle_external_interrupt_irqoff (struct kvm_vcpu *vcpu, +u32 intr_info) +{ +unsigned int vector = intr_info &INTR_INFO_VECTOR_MASK ; +gate_desc *desc = (gate_desc *)vmx_host_idt_base + vector; + +if (KVM_BUG (!is_external_intr (intr_info), vcpu->kvm, +"KVM: unexpected VM-Exit interrupt info: 0x%x" , intr_info)) +return ; + +vmx_handle_interrupt_nmi_irqoff (vcpu,gate_offset (desc)); +} + +#endif /* __KVM_X86_VMX_COMMON_H */
arch/x86/kvm/vmx/vmx.c +8 -44
95 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v3 42/59 KVM: VMX: Define EPT Violation architectural bits
In reply to [email protected]From: Sean Christopherson <[email protected]>
Define the EPT Violation #VE control bit, #VE info VMCS fields, and the suppress #VE bit for EPT entries.
--- arch/x86/include/asm/vmx.h | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/include/asm/vmx.h +4 -0
@@ -78,6 +78,7 @@ struct vmcs {#define SECONDARY_EXEC_ENCLS_EXITING VMCS_CONTROL_BIT(ENCLS_EXITING)#define SECONDARY_EXEC_RDSEED_EXITING VMCS_CONTROL_BIT(RDSEED_EXITING)#define SECONDARY_EXEC_ENABLE_PML VMCS_CONTROL_BIT(PAGE_MOD_LOGGING) +#define SECONDARY_EXEC_EPT_VIOLATION_VE VMCS_CONTROL_BIT(EPT_VIOLATION_VE)#define SECONDARY_EXEC_PT_CONCEAL_VMX VMCS_CONTROL_BIT(PT_CONCEAL_VMX)#define SECONDARY_EXEC_XSAVES VMCS_CONTROL_BIT(XSAVES)#define SECONDARY_EXEC_MODE_BASED_EPT_EXEC VMCS_CONTROL_BIT(MODE_BASED_EPT_EXEC)
@@ -226,6 +227,8 @@ enum vmcs_field {VMREAD_BITMAP_HIGH =0x00002027 ,VMWRITE_BITMAP =0x00002028 ,VMWRITE_BITMAP_HIGH =0x00002029 , +VE_INFO_ADDRESS =0x0000202A , +VE_INFO_ADDRESS_HIGH =0x0000202B ,XSS_EXIT_BITMAP =0x0000202C ,XSS_EXIT_BITMAP_HIGH =0x0000202D , ENCLS_EXITING_BITMAP = 0x0000202E,
@@ -509,6 +512,7 @@ enum vmcs_field {#define VMX_EPT_IPAT_BIT (1ull << 6)#define VMX_EPT_ACCESS_BIT (1ull << 8)#define VMX_EPT_DIRTY_BIT (1ull << 9) +#define VMX_EPT_SUPPRESS_VE_BIT (1ull << 63)#define VMX_EPT_RWX_MASK (VMX_EPT_READABLE_MASK | \ VMX_EPT_WRITABLE_MASK | \ VMX_EPT_EXECUTABLE_MASK)
Signature · 1 line
2.25.1
RFC PATCH v3 44/59 KVM: VMX: Add 'main.c' to wrap VMX and TDX
In reply to [email protected]From: Sean Christopherson <[email protected]>
Wrap the VMX kvm_x86_ops hooks in preparation of adding TDX, which can coexist with VMX, i.e. KVM can run both VMs and TDs. Use 'vt' for the naming scheme as a nod to VT-x and as a concatenation of VmxTdx.
--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/main.c | 729 +++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 463 ++++++++--------------- arch/x86/kvm/vmx/x86_ops.h | 124 +++++++ 4 files changed, 1000 insertions(+), 318 deletions(-) create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/x86_ops.h
arch/x86/kvm/Makefile +1 -1
@@ -27,7 +27,7 @@ kvm-$(CONFIG_X86_64) += mmu/tdp_iter.o mmu/tdp_mmu.o kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ - vmx/evmcs.o vmx/nested.o vmx/posted_intr.o + vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx_error.o
arch/x86/kvm/vmx/main.c new file +729 -0
729 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.c +146 -317
1,017 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h new file +124 -0
124 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v3 47/59 KVM: TDX: Define TDCALL exit reason
In reply to [email protected]From: Sean Christopherson <[email protected]>
Define the TDCALL exit reason, which is carved out from the VMX exit reason namespace as the TDCALL exit from TDX guest to TDX-SEAM is really just a VM-Exit.
--- arch/x86/include/uapi/asm/vmx.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/include/uapi/asm/vmx.h +3 -1
@@ -91,6 +91,7 @@#define EXIT_REASON_UMWAIT 67#define EXIT_REASON_TPAUSE 68#define EXIT_REASON_BUS_LOCK 74 +#define EXIT_REASON_TDCALL 77#define VMX_EXIT_REASONS \ { EXIT_REASON_EXCEPTION_NMI, "EXCEPTION_NMI" }, \
@@ -153,7 +154,8 @@ { EXIT_REASON_XRSTORS, "XRSTORS" }, \ { EXIT_REASON_UMWAIT, "UMWAIT" }, \ { EXIT_REASON_TPAUSE, "TPAUSE" }, \ - { EXIT_REASON_BUS_LOCK, "BUS_LOCK" } + { EXIT_REASON_BUS_LOCK, "BUS_LOCK" }, \ + { EXIT_REASON_TDCALL, "TDCALL" } #define VMX_EXIT_REASON_FLAGS \ { VMX_EXIT_REASONS_FAILED_VMENTRY, "FAILED_VMENTRY" }
Signature · 1 line
2.25.1
RFC PATCH v3 46/59 KVM: VMX: Move register caching logic to common code
In reply to [email protected]From: Sean Christopherson <[email protected]>
Move the guts of vmx_cache_reg() to vt_cache_reg() in preparation for reusing the bulk of the code for TDX, which can access guest state for debug TDs.
Use kvm_x86_ops.cache_reg() in ept_update_paging_mode_cr0() rather than trying to expose vt_cache_reg() to vmx.c, even though it means taking a retpoline. The code runs if and only if EPT is enabled but unrestricted guest. Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.
--- arch/x86/kvm/vmx/main.c | 44 ++++++++++++++++++++++++++++++++++--- arch/x86/kvm/vmx/vmx.c | 45 +------------------------------------- arch/x86/kvm/vmx/x86_ops.h | 1 + 3 files changed, 43 insertions(+), 47 deletions(-)
arch/x86/kvm/vmx/main.c +41 -3
@@ -339,9 +339,47 @@ static void vt_sync_dirty_debug_regs(struct kvm_vcpu *vcpu)vmx_sync_dirty_debug_regs (vcpu); } -static void vt_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg) -{ -vmx_cache_reg (vcpu, reg); +void vt_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg) +{ +unsigned long guest_owned_bits; + +kvm_register_mark_available (vcpu, reg); + +switch (reg) { +case VCPU_REGS_RSP : + vcpu->arch.regs[VCPU_REGS_RSP ] =vmcs_readl (GUEST_RSP ); +break ; +case VCPU_REGS_RIP : + vcpu->arch.regs[VCPU_REGS_RIP ] =vmcs_readl (GUEST_RIP ); +break ; +case VCPU_EXREG_PDPTR : +if (enable_ept) +ept_save_pdptrs (vcpu); +break ; +case VCPU_EXREG_CR0 : + guest_owned_bits = vcpu->arch.cr0_guest_owned_bits; + + vcpu->arch.cr0 &= ~guest_owned_bits; + vcpu->arch.cr0 |=vmcs_readl (GUEST_CR0 ) & guest_owned_bits; +break ; +case VCPU_EXREG_CR3 : +/* +* When intercepting CR3 loads, e.g. for shadowing paging, KVM's +* CR3 is loaded into hardware, not the guest's CR3. +*/ +if (!(exec_controls_get (to_vmx (vcpu)) &CPU_BASED_CR3_LOAD_EXITING )) + vcpu->arch.cr3 =vmcs_readl (GUEST_CR3 ); +break ; +case VCPU_EXREG_CR4 : + guest_owned_bits = vcpu->arch.cr4_guest_owned_bits; + + vcpu->arch.cr4 &= ~guest_owned_bits; + vcpu->arch.cr4 |=vmcs_readl (GUEST_CR4 ) & guest_owned_bits; +break ; +default : +KVM_BUG_ON (1 , vcpu->kvm); +break ; + } }static unsigned long vt_get_rflags (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/vmx.c +1 -44
57 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h +1 -0
@@ -9,6 +9,7 @@#include "x86.h" extern struct kvm_x86_ops vt_x86_ops__initdata ; +void vt_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg);__init int vmx_disabled_by_bios (void );int __init vmx_check_processor_compat (void );
Signature · 1 line
2.25.1
RFC PATCH v3 48/59 KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
In reply to [email protected]From: Sean Christopherson <[email protected]>
Stub in kvm_tdx, vcpu_tdx, their various accessors, and VMCS helpers. The VMCS helpers, which rely on the stubs, will be used by preparatory patches to move VMX functions for accessing VMCS state to common code.
--- arch/x86/kvm/vmx/tdx.h | 169 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx.h
arch/x86/kvm/vmx/tdx.h new file +169 -0
@@ -0,0 +1,169 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_TDX_H +#define __KVM_X86_TDX_H + +#include <linux/list.h> +#include <linux/kvm_host.h> + +#include "tdx_errno.h" +#include "tdx_arch.h" +#include "tdx_ops.h" + +#ifdef CONFIG_INTEL_TDX_HOST + +struct tdx_td_page { +unsigned long va; +hpa_t pa; +bool added; +}; + +struct kvm_tdx { +struct kvm kvm; + +struct tdx_td_page tdr; +struct tdx_td_page tdcs[TDX_NR_TDCX_PAGES ]; +}; + +struct vcpu_tdx { +struct kvm_vcpu vcpu; + +struct tdx_td_page tdvpr; +struct tdx_td_page tdvpx[TDX_NR_TDVPX_PAGES ]; +}; + +static inline bool is_td (struct kvm *kvm) +{ +return kvm->arch.vm_type ==KVM_X86_TDX_VM ; +} + +static inline bool is_td_vcpu (struct kvm_vcpu *vcpu) +{ +return is_td (vcpu->kvm); +} + +static inline bool is_debug_td (struct kvm_vcpu *vcpu) +{ +return !vcpu->arch.guest_state_protected; +} + +static inline struct kvm_tdx *to_kvm_tdx (struct kvm *kvm) +{ + return container_of(kvm, struct kvm_tdx, kvm); +} + +static inline struct vcpu_tdx *to_tdx (struct kvm_vcpu *vcpu) +{ + return container_of(vcpu, struct vcpu_tdx, vcpu); +} + +static __always_inline void tdvps_vmcs_check(u32 field, u8 bits) +{ +BUILD_BUG_ON_MSG (__builtin_constant_p (field) && (field) &0x1 , +"Read/Write to TD VMCS *_HIGH fields not supported" ); + +BUILD_BUG_ON (bits !=16 && bits !=32 && bits !=64 ); + +BUILD_BUG_ON_MSG (bits !=64 &&__builtin_constant_p (field) && + (((field) &0x6000 ) ==0x2000 || + ((field) &0x6000 ) ==0x6000 ), +"Invalid TD VMCS access for 64-bit field" ); +BUILD_BUG_ON_MSG (bits !=32 &&__builtin_constant_p (field) && + ((field) &0x6000 ) ==0x4000 , +"Invalid TD VMCS access for 32-bit field" ); +BUILD_BUG_ON_MSG (bits !=16 &&__builtin_constant_p (field) && + ((field) &0x6000 ) ==0x0000 , +"Invalid TD VMCS access for 16-bit field" ); +} + +static __always_inline void tdvps_gpr_check(u64 field, u8 bits) +{ +BUILD_BUG_ON_MSG (__builtin_constant_p (field) && (field) >=NR_VCPU_REGS , +"Invalid TD guest GPR index" ); +} + +static __always_inline void tdvps_apic_check(u64 field, u8 bits) {} +static __always_inline void tdvps_dr_check(u64 field, u8 bits) {} +static __always_inline void tdvps_state_check(u64 field, u8 bits) {} +static __always_inline void tdvps_msr_check(u64 field, u8 bits) {} +static __always_inline void tdvps_management_check(u64 field, u8 bits) {} + +#define TDX_BUILD_TDVPS_ACCESSORS (bits, uclass, lclass) \ +static __always_inline u##bits td_##lclass##_read##bits(struct vcpu_tdx *tdx, \ + u32 field) \ +{ \ + struct tdx_ex_ret ex_ret; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_rd(tdx->tdvpr.pa, TDVPS_##uclass(field), &ex_ret); \ + if (unlikely(err)) { \ + pr_err("TDH_VP_RD["#uclass".0x%x] failed: 0x%llx\n", \ + field, err); \ + return 0; \ + } \ + return (u##bits)ex_ret.regs.r8; \ +} \ +static __always_inline void td_##lclass##_write##bits(struct vcpu_tdx *tdx, \ + u32 field, u##bits val) \ +{ \ + struct tdx_ex_ret ex_ret; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), val, \ + GENMASK_ULL(bits - 1, 0), &ex_ret); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] = 0x%llx failed: 0x%llx\n", \ + field, (u64)val, err); \ +} \ +static __always_inline void td_##lclass##_setbit##bits(struct vcpu_tdx *tdx, \ + u32 field, u64 bit) \ +{ \ + struct tdx_ex_ret ex_ret; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), bit, bit, \ + &ex_ret); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] |= 0x%llx failed: 0x%llx\n", \ + field, bit, err); \ +} \ +static __always_inline void td_##lclass##_clearbit##bits(struct vcpu_tdx *tdx, \ + u32 field, u64 bit) \ +{ \ + struct tdx_ex_ret ex_ret; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), 0, bit, \ + &ex_ret); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] &= ~0x%llx failed: 0x%llx\n", \ + field, bit, err); \ +} + +TDX_BUILD_TDVPS_ACCESSORS (16 ,VMCS , vmcs); +TDX_BUILD_TDVPS_ACCESSORS (32 ,VMCS , vmcs); +TDX_BUILD_TDVPS_ACCESSORS (64 ,VMCS , vmcs); + +TDX_BUILD_TDVPS_ACCESSORS (64 ,APIC , apic); +TDX_BUILD_TDVPS_ACCESSORS (64 ,GPR , gpr); +TDX_BUILD_TDVPS_ACCESSORS (64 ,DR , dr); +TDX_BUILD_TDVPS_ACCESSORS (64 ,STATE , state); +TDX_BUILD_TDVPS_ACCESSORS (64 ,MSR , msr); +TDX_BUILD_TDVPS_ACCESSORS (8 ,MANAGEMENT , management); + +#else +struct kvm_tdx ; +struct vcpu_tdx ; + +static inline bool is_td (struct kvm *kvm) {return false; } +static inline bool is_td_vcpu (struct kvm_vcpu *vcpu) {return false; } +static inline bool is_debug_td (struct kvm_vcpu *vcpu) {return false; } +static inline struct kvm_tdx *to_kvm_tdx (struct kvm *kvm) {return NULL ; } +static inline struct vcpu_tdx *to_tdx (struct kvm_vcpu *vcpu) {return NULL ; } + +#endif /* CONFIG_INTEL_TDX_HOST */ + +#endif /* __KVM_X86_TDX_H */
Signature · 1 line
2.25.1
RFC PATCH v3 28/59 KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
In reply to [email protected]From: Sean Christopherson <[email protected]>
Return true for kvm_vcpu_has_events() if the vCPU has a pending APICv interrupt to support TDX's usage of APICv. Unlike VMX, TDX doesn't have access to vmcs.GUEST_INTR_STATUS and so can't emulate posted interrupts, i.e. needs to generate a posted interrupt and more importantly can't manually move requested interrupts into the vIRR (which it also doesn't have access to).
--- arch/x86/kvm/x86.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +3 -1
@@ -11914,7 +11914,9 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu)if (kvm_arch_interrupt_allowed (vcpu) && (kvm_cpu_has_interrupt (vcpu) || -kvm_guest_apic_has_interrupt (vcpu))) +kvm_guest_apic_has_interrupt (vcpu) || + (vcpu->arch.apicv_active && + kvm_x86_ops.dy_apicv_has_pending_interrupt (vcpu))))return true;if (kvm_hv_has_stimer_pending (vcpu))
Signature · 1 line
2.25.1
RFC PATCH v3 52/59 KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX code
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/vmx/common.h | 14 ++++++++++++++ arch/x86/kvm/vmx/vmx.c | 10 +--------- 2 files changed, 15 insertions(+), 9 deletions(-)
arch/x86/kvm/vmx/common.h +14 -0
@@ -120,6 +120,20 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa,return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); } +static inline u32 __vmx_get_interrupt_shadow (struct kvm_vcpu *vcpu) +{ +u32 interruptibility; +int ret =0 ; + + interruptibility =vmread32 (vcpu,GUEST_INTERRUPTIBILITY_INFO ); +if (interruptibility &GUEST_INTR_STATE_STI ) + ret |=KVM_X86_SHADOW_INT_STI ; +if (interruptibility &GUEST_INTR_STATE_MOV_SS ) + ret |=KVM_X86_SHADOW_INT_MOV_SS ; + +return ret; +} +static inline u32 vmx_encode_ar_bytes (struct kvm_segment *var) {u32 ar;
arch/x86/kvm/vmx/vmx.c +1 -9
@@ -1365,15 +1365,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)u32 vmx_get_interrupt_shadow (struct kvm_vcpu *vcpu) { -u32 interruptibility =vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ); -int ret =0 ; - -if (interruptibility &GUEST_INTR_STATE_STI ) - ret |=KVM_X86_SHADOW_INT_STI ; -if (interruptibility &GUEST_INTR_STATE_MOV_SS ) - ret |=KVM_X86_SHADOW_INT_MOV_SS ; - -return ret; +return __vmx_get_interrupt_shadow (vcpu); }void vmx_set_interrupt_shadow (struct kvm_vcpu *vcpu,int mask)
Signature · 1 line
2.25.1
RFC PATCH v3 53/59 KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
In reply to [email protected]From: Chao Gao <[email protected]>
The TDX module unconditionally reset 4 host MSRs (MSR_SYSCALL_MASK, MSR_START, MSR_LSTAR, MSR_TSC_AUX) to architectural INIT state on exit from TDX VM to KVM. KVM needs to save their values before TD enter and restore them on exit to userspace.
Reuse current kvm_user_return mechanism and introduce a function to update cached values and register the user return notifier in this new function.
The later patch will use the helper function to save/restore 4 host MSRs.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 25 ++++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1910,6 +1910,7 @@ int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low,int kvm_add_user_return_msr (u32 msr);int kvm_find_user_return_msr (u32 msr);int kvm_set_user_return_msr (unsigned index,u64 val,u64 mask); +void kvm_user_return_update_cache (unsigned int index,u64 val);static inline bool kvm_is_supported_user_return_msr (u32 msr) {
arch/x86/kvm/x86.c +20 -5
@@ -418,6 +418,15 @@ static void kvm_user_return_msr_cpu_online(void) } } +static void kvm_user_return_register_notifier (struct kvm_user_return_msrs *msrs) +{ +if (!msrs->registered) { + msrs->urn.on_user_return = kvm_on_user_return; +user_return_notifier_register (&msrs->urn); + msrs->registered = true; + } +} +int kvm_set_user_return_msr (unsigned slot,u64 value,u64 mask) {unsigned int cpu =smp_processor_id ();
@@ -432,15 +441,21 @@ int kvm_set_user_return_msr(unsigned slot, u64 value, u64 mask)return 1 ; msrs->values[slot].curr = value; -if (!msrs->registered) { - msrs->urn.on_user_return = kvm_on_user_return; -user_return_notifier_register (&msrs->urn); - msrs->registered = true; - } +kvm_user_return_register_notifier (msrs);return 0 ; }EXPORT_SYMBOL_GPL (kvm_set_user_return_msr); +/* Update the cache, "curr", and register the notifier */ +void kvm_user_return_update_cache (unsigned int slot,u64 value) +{ +struct kvm_user_return_msrs *msrs =this_cpu_ptr (user_return_msrs); + + msrs->values[slot].curr = value; +kvm_user_return_register_notifier (msrs); +} +EXPORT_SYMBOL_GPL (kvm_user_return_update_cache); +static void drop_user_return_notifiers (void ) {unsigned int cpu =smp_processor_id ();
Signature · 1 line
2.25.1
RFC PATCH v3 51/59 KVM: VMX: MOVE GDT and IDT accessors to common code
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/vmx/main.c | 7 +++++-- arch/x86/kvm/vmx/vmx.c | 12 ------------ arch/x86/kvm/vmx/x86_ops.h | 2 -- 3 files changed, 5 insertions(+), 16 deletions(-)
arch/x86/kvm/vmx/main.c +5 -2
@@ -3,6 +3,7 @@#include "x86_ops.h" #include "vmx.h" +#include "common.h" #include "nested.h" #include "mmu.h" #include "pmu.h"
@@ -311,7 +312,8 @@ static int vt_set_efer(struct kvm_vcpu *vcpu, u64 efer)static void vt_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) { -vmx_get_idt (vcpu, dt); + dt->size =vmread32 (vcpu,GUEST_IDTR_LIMIT ); + dt->address =vmreadl (vcpu,GUEST_IDTR_BASE ); }static void vt_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt)
@@ -321,7 +323,8 @@ static void vt_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)static void vt_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) { -vmx_get_gdt (vcpu, dt); + dt->size =vmread32 (vcpu,GUEST_GDTR_LIMIT ); + dt->address =vmreadl (vcpu,GUEST_GDTR_BASE ); }static void vt_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt)
arch/x86/kvm/vmx/vmx.c +0 -12
@@ -3227,24 +3227,12 @@ void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) *l = (ar >>13 ) &1 ; } -void vmx_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) -{ - dt->size =vmcs_read32 (GUEST_IDTR_LIMIT ); - dt->address =vmcs_readl (GUEST_IDTR_BASE ); -} -void vmx_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) {vmcs_write32 (GUEST_IDTR_LIMIT , dt->size);vmcs_writel (GUEST_IDTR_BASE , dt->address); } -void vmx_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) -{ - dt->size =vmcs_read32 (GUEST_GDTR_LIMIT ); - dt->address =vmcs_readl (GUEST_GDTR_BASE ); -} -void vmx_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) {vmcs_write32 (GUEST_GDTR_LIMIT , dt->size);
arch/x86/kvm/vmx/x86_ops.h +0 -2
@@ -66,9 +66,7 @@ void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level);void vmx_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4);bool vmx_is_valid_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4);int vmx_set_efer (struct kvm_vcpu *vcpu,u64 efer); -void vmx_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt);void vmx_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); -void vmx_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt);void vmx_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt);void vmx_set_dr7 (struct kvm_vcpu *vcpu,unsigned long val);void vmx_sync_dirty_debug_regs (struct kvm_vcpu *vcpu);
Signature · 1 line
2.25.1
RFC PATCH v3 50/59 KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h
In reply to [email protected]From: Sean Christopherson <[email protected]>
Move the AR_BYTES helpers to common.h so that future patches can reuse them to decode/encode AR for TDX.
--- arch/x86/kvm/vmx/common.h | 41 ++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 47 ++++----------------------------------- 2 files changed, 45 insertions(+), 43 deletions(-)
arch/x86/kvm/vmx/common.h +41 -0
@@ -4,6 +4,7 @@#include <linux/kvm_host.h> +#include <asm/kvm.h> #include <asm/traps.h> #include <asm/vmx.h>
@@ -119,4 +120,44 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa,return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); } +static inline u32 vmx_encode_ar_bytes (struct kvm_segment *var) +{ +u32 ar; + +if (var->unusable || !var->present) + ar =1 <<16 ; +else { + ar = var->type &15 ; + ar |= (var->s &1 ) <<4 ; + ar |= (var->dpl &3 ) <<5 ; + ar |= (var->present &1 ) <<7 ; + ar |= (var->avl &1 ) <<12 ; + ar |= (var->l &1 ) <<13 ; + ar |= (var->db &1 ) <<14 ; + ar |= (var->g &1 ) <<15 ; + } + +return ar; +} + +static inline void vmx_decode_ar_bytes (u32 ar,struct kvm_segment *var) +{ + var->unusable = (ar >>16 ) &1 ; + var->type = ar &15 ; + var->s = (ar >>4 ) &1 ; + var->dpl = (ar >>5 ) &3 ; +/* +* Some userspaces do not preserve unusable property. Since usable +* segment has to be present according to VMX spec we can use present +* property to amend userspace bug by making unusable segment always +* nonpresent. vmx_encode_ar_bytes() already marks nonpresent +* segment as unusable. +*/ + var->present = !var->unusable; + var->avl = (ar >>12 ) &1 ; + var->l = (ar >>13 ) &1 ; + var->db = (ar >>14 ) &1 ; + var->g = (ar >>15 ) &1 ; +} +#endif /* __KVM_X86_VMX_COMMON_H */
arch/x86/kvm/vmx/vmx.c +4 -43
89 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v3 49/59 KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a macro framework to hide VMX vs. TDX details of VMREAD and VMWRITE so the VMX and TDX can shared common flows, e.g. accessing DTs.
Note, the TDX paths are dead code at this time. There is no great way to deal with the chicken-and-egg scenario of having things in place for TDX without first having TDX.
--- arch/x86/kvm/vmx/common.h | 41 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+)
arch/x86/kvm/vmx/common.h +41 -0
@@ -11,6 +11,47 @@#include "vmcs.h" #include "vmx.h" #include "x86.h" +#include "tdx.h" + +#ifdef CONFIG_INTEL_TDX_HOST +#define VT_BUILD_VMCS_HELPERS (type, bits, tdbits) \ +static __always_inline type vmread##bits(struct kvm_vcpu *vcpu, \ + unsigned long field) \ +{ \ + if (unlikely(is_td_vcpu(vcpu))) { \ + if (KVM_BUG_ON(!is_debug_td(vcpu), vcpu->kvm)) \ + return 0; \ + return td_vmcs_read##tdbits(to_tdx(vcpu), field); \ + } \ + return vmcs_read##bits(field); \ +} \ +static __always_inline void vmwrite##bits(struct kvm_vcpu *vcpu, \ + unsigned long field, type value) \ +{ \ + if (unlikely(is_td_vcpu(vcpu))) { \ + if (KVM_BUG_ON(!is_debug_td(vcpu), vcpu->kvm)) \ + return; \ + return td_vmcs_write##tdbits(to_tdx(vcpu), field, value); \ + } \ + vmcs_write##bits(field, value); \ +} +#else +#define VT_BUILD_VMCS_HELPERS (type, bits, tdbits) \ +static __always_inline type vmread##bits(struct kvm_vcpu *vcpu, \ + unsigned long field) \ +{ \ + return vmcs_read##bits(field); \ +} \ +static __always_inline void vmwrite##bits(struct kvm_vcpu *vcpu, \ + unsigned long field, type value) \ +{ \ + vmcs_write##bits(field, value); \ +} +#endif /* CONFIG_INTEL_TDX_HOST */ +VT_BUILD_VMCS_HELPERS (u16,16 ,16 ); +VT_BUILD_VMCS_HELPERS (u32,32 ,32 ); +VT_BUILD_VMCS_HELPERS (u64,64 ,64 ); +VT_BUILD_VMCS_HELPERS(unsigned long, l, 64);extern unsigned long vmx_host_idt_base;void vmx_do_interrupt_nmi_irqoff (unsigned long entry);
Signature · 1 line
2.25.1
RFC PATCH v3 24/59 KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add hooks to cache and flush GPRs and invoke them from KVM_GET_REGS and KVM_SET_REGS respecitively. TDX will use the hooks to read/write GPRs from TDX-SEAM on-demand (for debug TDs).
--- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 4 ++++ 3 files changed, 8 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +2 -0
@@ -45,6 +45,8 @@ KVM_X86_OP(get_gdt) KVM_X86_OP(set_gdt) KVM_X86_OP(sync_dirty_debug_regs) KVM_X86_OP(set_dr7) +KVM_X86_OP_NULL(cache_gprs) +KVM_X86_OP_NULL(flush_gprs) KVM_X86_OP(cache_reg) KVM_X86_OP(get_rflags) KVM_X86_OP(set_rflags)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1352,6 +1352,8 @@ struct kvm_x86_ops {void (*set_gdt)(struct kvm_vcpu *vcpu,struct desc_ptr *dt);void (*sync_dirty_debug_regs)(struct kvm_vcpu *vcpu);void (*set_dr7)(struct kvm_vcpu *vcpu,unsigned long value); +void (*cache_gprs)(struct kvm_vcpu *vcpu); +void (*flush_gprs)(struct kvm_vcpu *vcpu);void (*cache_reg)(struct kvm_vcpu *vcpu,enum kvm_reg reg);unsigned long (*get_rflags)(struct kvm_vcpu *vcpu);void (*set_rflags)(struct kvm_vcpu *vcpu,unsigned long rflags);
arch/x86/kvm/x86.c +4 -0
@@ -10293,6 +10293,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)static void __get_regs (struct kvm_vcpu *vcpu,struct kvm_regs *regs) { +static_call_cond (kvm_x86_cache_gprs)(vcpu); +if (vcpu->arch.emulate_regs_need_sync_to_vcpu) {/* * We are here if userspace calls get_regs() in the middle of
@@ -10367,6 +10369,8 @@ static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) vcpu->arch.exception.pending = false; +static_call_cond (kvm_x86_flush_gprs); +kvm_make_request (KVM_REQ_EVENT , vcpu); }
Signature · 1 line
2.25.1
RFC PATCH v3 25/59 KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 12 ++++++++++++ 2 files changed, 14 insertions(+)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1483,7 +1483,9 @@ struct kvm_x86_ops {int (*leave_smm)(struct kvm_vcpu *vcpu,const char *smstate);void (*enable_smi_window)(struct kvm_vcpu *vcpu); +int (*mem_enc_op_dev)(void __user *argp);int (*mem_enc_op)(struct kvm *kvm,void __user *argp); +int (*mem_enc_op_vcpu)(struct kvm_vcpu *vcpu,void __user *argp);int (*mem_enc_reg_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*mem_enc_unreg_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*vm_copy_enc_context_from)(struct kvm *kvm,unsigned int source_fd);
arch/x86/kvm/x86.c +12 -0
@@ -4339,6 +4339,12 @@ long kvm_arch_dev_ioctl(struct file *filp,case KVM_GET_SUPPORTED_HV_CPUID : r =kvm_ioctl_get_supported_hv_cpuid (NULL , argp);break ; +case KVM_MEMORY_ENCRYPT_OP : + r = -EINVAL ; +if (!kvm_x86_ops.mem_enc_op_dev) + goto out; + r = kvm_x86_ops.mem_enc_op_dev (argp); +break ;default : r = -EINVAL ;break ;
@@ -5516,6 +5522,12 @@ long kvm_arch_vcpu_ioctl(struct file *filp,case KVM_SET_DEVICE_ATTR : r =kvm_vcpu_ioctl_device_attr (vcpu, ioctl, argp);break ; +case KVM_MEMORY_ENCRYPT_OP : + r = -EINVAL ; +if (!kvm_x86_ops.mem_enc_op_vcpu) + goto out; + r = kvm_x86_ops.mem_enc_op_vcpu (vcpu, argp); +break ;default : r = -EINVAL ; }
Signature · 1 line
2.25.1
RFC PATCH v3 32/59 KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
In reply to [email protected]From: Sean Christopherson <[email protected]>
Explicity check for an MMIO spte in the fast page fault flow. TDX will use a not-present entry for MMIO sptes, which can be mistaken for an access-tracked spte since both have SPTE_SPECIAL_MASK set.
--- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +1 -1
@@ -3186,7 +3186,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)break ; sp =sptep_to_sp (sptep); -if (!is_last_spte (spte, sp->role.level)) +if (!is_last_spte (spte, sp->role.level) ||is_mmio_spte (spte))break ;/*
Signature · 1 line
2.25.1
RFC PATCH v3 34/59 KVM: x86/mmu: Allow non-zero init value for shadow PTE
In reply to [email protected]From: Sean Christopherson <[email protected]>
TDX will run with EPT violation #VEs enabled, which means KVM needs to set the "suppress #VE" bit in unused PTEs to avoid unintentionally reflecting not-present EPT violations into the guest.
--- arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 50 +++++++++++++++++++++++++++++++++++------ arch/x86/kvm/mmu/spte.c | 10 +++++++++ arch/x86/kvm/mmu/spte.h | 2 ++ 4 files changed, 56 insertions(+), 7 deletions(-)
arch/x86/kvm/mmu.h +1 -0
@@ -66,6 +66,7 @@ static __always_inline u64 rsvd_bits(int s, int e)void kvm_mmu_set_mmio_spte_mask (u64 mmio_value,u64 mmio_mask,u64 access_mask);void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only); +void kvm_mmu_set_spte_init_value (u64 init_value);void kvm_init_mmu (struct kvm_vcpu *vcpu); void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr0,
arch/x86/kvm/mmu/mmu.c +43 -7
87 lines not on this page · raw message
arch/x86/kvm/mmu/spte.c +10 -0
@@ -34,6 +34,7 @@ u64 __read_mostly shadow_mmio_access_mask;u64 __read_mostly shadow_present_mask;u64 __read_mostly shadow_me_mask;u64 __read_mostly shadow_acc_track_mask; +u64 __read_mostly shadow_init_value;u64 __read_mostly shadow_nonpresent_or_rsvd_mask;u64 __read_mostly shadow_nonpresent_or_rsvd_lower_gfn_mask;
@@ -221,6 +222,14 @@ u64 kvm_mmu_changed_pte_notifier_make_spte(u64 old_spte, kvm_pfn_t new_pfn)return new_spte; } +void kvm_mmu_set_spte_init_value (u64 init_value) +{ +if (WARN_ON (!IS_ENABLED (CONFIG_X86_64 ) && init_value)) + init_value =0 ; + shadow_init_value = init_value; +} +EXPORT_SYMBOL_GPL (kvm_mmu_set_spte_init_value); +static u8 kvm_get_shadow_phys_bits (void ) {/*
@@ -365,6 +374,7 @@ void kvm_mmu_reset_all_pte_masks(void) shadow_present_mask =PT_PRESENT_MASK ; shadow_acc_track_mask =0 ; shadow_me_mask = sme_me_mask; + shadow_init_value =0 ; shadow_host_writable_mask =DEFAULT_SPTE_HOST_WRITEABLE ; shadow_mmu_writable_mask =DEFAULT_SPTE_MMU_WRITEABLE ;
arch/x86/kvm/mmu/spte.h +2 -0
@@ -146,6 +146,8 @@ extern u64 __read_mostly shadow_mmio_access_mask;extern u64 __read_mostly shadow_present_mask;extern u64 __read_mostly shadow_me_mask; +extern u64 __read_mostly shadow_init_value; +/* * SPTEs in MMUs without A/D bits are marked with SPTE_TDP_AD_DISABLED_MASK; * shadow_acc_track_mask is the set of bits to be cleared in non-accessed
Signature · 1 line
2.25.1
RFC PATCH v3 35/59 KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()
In reply to [email protected]From: Sean Christopherson <[email protected]>
Return the old SPTE when clearing a SPTE and push the "old SPTE present" check to the caller. Private shadow page support will use the old SPTE in rmap_remove() to determine whether or not there is a linked private shadow page.
--- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +1 -1
@@ -616,7 +616,7 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte)* state bits, it is used to clear the last level sptep. * Returns the old PTE. */ -static int mmu_spte_clear_track_bits (struct kvm *kvm,u64 *sptep) +static u64 mmu_spte_clear_track_bits (struct kvm *kvm,u64 *sptep) {kvm_pfn_t pfn;u64 old_spte = *sptep;
Signature · 1 line
2.25.1
RFC PATCH v3 30/59 KVM: x86: Add guest_supported_xss placholder
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a per-vcpu placeholder for the support XSS of the guest so that the TDX configuration code doesn't need to hack in manual computation of the supported XSS. KVM XSS enabling is currently being upstreamed, i.e. guest_supported_xss will no longer be a placeholder by the time TDX is ready for upstreaming (hopefully).
--- arch/x86/include/asm/kvm_host.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -701,6 +701,7 @@ struct kvm_vcpu_arch {u64 xcr0;u64 guest_supported_xcr0; +u64 guest_supported_xss;struct kvm_pio_request pio;void *pio_data;
Signature · 1 line
2.25.1
RFC PATCH v3 57/59 KVM, x86/mmu: Support TDX private mapping for TDP MMU
In reply to [email protected]From: Kai Huang <[email protected]>
Frame in TDX private mapping support to support running TD with TDP MMU.
Similar to legacy MMU, use private mapping related kvm_x86_ops hooks in __handle_changed_spte()/handle_removed_tdp_mmu_page() to support creating/removing private mapping. And support temporarily blocking private mapping upon receiving MMU notifier and later on unblocking it upon EPT violation, rather than completely removing the private page, because currently page migration for private page is not supported, therefore the page cannot be really removed from TD upon MMU notifier.
Similar to legacy MMU, zap aliasing mapping (truly remove the page) upon EPT violation. And only leaf page is zapped, but not intermediate page tables.
Support 4K page only at this stage. 2M page support can be done in future patches.
A key change to TDP MMU to support TD guest is, the read_lock() is changed to write_lock() for page fault on private GPA in direct_page_fault(), while for fault on shared GPA, read_lock() is still used. This is because for TD guest, at given time for given GFN, only one type of mapping (either private, or shared) can be valid, but not both, otherwise it may cause machine check and data lose. As a result, aliasing mapping is zapped in TDP MMU fault handler. In this case, running multiple fault threads with both private and shared address concurrently may end up with having both private and shared mapping for given GPN. Consider below case: vcpu 0 is accessing using private GPA, and vcpu 1 is accessing the shared GPA (i.e. right after MAP_GPA).
vcpu 0 vcpu 1
(fault with private GPA) (fault with shared GPA)
zap shared mapping
zap private mapping setup shared mapping
setup private mapping
This may end up with having both private and shared mappings. Perhaps it is arguable whether above case is valid, but for security, just don't allow running multiple fault threads concurrently.
--- arch/x86/kvm/mmu/mmu.c | 63 ++++- arch/x86/kvm/mmu/spte.h | 20 +- arch/x86/kvm/mmu/tdp_iter.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 544 ++++++++++++++++++++++++++++++++---- arch/x86/kvm/mmu/tdp_mmu.h | 15 +- 5 files changed, 567 insertions(+), 77 deletions(-)
arch/x86/kvm/mmu/mmu.c +57 -6
@@ -3692,7 +3692,11 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu) goto out_unlock;if (is_tdp_mmu_enabled (vcpu->kvm)) { - root =kvm_tdp_mmu_get_vcpu_root_hpa (vcpu); +if (gfn_shared && !VALID_PAGE (mmu->private_root_hpa)) { + root =kvm_tdp_mmu_get_vcpu_root_hpa (vcpu, true); + mmu->private_root_hpa = root; + } + root =kvm_tdp_mmu_get_vcpu_root_hpa (vcpu, false); mmu->root_hpa = root; }else if (shadow_root_level >=PT64_ROOT_4LEVEL ) {if (gfn_shared && !VALID_PAGE (vcpu->arch.mmu->private_root_hpa)) {
@@ -4348,7 +4352,33 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault r =RET_PF_RETRY ; -if (is_tdp_mmu_fault) +/* +* Unfortunately, when running TD, TDP MMU cannot always run multiple +* fault threads concurrently. TDX has two page tables supporting +* private and shared mapping simultaneously, but at given time, only +* one mapping can be valid for given GFN, otherwise it may cause +* machine check and data lose. Therefore, TDP MMU fault handler zaps +* aliasing mapping. +* +* Running fault threads for both private and shared GPA concurrently +* can potentially end up with having both private and shared mapping +* for one GPA. For instance, vcpu 0 is accessing using private GPA, +* and vcpu 1 is accessing using shared GPA (i.e. right after MAP_GPA): +* +* vcpu 0 vcpu 1 +* (fault with private GPA) (fault with shared GPA) +* +* zap shared mapping +* zap priavte mapping +* setup shared mapping +* setup private mapping +* +* This can be prevented by only allowing one type of fault (private +* or shared) to run concurrently. Choose to let private fault to run +* concurrently, because for TD, most pages should be private. +*/ +if (is_tdp_mmu_fault &&kvm_is_private_gfn (vcpu->kvm, + fault->addr >>PAGE_SHIFT ))read_lock (&vcpu->kvm->mmu_lock);else write_lock (&vcpu->kvm->mmu_lock);
@@ -4365,7 +4395,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault r =__direct_map (vcpu, fault); out_unlock: -if (is_tdp_mmu_fault) +if (is_tdp_mmu_fault &&kvm_is_private_gfn (vcpu->kvm, + fault->addr >>PAGE_SHIFT ))read_unlock (&vcpu->kvm->mmu_lock);else write_unlock (&vcpu->kvm->mmu_lock);
@@ -6057,6 +6088,10 @@ static void kvm_mmu_zap_all_fast(struct kvm *kvm)write_unlock (&kvm->mmu_lock); +/* +* For now private root is never invalidate during VM is running, +* so this can only happen for shared roots. +*/ if (is_tdp_mmu_enabled (kvm)) {read_lock (&kvm->mmu_lock);kvm_tdp_mmu_zap_invalidated_roots (kvm);
@@ -6162,7 +6197,8 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)if (is_tdp_mmu_enabled (kvm)) {for (i =0 ; i <KVM_ADDRESS_SPACE_NUM ; i++) flush =kvm_tdp_mmu_zap_gfn_range (kvm, i, gfn_start, - gfn_end, flush); + gfn_end, flush, + false); }if (flush)
@@ -6195,6 +6231,11 @@ void kvm_mmu_slot_remove_write_access(struct kvm *kvm,write_unlock (&kvm->mmu_lock); } +/* +* For now this can only happen for non-TD VM, because TD private +* mapping doesn't support write protection. kvm_tdp_mmu_wrprot_slot() +* will give a WARN() if it hits for TD. +*/ if (is_tdp_mmu_enabled (kvm)) {read_lock (&kvm->mmu_lock); flush |=kvm_tdp_mmu_wrprot_slot (kvm, memslot, start_level);
@@ -6277,6 +6318,11 @@ void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm,write_unlock (&kvm->mmu_lock); } +/* +* This should only be reachable in case of log-dirty, wihch TD private +* mapping doesn't support so far. kvm_tdp_mmu_zap_collapsible_sptes() +* internally gives a WARN() when it hits. +*/ if (is_tdp_mmu_enabled (kvm)) {read_lock (&kvm->mmu_lock); flush =kvm_tdp_mmu_zap_collapsible_sptes (kvm, slot, flush);
@@ -6316,6 +6362,7 @@ void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm,write_unlock (&kvm->mmu_lock); } +/* See comments in kvm_mmu_slot_remove_write_access() */ if (is_tdp_mmu_enabled (kvm)) {read_lock (&kvm->mmu_lock); flush |=kvm_tdp_mmu_clear_dirty_slot (kvm, memslot);
@@ -6350,8 +6397,12 @@ static void __kvm_mmu_zap_all(struct kvm *kvm, struct list_head *mmu_pages) }kvm_mmu_commit_zap_page (kvm, &invalid_list); -if (is_tdp_mmu_enabled (kvm)) -kvm_tdp_mmu_zap_all (kvm); +if (is_tdp_mmu_enabled (kvm)) { +bool zap_private = + (mmu_pages == &kvm->arch.private_mmu_pages) ? + true : false; +kvm_tdp_mmu_zap_all (kvm, zap_private); + }write_unlock (&kvm->mmu_lock); }
arch/x86/kvm/mmu/spte.h +17 -3
@@ -173,7 +173,9 @@ extern u64 __read_mostly shadow_nonpresent_or_rsvd_mask;* If a thread running without exclusive control of the MMU lock must perform a * multi-part operation on an SPTE, it can set the SPTE to REMOVED_SPTE as a * non-present intermediate value. Other threads which encounter this value -* should not modify the SPTE. +* should not modify the SPTE. When TDX is enabled, shadow_init_value, which +* is "suppress #VE" bit set, is also set to removed SPTE, because TDX module +* always enables "EPT violation #VE". * * Use a semi-arbitrary value that doesn't set RWX bits, i.e. is not-present on * bot AMD and Intel CPUs, and doesn't set PFN bits, i.e. doesn't create a L1TF
@@ -183,12 +185,24 @@ extern u64 __read_mostly shadow_nonpresent_or_rsvd_mask;*/ #define REMOVED_SPTE 0x5a0ULL -/* Removed SPTEs must not be misconstrued as shadow present PTEs. */ +/* +* Removed SPTEs must not be misconstrued as shadow present PTEs, and +* temporarily blocked private PTEs. +*/ static_assert (!(REMOVED_SPTE &SPTE_MMU_PRESENT_MASK )); +static_assert (!(REMOVED_SPTE &SPTE_PRIVATE_ZAPPED )); + +/* +* See above comment around REMOVED_SPTE. SHADOW_REMOVED_SPTE is the actual +* intermediate value set to the removed SPET. When TDX is enabled, it sets +* the "suppress #VE" bit, otherwise it's REMOVED_SPTE. +*/ +extern u64 __read_mostly shadow_init_value; +#define SHADOW_REMOVED_SPTE (shadow_init_value | REMOVED_SPTE)static inline bool is_removed_spte (u64 spte) { -return spte ==REMOVED_SPTE ; +return spte ==SHADOW_REMOVED_SPTE ; }/*
arch/x86/kvm/mmu/tdp_iter.h +1 -1
@@ -28,7 +28,7 @@ struct tdp_iter {tdp_ptep_t pt_path[PT64_ROOT_MAX_LEVEL ];/* A pointer to the current SPTE */ tdp_ptep_t sptep; -/* The lowest GFN mapped by the current SPTE */ +/* The lowest GFN (stolen bits included) mapped by the current SPTE */ gfn_t gfn;/* The level of the root page given to the iterator */ int root_level;
arch/x86/kvm/mmu/tdp_mmu.c +483 -61
972 lines not on this page · raw message
arch/x86/kvm/mmu/tdp_mmu.h +9 -6
@@ -5,7 +5,7 @@#include <linux/kvm_host.h> -hpa_t kvm_tdp_mmu_get_vcpu_root_hpa (struct kvm_vcpu *vcpu); +hpa_t kvm_tdp_mmu_get_vcpu_root_hpa (struct kvm_vcpu *vcpu,bool private);__must_check static inline bool kvm_tdp_mmu_get_root (struct kvm *kvm,struct kvm_mmu_page *root)
@@ -20,11 +20,14 @@ void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root, bool shared);bool __kvm_tdp_mmu_zap_gfn_range (struct kvm *kvm,int as_id,gfn_t start, -gfn_t end,bool can_yield,bool flush); +gfn_t end,bool can_yield,bool flush, +bool zap_private);static inline bool kvm_tdp_mmu_zap_gfn_range (struct kvm *kvm,int as_id, -gfn_t start,gfn_t end,bool flush) +gfn_t start,gfn_t end,bool flush, +bool zap_private) { -return __kvm_tdp_mmu_zap_gfn_range (kvm, as_id, start, end, true, flush); +return __kvm_tdp_mmu_zap_gfn_range (kvm, as_id, start, end, true, flush, + zap_private); }static inline bool kvm_tdp_mmu_zap_sp (struct kvm *kvm,struct kvm_mmu_page *sp) {
@@ -41,10 +44,10 @@ static inline bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp)*/ lockdep_assert_held_write (&kvm->mmu_lock);return __kvm_tdp_mmu_zap_gfn_range (kvm,kvm_mmu_page_as_id (sp), - sp->gfn, end, false, false); + sp->gfn, end, false, false, false); } -void kvm_tdp_mmu_zap_all (struct kvm *kvm); +void kvm_tdp_mmu_zap_all (struct kvm *kvm,bool zap_private);void kvm_tdp_mmu_invalidate_all_roots (struct kvm *kvm);void kvm_tdp_mmu_zap_invalidated_roots (struct kvm *kvm);
Signature · 1 line
2.25.1
RFC PATCH v3 56/59 KVM: TDX: Protect private mapping related SEAMCALLs with spinlock
In reply to [email protected]From: Kai Huang <[email protected]>
Unlike legacy MMU, TDP MMU runs page fault handler concurrently. However, TDX private mapping related SEAMCALLs cannot run concurrently even they operate on different pages, because they need to acquire exclusive access to secure EPT table. Protect those SEAMCALLs with spinlock, so they won't run concurrently even under TDP MMU.
--- arch/x86/kvm/vmx/tdx.c | 111 +++++++++++++++++++++++++++++++++++++---- arch/x86/kvm/vmx/tdx.h | 7 +++ 2 files changed, 108 insertions(+), 10 deletions(-)
arch/x86/kvm/vmx/tdx.c +101 -10
@@ -519,6 +519,8 @@ int tdx_vm_init(struct kvm *kvm)tdx_add_td_page (&kvm_tdx->tdcs[i]); } +spin_lock_init (&kvm_tdx->seamcall_lock); +/* * Note, TDH_MNG_INIT cannot be invoked here. TDH_MNG_INIT requires a dedicated * ioctl() to define the configure CPUID values for the TD.
@@ -1187,8 +1189,8 @@ static void tdx_measure_page(struct kvm_tdx *kvm_tdx, hpa_t gpa) } } -static void tdx_sept_set_private_spte (struct kvm *kvm,gfn_t gfn, -enum pg_level level,kvm_pfn_t pfn) +static void __tdx_sept_set_private_spte (struct kvm *kvm,gfn_t gfn, +enum pg_level level,kvm_pfn_t pfn) {struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm);hpa_t hpa = pfn <<PAGE_SHIFT ;
@@ -1215,6 +1217,15 @@ static void tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn,return ; } +/* +* In case of TDP MMU, fault handler can run concurrently. Note +* 'source_pa' is a TD scope variable, meaning if there are multiple +* threads reaching here with all needing to access 'source_pa', it +* will break. However fortunately this won't happen, because below +* TDH_MEM_PAGE_ADD code path is only used when VM is being created +* before it is running, using KVM_TDX_INIT_MEM_REGION ioctl (which +* always uses vcpu 0's page table and protected by vcpu->mutex). +*/ WARN_ON (kvm_tdx->source_pa ==INVALID_PAGE ); source_pa = kvm_tdx->source_pa & ~KVM_TDX_MEASURE_MEMORY_REGION ;
@@ -1227,8 +1238,25 @@ static void tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn, kvm_tdx->source_pa =INVALID_PAGE ; } -static void tdx_sept_drop_private_spte (struct kvm *kvm,gfn_t gfn,enum pg_level level, -kvm_pfn_t pfn) +static void tdx_sept_set_private_spte (struct kvm *kvm,gfn_t gfn, +enum pg_level level,kvm_pfn_t pfn) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); + +/* +* Only TDP MMU needs to use spinlock, however for simplicity, +* just always use spinlock for seamcall, regardless whether +* legacy MMU or TDP MMU is being used. For legacy MMU it +* should not have noticeable performance impact since taking +* spinlock w/o needing to wait should be fast. +*/ +spin_lock (&kvm_tdx->seamcall_lock); +__tdx_sept_set_private_spte (kvm, gfn, level, pfn); +spin_unlock (&kvm_tdx->seamcall_lock); +} + +static void __tdx_sept_drop_private_spte (struct kvm *kvm,gfn_t gfn,enum pg_level level, +kvm_pfn_t pfn) {int tdx_level =pg_level_to_tdx_sept_level (level);struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm);
@@ -1262,8 +1290,19 @@ static void tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_levelput_page (pfn_to_page (pfn)); } -static int tdx_sept_link_private_sp (struct kvm *kvm,gfn_t gfn, -enum pg_level level,void *sept_page) +static void tdx_sept_drop_private_spte (struct kvm *kvm,gfn_t gfn,enum pg_level level, +kvm_pfn_t pfn) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); + +/* See comment in tdx_sept_set_private_spte() */ +spin_lock (&kvm_tdx->seamcall_lock); +__tdx_sept_drop_private_spte (kvm, gfn, level, pfn); +spin_unlock (&kvm_tdx->seamcall_lock); +} + +static int __tdx_sept_link_private_sp (struct kvm *kvm,gfn_t gfn, +enum pg_level level,void *sept_page) {int tdx_level =pg_level_to_tdx_sept_level (level);struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm);
@@ -1281,7 +1320,22 @@ static int tdx_sept_link_private_sp(struct kvm *kvm, gfn_t gfn,return 0 ; } -static void tdx_sept_zap_private_spte (struct kvm *kvm,gfn_t gfn,enum pg_level level) +static int tdx_sept_link_private_sp (struct kvm *kvm,gfn_t gfn, +enum pg_level level,void *sept_page) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +int ret; + +/* See comment in tdx_sept_set_private_spte() */ +spin_lock (&kvm_tdx->seamcall_lock); + ret =__tdx_sept_link_private_sp (kvm, gfn, level, sept_page); +spin_unlock (&kvm_tdx->seamcall_lock); + +return ret; +} + +static void __tdx_sept_zap_private_spte (struct kvm *kvm,gfn_t gfn, +enum pg_level level) {int tdx_level =pg_level_to_tdx_sept_level (level);struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm);
@@ -1294,7 +1348,19 @@ static void tdx_sept_zap_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_levelpr_tdx_error (TDH_MEM_RANGE_BLOCK , err, &ex_ret); } -static void tdx_sept_unzap_private_spte (struct kvm *kvm,gfn_t gfn,enum pg_level level) +static void tdx_sept_zap_private_spte (struct kvm *kvm,gfn_t gfn, +enum pg_level level) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); + +/* See comment in tdx_sept_set_private_spte() */ +spin_lock (&kvm_tdx->seamcall_lock); +__tdx_sept_zap_private_spte (kvm, gfn, level); +spin_unlock (&kvm_tdx->seamcall_lock); +} + +static void __tdx_sept_unzap_private_spte (struct kvm *kvm,gfn_t gfn, +enum pg_level level) {int tdx_level =pg_level_to_tdx_sept_level (level);struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm);
@@ -1307,8 +1373,19 @@ static void tdx_sept_unzap_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_levepr_tdx_error (TDH_MEM_RANGE_UNBLOCK , err, &ex_ret); } -static int tdx_sept_free_private_sp (struct kvm *kvm,gfn_t gfn,enum pg_level level, -void *sept_page) +static void tdx_sept_unzap_private_spte (struct kvm *kvm,gfn_t gfn, +enum pg_level level) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); + +/* See comment in tdx_sept_set_private_spte() */ +spin_lock (&kvm_tdx->seamcall_lock); +__tdx_sept_unzap_private_spte (kvm, gfn, level); +spin_unlock (&kvm_tdx->seamcall_lock); +} + +static int __tdx_sept_free_private_sp (struct kvm *kvm,gfn_t gfn,enum pg_level level, +void *sept_page) {/* * free_private_sp() is (obviously) called when a shadow page is being
@@ -1320,6 +1397,20 @@ static int tdx_sept_free_private_sp(struct kvm *kvm, gfn_t gfn, enum pg_level lereturn tdx_reclaim_page ((unsigned long )sept_page,__pa (sept_page)); } +static int tdx_sept_free_private_sp (struct kvm *kvm,gfn_t gfn,enum pg_level level, +void *sept_page) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +int ret; + +/* See comment in tdx_sept_set_private_spte() */ +spin_lock (&kvm_tdx->seamcall_lock); + ret =__tdx_sept_free_private_sp (kvm, gfn, level, sept_page); +spin_unlock (&kvm_tdx->seamcall_lock); + +return ret; +} +static int tdx_sept_tlb_remote_flush (struct kvm *kvm) {struct kvm_tdx *kvm_tdx;
arch/x86/kvm/vmx/tdx.h +7 -0
@@ -37,6 +37,13 @@ struct kvm_tdx {hpa_t source_pa;u64 tsc_offset; + +/* +* Lock to prevent seamcalls from running concurrently +* when TDP MMU is enabled, because TDP fault handler +* runs concurrently. +*/ +spinlock_t seamcall_lock; };union tdx_exit_reason {
Signature · 1 line
2.25.1
RFC PATCH v3 55/59 KVM: TDX: Add "basic" support for building and running Trust Domains
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add what is effectively a TDX-specific ioctl for initializing the guest Trust Domain. Implement the functionality as a subcommand of KVM_MEMORY_ENCRYPT_OP, analogous to how the ioctl is used by SVM to manage SEV guests.
For easy compatibility with future versions of TDX-SEAM, add a KVM-defined struct, tdx_capabilities, to track requirements/capabilities for the overall system, and define a global instance to serve as the canonical reference.
--- arch/x86/events/intel/ds.c | 1 + arch/x86/include/uapi/asm/kvm.h | 56 ++++ arch/x86/include/uapi/asm/vmx.h | 3 +- arch/x86/kvm/Makefile | 5 +- arch/x86/kvm/mmu.h | 2 +- arch/x86/kvm/mmu/mmu.c | 1 + arch/x86/kvm/mmu/spte.c | 5 +- arch/x86/kvm/vmx/common.h | 1 + arch/x86/kvm/vmx/main.c | 403 +++++++++++++++++++++++++- arch/x86/kvm/vmx/posted_intr.c | 6 + arch/x86/kvm/vmx/tdx.h | 114 ++++++++ arch/x86/kvm/vmx/tdx_arch.h | 24 +- arch/x86/kvm/vmx/tdx_ops.h | 14 + arch/x86/kvm/vmx/tdx_stubs.c | 50 ++++ arch/x86/kvm/vmx/vmenter.S | 146 ++++++++++ arch/x86/kvm/vmx/vmx.c | 39 --- arch/x86/kvm/vmx/x86_ops.h | 80 +++++ arch/x86/kvm/x86.c | 8 +- tools/arch/x86/include/uapi/asm/kvm.h | 51 ++++ 19 files changed, 946 insertions(+), 63 deletions(-) create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c
arch/x86/events/intel/ds.c +1 -0
@@ -2248,3 +2248,4 @@ void perf_restore_debug_store(void)wrmsrl (MSR_IA32_DS_AREA , (unsigned long )ds); } +EXPORT_SYMBOL_GPL (perf_restore_debug_store);
arch/x86/include/uapi/asm/kvm.h +56 -0
60 lines not on this page · raw message
arch/x86/include/uapi/asm/vmx.h +2 -1
@@ -32,8 +32,9 @@#define EXIT_REASON_EXCEPTION_NMI 0#define EXIT_REASON_EXTERNAL_INTERRUPT 1#define EXIT_REASON_TRIPLE_FAULT 2 -#define EXIT_REASON_INIT_SIGNAL 3 +#define EXIT_REASON_INIT_SIGNAL 3#define EXIT_REASON_SIPI_SIGNAL 4 +#define EXIT_REASON_OTHER_SMI 6#define EXIT_REASON_INTERRUPT_WINDOW 7#define EXIT_REASON_NMI_WINDOW 8
arch/x86/kvm/Makefile +4 -1
@@ -29,7 +29,10 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o -kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx_error.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx_error.o vmx/tdx.o +ifneq ($(CONFIG_INTEL_TDX_HOST),y) +kvm-intel-y += vmx/tdx_stubs.o +endif kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o
arch/x86/kvm/mmu.h +1 -1
@@ -65,7 +65,7 @@ static __always_inline u64 rsvd_bits(int s, int e) }void kvm_mmu_set_mmio_spte_mask (u64 mmio_value,u64 mmio_mask,u64 access_mask); -void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only); +void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only,u64 init_value);void kvm_mmu_set_spte_init_value (u64 init_value);void kvm_init_mmu (struct kvm_vcpu *vcpu);
arch/x86/kvm/mmu/mmu.c +1 -0
@@ -5461,6 +5461,7 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu) out:return r; } +EXPORT_SYMBOL_GPL (kvm_mmu_load);static void __kvm_mmu_unload (struct kvm_vcpu *vcpu,u32 roots_to_free) {
arch/x86/kvm/mmu/spte.c +3 -2
@@ -312,14 +312,15 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask) }EXPORT_SYMBOL_GPL (kvm_mmu_set_mmio_spte_mask); -void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only) +void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only,u64 init_value) { shadow_user_mask =VMX_EPT_READABLE_MASK ; shadow_accessed_mask = has_ad_bits ?VMX_EPT_ACCESS_BIT :0ull ; shadow_dirty_mask = has_ad_bits ?VMX_EPT_DIRTY_BIT :0ull ; shadow_nx_mask =0ull ; shadow_x_mask =VMX_EPT_EXECUTABLE_MASK ; - shadow_present_mask = has_exec_only ?0ull :VMX_EPT_READABLE_MASK ; + shadow_present_mask = + (has_exec_only ?0ull :VMX_EPT_READABLE_MASK ) | init_value; shadow_acc_track_mask =VMX_EPT_RWX_MASK ; shadow_me_mask =0ull ;
arch/x86/kvm/vmx/common.h +1 -0
@@ -9,6 +9,7 @@#include <asm/vmx.h> #include "mmu.h" +#include "tdx.h" #include "vmcs.h" #include "vmx.h" #include "x86.h"
arch/x86/kvm/vmx/main.c +390 -13
919 lines not on this page · raw message
arch/x86/kvm/vmx/posted_intr.c +6 -0
@@ -6,6 +6,7 @@#include "lapic.h" #include "posted_intr.h" +#include "tdx.h" #include "trace.h" #include "vmx.h"
@@ -18,6 +19,11 @@ static DEFINE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock);static inline struct pi_desc *vcpu_to_pi_desc (struct kvm_vcpu *vcpu) { +#ifdef CONFIG_INTEL_TDX_HOST +if (is_td_vcpu (vcpu)) +return &(to_tdx (vcpu)->pi_desc); +#endif +return &(to_vmx (vcpu)->pi_desc); }
arch/x86/kvm/vmx/tdx.h +114 -0
159 lines not on this page · raw message
arch/x86/kvm/vmx/tdx_arch.h +22 -2
@@ -54,11 +54,21 @@#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004/* TDX control structure (TDR/TDCS/TDVPS) field access codes */ +#define TDX_NON_ARCH BIT_ULL(63)#define TDX_CLASS_SHIFT 56#define TDX_FIELD_MASK GENMASK_ULL(31, 0) -#define BUILD_TDX_FIELD (class, field) \ - (((u64)(class) << TDX_CLASS_SHIFT) | ((u64)(field) & TDX_FIELD_MASK)) +#define __BUILD_TDX_FIELD (non_arch, class, field) \ + (((non_arch) ? TDX_NON_ARCH : 0) | \ + ((u64)(class) << TDX_CLASS_SHIFT) | \ + ((u64)(field) & TDX_FIELD_MASK)) + +#define BUILD_TDX_FIELD (class, field) \ + __BUILD_TDX_FIELD(false, (class), (field)) + +#define BUILD_TDX_FIELD_NON_ARCH (class, field) \ + __BUILD_TDX_FIELD(true, (class), (field)) +/* @field is the VMCS field encoding */ #define TDVPS_VMCS (field) BUILD_TDX_FIELD(0, (field))
@@ -83,10 +93,20 @@ enum tdx_guest_other_state {TD_VCPU_IWK_INTKEY0 =68 ,TD_VCPU_IWK_INTKEY1 ,TD_VCPU_IWK_FLAGS =70 , + TD_VCPU_STATE_DETAILS_NON_ARCH = 0x100, +}; + +union tdx_vcpu_state_details { +struct { +u64 vmxip :1 ; +u64 reserved :63 ; + }; +u64 full; };/* @field is any of enum tdx_guest_other_state */ #define TDVPS_STATE (field) BUILD_TDX_FIELD(17, (field)) +#define TDVPS_STATE_NON_ARCH (field) BUILD_TDX_FIELD_NON_ARCH(17, field)/* @msr is the MSR index */ #define TDVPS_MSR (msr) BUILD_TDX_FIELD(19, (msr))
arch/x86/kvm/vmx/tdx_ops.h +14 -0
@@ -8,36 +8,48 @@#include <asm/asm.h> #include <asm/kvm_host.h> +#include <asm/cacheflush.h> #include "seamcall.h" +#include "tdx_arch.h" #ifdef CONFIG_INTEL_TDX_HOST +static inline void tdx_clflush_page (hpa_t addr) +{ +clflush_cache_range (__va (addr),PAGE_SIZE ); +} +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) { +tdx_clflush_page (addr);return seamcall (TDH_MNG_ADDCX , addr, tdr,0 ,0 ,0 ,NULL ); }static inline u64 tdh_mem_page_add (hpa_t tdr,gpa_t gpa,hpa_t hpa,hpa_t source,struct tdx_ex_ret *ex) { +tdx_clflush_page (hpa);return seamcall (TDH_MEM_PAGE_ADD , gpa, tdr, hpa, source,0 , ex); }static inline u64 tdh_mem_sept_add (hpa_t tdr,gpa_t gpa,int level,hpa_t page,struct tdx_ex_ret *ex) { +tdx_clflush_page (page);return seamcall (TDH_MEM_SEPT_ADD , gpa | level, tdr, page,0 ,0 , ex); }static inline u64 tdh_vp_addcx (hpa_t tdvpr,hpa_t addr) { +tdx_clflush_page (addr);return seamcall (TDH_VP_ADDCX , addr, tdvpr,0 ,0 ,0 ,NULL ); }static inline u64 tdh_mem_page_aug (hpa_t tdr,gpa_t gpa,hpa_t hpa,struct tdx_ex_ret *ex) { +tdx_clflush_page (hpa);return seamcall (TDH_MEM_PAGE_AUG , gpa, tdr, hpa,0 ,0 , ex); }
@@ -54,11 +66,13 @@ static inline u64 tdh_mng_key_config(hpa_t tdr)static inline u64 tdh_mng_create (hpa_t tdr,int hkid) { +tdx_clflush_page (tdr);return seamcall (TDH_MNG_CREATE , tdr, hkid,0 ,0 ,0 ,NULL ); }static inline u64 tdh_vp_create (hpa_t tdr,hpa_t tdvpr) { +tdx_clflush_page (tdvpr);return seamcall (TDH_VP_CREATE , tdvpr, tdr,0 ,0 ,0 ,NULL ); }
arch/x86/kvm/vmx/tdx_stubs.c new file +50 -0
50 lines not on this page · raw message
arch/x86/kvm/vmx/vmenter.S +146 -0
161 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.c +0 -39
@@ -3752,45 +3752,6 @@ void vmx_msr_filter_changed(struct kvm_vcpu *vcpu)pt_update_intercept_for_msr (vcpu); } -static inline bool kvm_vcpu_trigger_posted_interrupt (struct kvm_vcpu *vcpu, -bool nested) -{ -#ifdef CONFIG_SMP -int pi_vec = nested ?POSTED_INTR_NESTED_VECTOR :POSTED_INTR_VECTOR ; - -if (vcpu->mode ==IN_GUEST_MODE ) { -/* -* The vector of interrupt to be delivered to vcpu had -* been set in PIR before this function. -* -* Following cases will be reached in this block, and -* we always send a notification event in all cases as -* explained below. -* -* Case 1: vcpu keeps in non-root mode. Sending a -* notification event posts the interrupt to vcpu. -* -* Case 2: vcpu exits to root mode and is still -* runnable. PIR will be synced to vIRR before the -* next vcpu entry. Sending a notification event in -* this case has no effect, as vcpu is not in root -* mode. -* -* Case 3: vcpu exits to root mode and is blocked. -* vcpu_block() has already synced PIR to vIRR and -* never blocks vcpu if vIRR is not cleared. Therefore, -* a blocked vcpu here does not wait for any requested -* interrupts in PIR, and sending a notification event -* which has no effect is safe here. -*/ - - apic->send_IPI_mask (get_cpu_mask (vcpu->cpu), pi_vec); -return true; - } -#endif -return false; -} -static int vmx_deliver_nested_posted_interrupt (struct kvm_vcpu *vcpu,int vector) {
arch/x86/kvm/vmx/x86_ops.h +80 -0
90 lines not on this page · raw message
arch/x86/kvm/x86.c +4 -4
@@ -292,6 +292,7 @@ const struct kvm_stats_header kvm_vcpu_stats_header = { };u64 __read_mostly host_xcr0; +EXPORT_SYMBOL_GPL (host_xcr0);u64 __read_mostly supported_xcr0;EXPORT_SYMBOL_GPL (supported_xcr0);
@@ -2265,9 +2266,7 @@ static int set_tsc_khz(struct kvm_vcpu *vcpu, u32 user_tsc_khz, bool scale)u64 ratio;/* Guest TSC same frequency as host TSC? */ -if (!scale || vcpu->kvm->arch.tsc_immutable) { -if (scale) -pr_warn_ratelimited ("Guest TSC immutable, scaling not supported\n" ); +if (!scale) {kvm_vcpu_write_tsc_multiplier (vcpu, kvm_default_tsc_scaling_ratio);return 0 ; }
@@ -10740,7 +10739,8 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, {int ret; -if (vcpu->arch.guest_state_protected) +if (vcpu->arch.guest_state_protected || + vcpu->kvm->arch.vm_type ==KVM_X86_TDX_VM )return -EINVAL ;vcpu_load (vcpu);
tools/arch/x86/include/uapi/asm/kvm.h +51 -0
55 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v3 58/59 KVM: TDX: exit to user space on GET_QUOTE, SETUP_EVENT_NOTIFY_INTERRUPT
In reply to [email protected]From: Isaku Yamahata <[email protected]>
GET_QUOTE, SETUP_EVENT_NOTIFY_INTERRUPT TDG.VP.VMCALL requires user space to handle them on behalf of kvm kernel module.
Introduce new kvm exit, KVM_EXIT_TDX, and when GET_QUOTE and SETUP_EVENT_NOTIFY_INTERRUPT is called by TD guest, transfer the execution to user space as KVM exit to the user space. TDG_VP_VMCALL_INVALID_OPERAND is set as default return value to avoid random value. User space should update R10 if necessary.
--- arch/x86/kvm/vmx/tdx.c | 113 +++++++++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 57 ++++++++++++++++++++ 2 files changed, 170 insertions(+)
arch/x86/kvm/vmx/tdx.c +113 -0
@@ -152,6 +152,18 @@ BUILD_TDVMCALL_ACCESSORS(p2, r13);BUILD_TDVMCALL_ACCESSORS (p3, r14);BUILD_TDVMCALL_ACCESSORS (p4, r15); +#define TDX_VMCALL_REG_MASK_RBX BIT_ULL(2) +#define TDX_VMCALL_REG_MASK_RDX BIT_ULL(3) +#define TDX_VMCALL_REG_MASK_RBP BIT_ULL(5) +#define TDX_VMCALL_REG_MASK_RSI BIT_ULL(6) +#define TDX_VMCALL_REG_MASK_RDI BIT_ULL(7) +#define TDX_VMCALL_REG_MASK_R8 BIT_ULL(8) +#define TDX_VMCALL_REG_MASK_R9 BIT_ULL(9) +#define TDX_VMCALL_REG_MASK_R12 BIT_ULL(12) +#define TDX_VMCALL_REG_MASK_R13 BIT_ULL(13) +#define TDX_VMCALL_REG_MASK_R14 BIT_ULL(14) +#define TDX_VMCALL_REG_MASK_R15 BIT_ULL(15) +static __always_inline unsigned long tdvmcall_exit_type (struct kvm_vcpu *vcpu) {return kvm_r10_read (vcpu);
@@ -1122,6 +1134,91 @@ static int tdx_map_gpa(struct kvm_vcpu *vcpu)return 1 ; } +static int tdx_complete_vp_vmcall (struct kvm_vcpu *vcpu) +{ +struct kvm_tdx_vmcall *tdx_vmcall = &vcpu->run->tdx.u.vmcall; +__u64 reg_mask; + +tdvmcall_set_return_code (vcpu, tdx_vmcall->status_code); +tdvmcall_set_return_val (vcpu, tdx_vmcall->out_r11); + + reg_mask =kvm_rcx_read (vcpu); +if (reg_mask &TDX_VMCALL_REG_MASK_R12 ) +kvm_r12_write (vcpu, tdx_vmcall->out_r12); +if (reg_mask &TDX_VMCALL_REG_MASK_R13 ) +kvm_r13_write (vcpu, tdx_vmcall->out_r13); +if (reg_mask &TDX_VMCALL_REG_MASK_R14 ) +kvm_r14_write (vcpu, tdx_vmcall->out_r14); +if (reg_mask &TDX_VMCALL_REG_MASK_R15 ) +kvm_r15_write (vcpu, tdx_vmcall->out_r15); +if (reg_mask &TDX_VMCALL_REG_MASK_RBX ) +kvm_rbx_write (vcpu, tdx_vmcall->out_rbx); +if (reg_mask &TDX_VMCALL_REG_MASK_RDI ) +kvm_rdi_write (vcpu, tdx_vmcall->out_rdi); +if (reg_mask &TDX_VMCALL_REG_MASK_RSI ) +kvm_rsi_write (vcpu, tdx_vmcall->out_rsi); +if (reg_mask &TDX_VMCALL_REG_MASK_R8 ) +kvm_r8_write (vcpu, tdx_vmcall->out_r8); +if (reg_mask &TDX_VMCALL_REG_MASK_R9 ) +kvm_r9_write (vcpu, tdx_vmcall->out_r9); +if (reg_mask &TDX_VMCALL_REG_MASK_RDX ) +kvm_rdx_write (vcpu, tdx_vmcall->out_rdx); + +return 1 ; +} + +static int tdx_vp_vmcall_to_user (struct kvm_vcpu *vcpu) +{ +struct kvm_tdx_vmcall *tdx_vmcall = &vcpu->run->tdx.u.vmcall; +__u64 reg_mask; + + vcpu->arch.complete_userspace_io = tdx_complete_vp_vmcall; +memset (tdx_vmcall,0 ,sizeof (*tdx_vmcall)); + + vcpu->run->exit_reason =KVM_EXIT_TDX ; + vcpu->run->tdx.type =KVM_EXIT_TDX_VMCALL ; + tdx_vmcall->type =tdvmcall_exit_type (vcpu); + tdx_vmcall->subfunction =tdvmcall_exit_reason (vcpu); + + reg_mask =kvm_rcx_read (vcpu); + tdx_vmcall->reg_mask = reg_mask; +if (reg_mask &TDX_VMCALL_REG_MASK_R12 ) + tdx_vmcall->in_r12 =kvm_r12_read (vcpu); +if (reg_mask &TDX_VMCALL_REG_MASK_R13 ) + tdx_vmcall->in_r13 =kvm_r13_read (vcpu); +if (reg_mask &TDX_VMCALL_REG_MASK_R14 ) + tdx_vmcall->in_r14 =kvm_r14_read (vcpu); +if (reg_mask &TDX_VMCALL_REG_MASK_R15 ) + tdx_vmcall->in_r15 =kvm_r15_read (vcpu); +if (reg_mask &TDX_VMCALL_REG_MASK_RBX ) + tdx_vmcall->in_rbx =kvm_rbx_read (vcpu); +if (reg_mask &TDX_VMCALL_REG_MASK_RDI ) + tdx_vmcall->in_rdi =kvm_rdi_read (vcpu); +if (reg_mask &TDX_VMCALL_REG_MASK_RSI ) + tdx_vmcall->in_rsi =kvm_rsi_read (vcpu); +if (reg_mask &TDX_VMCALL_REG_MASK_R8 ) + tdx_vmcall->in_r8 =kvm_r8_read (vcpu); +if (reg_mask &TDX_VMCALL_REG_MASK_R9 ) + tdx_vmcall->in_r9 =kvm_r9_read (vcpu); +if (reg_mask &TDX_VMCALL_REG_MASK_RDX ) + tdx_vmcall->in_rdx =kvm_rdx_read (vcpu); + +/* notify userspace to handle the request */ +return 0 ; +} + +static int tdx_get_quote (struct kvm_vcpu *vcpu) +{ +gpa_t gpa =tdvmcall_p1_read (vcpu); + +if (!IS_ALIGNED (gpa,PAGE_SIZE )) { +tdvmcall_set_return_code (vcpu,TDG_VP_VMCALL_INVALID_OPERAND ); +return 1 ; + } + +return tdx_vp_vmcall_to_user (vcpu); +} +static int tdx_report_fatal_error (struct kvm_vcpu *vcpu) { vcpu->run->exit_reason =KVM_EXIT_SYSTEM_EVENT ;
@@ -1130,6 +1227,18 @@ static int tdx_report_fatal_error(struct kvm_vcpu *vcpu)return 0 ; } +static int tdx_setup_event_notify_interrupt (struct kvm_vcpu *vcpu) +{ +u64 vector =tdvmcall_p1_read (vcpu); + +if (!(vector >=32 && vector <=255 )) { +tdvmcall_set_return_code (vcpu,TDG_VP_VMCALL_INVALID_OPERAND ); +return 1 ; + } + +return tdx_vp_vmcall_to_user (vcpu); +} +static int handle_tdvmcall (struct kvm_vcpu *vcpu) {struct vcpu_tdx *tdx =to_tdx (vcpu);
@@ -1158,8 +1267,12 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu)return tdx_emulate_mmio (vcpu);case TDG_VP_VMCALL_MAP_GPA :return tdx_map_gpa (vcpu); +case TDG_VP_VMCALL_GET_QUOTE : +return tdx_get_quote (vcpu);case TDG_VP_VMCALL_REPORT_FATAL_ERROR :return tdx_report_fatal_error (vcpu); +case TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT : +return tdx_setup_event_notify_interrupt (vcpu);default :break ; }
include/uapi/linux/kvm.h +57 -0
75 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v3 54/59 KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
In reply to [email protected]From: Xiaoyao Li <[email protected]>
Introduce a per-vm variable initial_tsc_khz to hold the default tsc_khz for kvm_arch_vcpu_create().
This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/tdx.c | 2233 +++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 3 +- 3 files changed, 2236 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/vmx/tdx.c
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1111,6 +1111,7 @@ struct kvm_arch {u64 last_tsc_write;u32 last_tsc_khz;u64 last_tsc_offset; +u32 initial_tsc_khz;u64 cur_tsc_nsec;u64 cur_tsc_write;u64 cur_tsc_offset;
arch/x86/kvm/vmx/tdx.c new file +2233 -0
2,233 lines not on this page · raw message
arch/x86/kvm/x86.c +2 -1
@@ -11019,7 +11019,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) vcpu->arch.msr_platform_info =MSR_PLATFORM_INFO_CPUID_FAULT ;kvm_vcpu_mtrr_init (vcpu);vcpu_load (vcpu); -kvm_set_tsc_khz (vcpu, max_tsc_khz); +kvm_set_tsc_khz (vcpu, vcpu->kvm->arch.initial_tsc_khz);kvm_vcpu_reset (vcpu, false);kvm_init_mmu (vcpu);vcpu_put (vcpu);
@@ -11473,6 +11473,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)raw_spin_unlock_irqrestore (&kvm->arch.tsc_write_lock, flags); kvm->arch.guest_can_read_msr_platform_info = true; + kvm->arch.initial_tsc_khz = max_tsc_khz;#if IS_ENABLED (CONFIG_HYPERV )spin_lock_init (&kvm->arch.hv_root_tdp_lock);
Signature · 1 line
2.25.1
RFC PATCH v3 22/59 KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add a flag to let TDX disallow to inject interrupt with delivery mode of SMI/INIT/SIPI. add a check to reject SMI/INIT interrupt delivery mode.
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/irq_comm.c | 4 ++++ arch/x86/kvm/x86.c | 3 +-- 3 files changed, 7 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1133,6 +1133,8 @@ struct kvm_arch {enum kvm_irqchip_mode irqchip_mode;u8 nr_reserved_ioapic_pins;bool eoi_intercept_unsupported; +bool smm_unsupported; +bool init_sipi_unsupported;bool disabled_lapic_found;
arch/x86/kvm/irq_comm.c +4 -0
@@ -128,6 +128,10 @@ static inline bool kvm_msi_route_invalid(struct kvm *kvm, .data = e->msi.data };return (kvm->arch.eoi_intercept_unsupported && msg.arch_data.is_level) || + (kvm->arch.smm_unsupported && + msg.arch_data.delivery_mode ==APIC_DELIVERY_MODE_SMI ) || + (kvm->arch.init_sipi_unsupported && + msg.arch_data.delivery_mode ==APIC_DELIVERY_MODE_INIT ) || (kvm->arch.x2apic_format && (msg.address_hi &0xff )); }
arch/x86/kvm/x86.c +1 -2
@@ -4553,8 +4553,7 @@ static int kvm_vcpu_ioctl_nmi(struct kvm_vcpu *vcpu)static int kvm_vcpu_ioctl_smi (struct kvm_vcpu *vcpu) { -/* TODO: use more precise flag */ -if (vcpu->arch.guest_state_protected) +if (vcpu->kvm->arch.smm_unsupported)return -EINVAL ;kvm_make_request (KVM_REQ_SMI , vcpu);
Signature · 1 line
2.25.1
RFC PATCH v3 23/59 KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
In reply to [email protected]From: Sean Christopherson <[email protected]>
Let userspace, or in the case of TDX, KVM itself, enable X2APIC even if X2APIC is not reported as supported in the guest's CPU model. KVM generally does not force specific ordering between ioctls(), e.g. this forces userspace to configure CPUID before MSRs. And for TDX, vCPUs will always run with X2APIC enabled, e.g. KVM will want/need to enable X2APIC from time zero.
--- arch/x86/kvm/x86.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +5 -2
@@ -466,8 +466,11 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info) {enum lapic_mode old_mode =kvm_get_apic_mode (vcpu);enum lapic_mode new_mode =kvm_apic_mode (msr_info->data); -u64 reserved_bits =kvm_vcpu_reserved_gpa_bits_raw (vcpu) |0x2ff | - (guest_cpuid_has (vcpu,X86_FEATURE_X2APIC ) ?0 :X2APIC_ENABLE ); +u64 reserved_bits =kvm_vcpu_reserved_gpa_bits_raw (vcpu) |0x2ff ; + +if (!msr_info->host_initiated && + !guest_cpuid_has (vcpu,X86_FEATURE_X2APIC )) + reserved_bits |=X2APIC_ENABLE ;if ((msr_info->data & reserved_bits) !=0 || new_mode ==LAPIC_MODE_INVALID )return 1 ;
Signature · 1 line
2.25.1
RFC PATCH v3 59/59 Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add a documentation to Intel Trusted Docmain Extensions(TDX) support.
--- Documentation/virt/kvm/api.rst | 9 +- Documentation/virt/kvm/intel-tdx.rst | 359 +++++++++++++++++++++++++++ 2 files changed, 367 insertions(+), 1 deletion(-) create mode 100644 Documentation/virt/kvm/intel-tdx.rst
Documentation/virt/kvm/api.rst +8 -1
@@ -1366,6 +1366,9 @@ It is recommended to use this API instead of the KVM_SET_MEMORY_REGION ioctl. The KVM_SET_MEMORY_REGION does not allow fine grained control over memory allocation and is deprecated. +For TDX guest, deleting/moving memory region loses guest memory contents. +Read only region isn't supported. Only as-id 0 is supported. + 4.36 KVM_SET_TSS_ADDR ---------------------
@@ -4489,7 +4492,7 @@ H_GET_CPU_CHARACTERISTICS hypercall. :Capability: basic :Architectures: x86 -:Type: vm +:Type: vm ioctl, vcpu ioctl :Parameters: an opaque platform specific structure (in/out) :Returns: 0 on success; -1 on error
@@ -4501,6 +4504,10 @@ Currently, this ioctl is used for issuing Secure Encrypted Virtualization (SEV) commands on AMD Processors. The SEV commands are defined in Documentation/virt/kvm/amd-memory-encryption.rst. +Currently, this ioctl is used for issuing Trusted Domain Extensions +(TDX) commands on Intel Processors. The TDX commands are defined in +Documentation/virt/kvm/intel-tdx.rst. + 4.111 KVM_MEMORY_ENCRYPT_REG_REGION -----------------------------------
Documentation/virt/kvm/intel-tdx.rst new file +359 -0
359 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v3 20/59 KVM: Add per-VM flag to disable dirty logging of memslots for TDs
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a flag for TDX to mark dirty logging as unsupported.
--- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-)
include/linux/kvm_host.h +1 -0
@@ -624,6 +624,7 @@ struct kvm {struct notifier_block pm_notifier;#endif char stats_id[KVM_STATS_NAME_SIZE ]; +bool dirty_log_unsupported;#ifdef __KVM_HAVE_READONLY_MEMbool readonly_mem_unsupported;#endif
virt/kvm/kvm_main.c +4 -1
@@ -1425,7 +1425,10 @@ static void update_memslots(struct kvm_memslots *slots,static int check_memory_region_flags (struct kvm *kvm,const struct kvm_userspace_memory_region *mem) { -u32 valid_flags =KVM_MEM_LOG_DIRTY_PAGES ; +u32 valid_flags =0 ; + +if (!kvm->dirty_log_unsupported) + valid_flags |=KVM_MEM_LOG_DIRTY_PAGES ;#ifdef __KVM_HAVE_READONLY_MEMif (!kvm->readonly_mem_unsupported)
Signature · 1 line
2.25.1
RFC PATCH v3 19/59 KVM: Add per-VM flag to mark read-only memory as unsupported
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add a flag for TDX to flag RO memory as unsupported and propagate it to KVM_MEM_READONLY to allow reporting RO memory as unsupported on a per-VM basis. TDX1 doesn't expose permission bits to the VMM in the SEPT tables, i.e. doesn't support read-only private memory.
--- arch/x86/kvm/x86.c | 4 +++- include/linux/kvm_host.h | 3 +++ virt/kvm/kvm_main.c | 8 +++++--- 3 files changed, 11 insertions(+), 4 deletions(-)
arch/x86/kvm/x86.c +3 -1
@@ -4121,7 +4121,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_ASYNC_PF_INT :case KVM_CAP_GET_TSC_KHZ :case KVM_CAP_KVMCLOCK_CTRL : -case KVM_CAP_READONLY_MEM :case KVM_CAP_HYPERV_TIME :case KVM_CAP_IOAPIC_POLARITY_IGNORED :case KVM_CAP_TSC_DEADLINE_TIMER :
@@ -4239,6 +4238,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)if (static_call (kvm_x86_is_vm_type_supported)(KVM_X86_TDX_VM )) r |=BIT (KVM_X86_TDX_VM );break ; +case KVM_CAP_READONLY_MEM : + r = kvm && kvm->readonly_mem_unsupported ?0 :1 ; +break ;default :break ; }
include/linux/kvm_host.h +3 -0
@@ -624,6 +624,9 @@ struct kvm {struct notifier_block pm_notifier;#endif char stats_id[KVM_STATS_NAME_SIZE ]; +#ifdef __KVM_HAVE_READONLY_MEM +bool readonly_mem_unsupported; +#endif };#define kvm_err (fmt, ...) \
virt/kvm/kvm_main.c +5 -3
@@ -1422,12 +1422,14 @@ static void update_memslots(struct kvm_memslots *slots, } } -static int check_memory_region_flags (const struct kvm_userspace_memory_region *mem) +static int check_memory_region_flags (struct kvm *kvm, +const struct kvm_userspace_memory_region *mem) {u32 valid_flags =KVM_MEM_LOG_DIRTY_PAGES ;#ifdef __KVM_HAVE_READONLY_MEM - valid_flags |=KVM_MEM_READONLY ; +if (!kvm->readonly_mem_unsupported) + valid_flags |=KVM_MEM_READONLY ;#endif if (mem->flags & ~valid_flags)
@@ -1665,7 +1667,7 @@ int __kvm_set_memory_region(struct kvm *kvm,int as_id, id;int r; - r =check_memory_region_flags (mem); + r =check_memory_region_flags (kvm, mem);if (r)return r;
Signature · 1 line
2.25.1
RFC PATCH v3 17/59 KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a flag to disallow MCE injection and reject KVM_X86_SETUP_MCE with -EINVAL when set. TDX doesn't support injecting exceptions, including (virtual) #MCs.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 14 +++++++------- 2 files changed, 8 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1142,6 +1142,7 @@ struct kvm_arch {bool bus_lock_detection_enabled;bool irq_injection_disallowed; +bool mce_injection_disallowed;/* * If exit_on_emulation_error is set, and the in-kernel instruction * emulator fails to emulate an instruction, allow userspace
arch/x86/kvm/x86.c +7 -7
@@ -4561,15 +4561,16 @@ static int vcpu_ioctl_tpr_access_reporting(struct kvm_vcpu *vcpu,static int kvm_vcpu_ioctl_x86_setup_mce (struct kvm_vcpu *vcpu,u64 mcg_cap) { -int r;unsigned bank_num = mcg_cap &0xff , bank; - r = -EINVAL ; +if (vcpu->kvm->arch.mce_injection_disallowed) +return -EINVAL ; +if (!bank_num || bank_num >KVM_MAX_MCE_BANKS ) - goto out; +return -EINVAL ;if (mcg_cap & ~(kvm_mce_cap_supported |0xff |0xff0000 )) - goto out; - r =0 ; +return -EINVAL ; + vcpu->arch.mcg_cap = mcg_cap;/* Init IA32_MCG_CTL to all 1s */ if (mcg_cap &MCG_CTL_P )
@@ -4579,8 +4580,7 @@ static int kvm_vcpu_ioctl_x86_setup_mce(struct kvm_vcpu *vcpu, vcpu->arch.mce_banks[bank*4 ] = ~(u64 )0 ;static_call (kvm_x86_setup_mce)(vcpu); -out: -return r; +return 0 ; } static int kvm_vcpu_ioctl_x86_set_mce(struct kvm_vcpu *vcpu,
Signature · 1 line
2.25.1
RFC PATCH v3 15/59 KVM: x86: Introduce "protected guest" concept and block disallowed ioctls
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add 'guest_state_protected' to mark a VM's state as being protected by hardware/firmware, e.g. SEV-ES or TDX-SEAM. Use the flag to disallow ioctls() and/or flows that attempt to access protected state.
Return an error if userspace attempts to get/set register state for a protected VM, e.g. a non-debug TDX guest. KVM can't provide sane data, it's userspace's responsibility to avoid attempting to read guest state when it's known to be inaccessible.
Retrieving vCPU events is the one exception, as the userspace VMM is allowed to inject NMIs.
--- arch/x86/kvm/x86.c | 80 ++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 66 insertions(+), 14 deletions(-)
arch/x86/kvm/x86.c +66 -14
@@ -4539,6 +4539,10 @@ static int kvm_vcpu_ioctl_nmi(struct kvm_vcpu *vcpu)static int kvm_vcpu_ioctl_smi (struct kvm_vcpu *vcpu) { +/* TODO: use more precise flag */ +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +kvm_make_request (KVM_REQ_SMI , vcpu);return 0 ;
@@ -4585,6 +4589,10 @@ static int kvm_vcpu_ioctl_x86_set_mce(struct kvm_vcpu *vcpu,unsigned bank_num = mcg_cap &0xff ;u64 *banks = vcpu->arch.mce_banks; +/* TODO: use more precise flag */ +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (mce->bank >= bank_num || !(mce->status &MCI_STATUS_VAL ))return -EINVAL ;/*
@@ -4680,7 +4688,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu *vcpu, vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft; events->interrupt.nr = vcpu->arch.interrupt.nr; events->interrupt.soft =0 ; - events->interrupt.shadow =static_call (kvm_x86_get_interrupt_shadow)(vcpu); +if (!vcpu->arch.guest_state_protected) + events->interrupt.shadow =static_call (kvm_x86_get_interrupt_shadow)(vcpu); events->nmi.injected = vcpu->arch.nmi_injected; events->nmi.pending = vcpu->arch.nmi_pending !=0 ;
@@ -4709,11 +4718,17 @@ static void kvm_smm_changed(struct kvm_vcpu *vcpu, bool entering_smm);static int kvm_vcpu_ioctl_x86_set_vcpu_events (struct kvm_vcpu *vcpu,struct kvm_vcpu_events *events) { -if (events->flags & ~(KVM_VCPUEVENT_VALID_NMI_PENDING - |KVM_VCPUEVENT_VALID_SIPI_VECTOR - |KVM_VCPUEVENT_VALID_SHADOW - |KVM_VCPUEVENT_VALID_SMM - |KVM_VCPUEVENT_VALID_PAYLOAD )) +u32 allowed_flags =KVM_VCPUEVENT_VALID_NMI_PENDING | +KVM_VCPUEVENT_VALID_SIPI_VECTOR | +KVM_VCPUEVENT_VALID_SHADOW | +KVM_VCPUEVENT_VALID_SMM | +KVM_VCPUEVENT_VALID_PAYLOAD ; + +/* TODO: introduce more precise flag */ +if (vcpu->arch.guest_state_protected) + allowed_flags =KVM_VCPUEVENT_VALID_NMI_PENDING ; + +if (events->flags & ~allowed_flags)return -EINVAL ;if (events->flags &KVM_VCPUEVENT_VALID_PAYLOAD ) {
@@ -4789,17 +4804,22 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu,return 0 ; } -static void kvm_vcpu_ioctl_x86_get_debugregs (struct kvm_vcpu *vcpu, -struct kvm_debugregs *dbgregs) +static int kvm_vcpu_ioctl_x86_get_debugregs (struct kvm_vcpu *vcpu, +struct kvm_debugregs *dbgregs) {unsigned long val; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +memcpy (dbgregs->db, vcpu->arch.db,sizeof (vcpu->arch.db));kvm_get_dr (vcpu,6 , &val); dbgregs->dr6 = val; dbgregs->dr7 = vcpu->arch.dr7; dbgregs->flags =0 ;memset (&dbgregs->reserved,0 ,sizeof (dbgregs->reserved)); + +return 0 ; } static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu,
@@ -4813,6 +4833,9 @@ static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu,if (!kvm_dr7_valid (dbgregs->dr7))return -EINVAL ; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +memcpy (vcpu->arch.db, dbgregs->db,sizeof (vcpu->arch.db));kvm_update_dr0123 (vcpu); vcpu->arch.dr6 = dbgregs->dr6;
@@ -4845,18 +4868,22 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu, supported_xcr0, &vcpu->arch.pkru); } -static void kvm_vcpu_ioctl_x86_get_xcrs (struct kvm_vcpu *vcpu, -struct kvm_xcrs *guest_xcrs) +static int kvm_vcpu_ioctl_x86_get_xcrs (struct kvm_vcpu *vcpu, +struct kvm_xcrs *guest_xcrs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!boot_cpu_has (X86_FEATURE_XSAVE )) { guest_xcrs->nr_xcrs =0 ; -return ; +return 0 ; } guest_xcrs->nr_xcrs =1 ; guest_xcrs->flags =0 ; guest_xcrs->xcrs[0 ].xcr =XCR_XFEATURE_ENABLED_MASK ; guest_xcrs->xcrs[0 ].value = vcpu->arch.xcr0; +return 0 ; } static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu *vcpu,
@@ -4864,6 +4891,9 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu *vcpu, {int i, r =0 ; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!boot_cpu_has (X86_FEATURE_XSAVE ))return -EINVAL ;
@@ -5247,7 +5277,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,case KVM_GET_DEBUGREGS : {struct kvm_debugregs dbgregs; -kvm_vcpu_ioctl_x86_get_debugregs (vcpu, &dbgregs); + r =kvm_vcpu_ioctl_x86_get_debugregs (vcpu, &dbgregs); +if (r) +break ; r = -EFAULT ; if (copy_to_user(argp, &dbgregs,
@@ -5297,7 +5329,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,if (!u.xcrs)break ; -kvm_vcpu_ioctl_x86_get_xcrs (vcpu, u.xcrs); + r =kvm_vcpu_ioctl_x86_get_xcrs (vcpu, u.xcrs); +if (r) +break ; r = -EFAULT ; if (copy_to_user(argp, u.xcrs,
@@ -10188,6 +10222,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) goto out; } +if (vcpu->arch.guest_state_protected && + (kvm_run->kvm_valid_regs || kvm_run->kvm_dirty_regs)) { + r = -EINVAL ; + goto out; + } +if (kvm_run->kvm_dirty_regs) { r =sync_regs (vcpu);if (r !=0 )
@@ -10218,7 +10258,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) out:kvm_put_guest_fpu (vcpu); -if (kvm_run->kvm_valid_regs) +if (kvm_run->kvm_valid_regs && !vcpu->arch.guest_state_protected)store_regs (vcpu);post_kvm_run_save (vcpu);kvm_sigset_deactivate (vcpu);
@@ -10265,6 +10305,9 @@ static void __get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)int kvm_arch_vcpu_ioctl_get_regs (struct kvm_vcpu *vcpu,struct kvm_regs *regs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu);__get_regs (vcpu, regs);vcpu_put (vcpu);
@@ -10305,6 +10348,9 @@ static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)int kvm_arch_vcpu_ioctl_set_regs (struct kvm_vcpu *vcpu,struct kvm_regs *regs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu);__set_regs (vcpu, regs);vcpu_put (vcpu);
@@ -10387,6 +10433,9 @@ static void __get_sregs2(struct kvm_vcpu *vcpu, struct kvm_sregs2 *sregs2)int kvm_arch_vcpu_ioctl_get_sregs (struct kvm_vcpu *vcpu,struct kvm_sregs *sregs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu);__get_sregs (vcpu, sregs);vcpu_put (vcpu);
@@ -10635,6 +10684,9 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, {int ret; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu); ret =__set_sregs (vcpu, sregs);vcpu_put (vcpu);
Signature · 1 line
2.25.1
RFC PATCH v3 16/59 KVM: x86: Add per-VM flag to disable direct IRQ injection
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a flag to disable IRQ injection, which is not supported by TDX.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1141,6 +1141,7 @@ struct kvm_arch {bool exception_payload_enabled;bool bus_lock_detection_enabled; +bool irq_injection_disallowed;/* * If exit_on_emulation_error is set, and the in-kernel instruction * emulator fails to emulate an instruction, allow userspace
arch/x86/kvm/x86.c +3 -1
@@ -4506,7 +4506,8 @@ static int kvm_vcpu_ready_for_interrupt_injection(struct kvm_vcpu *vcpu)static int kvm_vcpu_ioctl_interrupt (struct kvm_vcpu *vcpu,struct kvm_interrupt *irq) { -if (irq->irq >=KVM_NR_INTERRUPTS ) +if (irq->irq >=KVM_NR_INTERRUPTS || + vcpu->kvm->arch.irq_injection_disallowed)return -EINVAL ;if (!irqchip_in_kernel (vcpu->kvm)) {
@@ -8997,6 +8998,7 @@ static int emulator_fix_hypercall(struct x86_emulate_ctxt *ctxt)static int dm_request_for_irq_injection (struct kvm_vcpu *vcpu) {return vcpu->run->request_interrupt_window && + !vcpu->kvm->arch.irq_injection_disallowed &&likely (!pic_in_kernel (vcpu->kvm)); }
Signature · 1 line
2.25.1
RFC PATCH v3 13/59 KVM: Add max_vcpus field in common 'struct kvm'
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/arm64/include/asm/kvm_host.h | 3 --- arch/arm64/kvm/arm.c | 7 ++----- arch/arm64/kvm/vgic/vgic-init.c | 6 +++--- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 3 ++- 5 files changed, 8 insertions(+), 12 deletions(-)
arch/arm64/include/asm/kvm_host.h +0 -3
@@ -108,9 +108,6 @@ struct kvm_arch {/* VTCR_EL2 value for this VM */ u64 vtcr; -/* The maximum number of vCPUs depends on the used GIC model */ -int max_vcpus; -/* Interrupt controller */ struct vgic_dist vgic;
arch/arm64/kvm/arm.c +2 -5
@@ -153,7 +153,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)kvm_vgic_early_init (kvm);/* The maximum number of VCPUs is limited by the host's GIC model */ - kvm->arch.max_vcpus =kvm_arm_default_max_vcpus (); + kvm->max_vcpus =kvm_arm_default_max_vcpus ();set_default_spectre (kvm);
@@ -228,7 +228,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_MAX_VCPUS :case KVM_CAP_MAX_VCPU_ID :if (kvm) - r = kvm->arch.max_vcpus; + r = kvm->max_vcpus;else r =kvm_arm_default_max_vcpus ();break ;
@@ -304,9 +304,6 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)if (irqchip_in_kernel (kvm) &&vgic_initialized (kvm))return -EBUSY ; -if (id >= kvm->arch.max_vcpus) -return -EINVAL ; -return 0 ; }
arch/arm64/kvm/vgic/vgic-init.c +3 -3
@@ -97,11 +97,11 @@ int kvm_vgic_create(struct kvm *kvm, u32 type) ret =0 ;if (type ==KVM_DEV_TYPE_ARM_VGIC_V2 ) - kvm->arch.max_vcpus =VGIC_V2_MAX_CPUS ; + kvm->max_vcpus =VGIC_V2_MAX_CPUS ;else - kvm->arch.max_vcpus =VGIC_V3_MAX_CPUS ; + kvm->max_vcpus =VGIC_V3_MAX_CPUS ; -if (atomic_read (&kvm->online_vcpus) > kvm->arch.max_vcpus) { +if (atomic_read (&kvm->online_vcpus) > kvm->max_vcpus) { ret = -E2BIG ; goto out_unlock; }
include/linux/kvm_host.h +1 -0
@@ -566,6 +566,7 @@ struct kvm {* and is accessed atomically. */ atomic_t online_vcpus; +int max_vcpus;int created_vcpus;int last_boosted_vcpu;struct list_head vm_list;
virt/kvm/kvm_main.c +2 -1
@@ -1052,6 +1052,7 @@ static struct kvm *kvm_create_vm(unsigned long type)rcuwait_init (&kvm->mn_memslots_update_rcuwait);INIT_LIST_HEAD (&kvm->devices); + kvm->max_vcpus =KVM_MAX_VCPUS ;BUILD_BUG_ON (KVM_MEM_SLOTS_NUM >SHRT_MAX );
@@ -3599,7 +3600,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, u32 id)return -EINVAL ;mutex_lock (&kvm->lock); -if (kvm->created_vcpus ==KVM_MAX_VCPUS ) { +if (kvm->created_vcpus >= kvm->max_vcpus) {mutex_unlock (&kvm->lock);return -EINVAL ; }
Signature · 1 line
2.25.1
Quoted 9 lines · On 11/25/2021 8:19 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Changes from v2:
- update based on patch review
- support TDP MMU
- drop non-essential fetures (ftrace etc.) to reduce patch sizeTODO:
- integrate vm type patch
Hi, everyone plans to review this series,
Please skip patch 14-22, which aer old and a separate series can be found at https://lore.kernel.org/all/[email protected]/
I will post v2 with all the comments from Sean addressed.
thanks,
-Xiaoyao
Quoted 1 line
- integrate unmapping user space mapping
Re: [RFC PATCH v3 08/59] KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
In reply to [email protected]On Wed, Nov 24 2021 at 16:19, isaku yamahata wrote:
What is PV MMIO? This has nothing to do with PV=, aka paravirt because it's used in the tdx exit handler for emulation. Please stop confusing concepts.
Quoted 3 lines
From: Sean Christopherson <[email protected]>
Later kvm_intel.ko will use it.
That sentence is useless, as are the other 'later patches' phrases all over the place. At submission time it's obvious and once this is merged it's not helpful at all.
What's even worse is that this happens right at the beginning of the series and the actual use case is introduced 50 patches later. That's not how exports are done. Exports are next to the use case and in the best case they can be just part of the use case patch.
This is not how fine granular patching works. Just splitting out stupid things into separate patches does not make a proper patch series. It creates an illusion, that's it.
And if I look at the patch which makes actual use of that export then it's just the proof. I'll come to that later.
Thanks,
tglx
Re: [RFC PATCH v3 09/59] KVM: Enable hardware before doing arch VM initialization
In reply to [email protected]Quoted 9 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Swap the order of
hardware_enable_all()andkvm_arch_init_vm()to accommodate Intel's TDX, which needs VMX to be enabled during VM init in order to make SEAMCALLs.This also provides consistent ordering between
kvm_create_vm()andkvm_destroy_vm()with respect to callingkvm_arch_destroy_vm()andhardware_disable_all().
So far so good. This should also explain that there is no downside with that order change especially as this order has been that way for a long time.
Thanks,
tglx
Re: [RFC PATCH v3 12/59] KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
In reply to [email protected]Quoted 5 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Zap only leaf SPTEs when deleting/moving a memslot by default, and add a module param to allow reverting to the old behavior of zapping all SPTEs at all levels and memslots when any memslot is updated.
This changelog describes WHAT the change is doing, which can be seen in the patch itself, but it gives zero justification neither for the change nor for the existance of the module parameter.
Thanks,
tglx
Re: [RFC PATCH v3 13/59] KVM: Add max_vcpus field in common 'struct kvm'
In reply to [email protected]Quoted 1 line · isaku yamahata
From: Sean Christopherson <[email protected]>
Why?
Re: [RFC PATCH v3 14/59] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
In reply to [email protected]Quoted 4 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Add a capability to effectively allow userspace to query what VM types are supported by KVM.
I really don't see why this has to be named legacy. There are enough reasonable use cases which are perfectly fine using the non-encrypted muck. Just because there is a new hyped feature does not make anything else legacy.
Thanks,
tglx
Re: [RFC PATCH v3 15/59] KVM: x86: Introduce "protected guest" concept and block disallowed ioctls
In reply to [email protected]Review
Quoted 4 lines · isaku yamahata
static int
kvm_vcpu_ioctl_smi(struct kvm_vcpu *vcpu){(quoted diff) +1
-0+ /* TODO: use more precise flag */
Why is this still a todo and not implemented properly from the very beginning?
And then you have this:
Quoted 2 lines
(quoted diff) +2
-0+ if (vcpu->arch.guest_state_protected) + return -EINVAL;
...
arch/x86/kvm/x86.c static int kvm_vcpu_ioctl_nmi(struct kvm_vcpu *vcpu) · line 4544
Quoted 3 lines
arch/x86/kvm/x86.c +3
-0+/* TODO: use more precise flag */ +if (vcpu->arch.guest_state_protected) +return -EINVAL ;
and a gazillion of other places. That's beyond lame.
The obvious place to do such a decision is kvm_arch_vcpu_ioctl(), no?
kvm_arch_vcpu_ioctl(.., unsigneg int ioctl, ...)
if (vcpu->arch.guest_state_protected) {
if (!(test_bit(_IOC_NR(ioctl), vcpu->ioctl_allowed))
return -EINVAL;
}is way too simple and obvious, right?
Even if you want more fine grained control, then having an array of flags per ioctl number is way better than sprinkling this protected muck conditions all over the place.
Thanks,
tglx
Re: [RFC PATCH v3 16/59] KVM: x86: Add per-VM flag to disable direct IRQ injection
In reply to [email protected]Quoted 3 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Add a flag to disable IRQ injection, which is not supported by TDX.
...
arch/x86/kvm/x86.c static int kvm_vcpu_ready_for_interrupt_injection(struct kvm_vcpu *vcpu) · line 4511
Quoted 8 lines
arch/x86/kvm/x86.c +2
-1@@ -4506,7 +4506,8 @@ static int kvm_vcpu_ready_for_interrupt_injection(struct kvm_vcpu *vcpu)static int kvm_vcpu_ioctl_interrupt (struct kvm_vcpu *vcpu,struct kvm_interrupt *irq) { -if (irq->irq >=KVM_NR_INTERRUPTS ) +if (irq->irq >=KVM_NR_INTERRUPTS || + vcpu->kvm->arch.irq_injection_disallowed)return -EINVAL ;
That's required here because you forgot to copy & pasta the protect guest condition muck into that ioctl, right?
Re: [RFC PATCH v3 17/59] KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE
In reply to [email protected]arch/x86/kvm/x86.c static int vcpu_ioctl_tpr_access_reporting(struct kvm_vcpu *vcpu, · line 4567
Quoted 9 lines · isaku yamahata
static int kvm_vcpu_ioctl_x86_setup_mce(struct
kvm_vcpu*vcpu,arch/x86/kvm/x86.c +2
-2u64 mcg_cap) { -int r;unsigned bank_num = mcg_cap &0xff , bank; - r = -EINVAL ; +if (vcpu->kvm->arch.mce_injection_disallowed) +return -EINVAL ;
Yet another flag because the copy & pasta orgy did not cover that ioctl. What for?
Re: [RFC PATCH v3 18/59] KVM: x86: Add flag to mark TSC as immutable (for TDX)
In reply to [email protected]Quoted 3 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
The TSC for TDX1 guests is fixed at TD creation time. Add
tsc_immutable
What's a TDX1 guest?
Quoted 3 lines
to reflect that the TSC of the guest cannot be changed in any way, and use it to short circuit all paths that lead to one of the myriad TSC adjustment flows.
I can kinda see the reason for this being valuable on it's own, but in general why does TDX need a gazillion flags to disable tons of different things if _ALL_ these flags are going to be set by for TDX guests anyway?
Seperate flags make only sense when they have a value on their own, i.e. are useful for things outside of TDX. If not they are just useless ballast.
Thanks,
tglx
Re: [RFC PATCH v3 23/59] KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
In reply to [email protected]Quoted 6 lines · isaku yamahata
Let userspace, or in the case of TDX, KVM itself, enable X2APIC even if X2APIC is not reported as supported in the guest's CPU model. KVM generally does not force specific ordering between
ioctls(), e.g. this forces userspace to configure CPUID before MSRs. And for TDX, vCPUs will always run with X2APIC enabled, e.g. KVM will want/need to enable X2APIC from time zero.
This is complete crap. Fix the broken user space and do not add horrible hacks to the kernel.
Thanks,
tglx
Re: [RFC PATCH v3 25/59] KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
In reply to [email protected]Quoted 1 line · isaku yamahata
From: Sean Christopherson <[email protected]>
Yet another changelog with a significant amount of void content.
Re: [RFC PATCH v3 26/59] KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
In reply to [email protected]Quoted 8 lines · isaku yamahata
Add a second
kvm_x86_opshook inkvm_arch_vm_destroy()to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.Note, this knowingly creates a discrepancy in nomenclature for SVM as
svm_vm_teardown()invokesavic_vm_destroy()andsev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.
That's just the wrong way around. Fixup SVM first and then add the TDX muck on top. Stop this 'left to a future patch' nonsense. I know for sure that those future patches never materialize.
The way it works is:
1) Refactor the code to make room for your new functionality in a way that the existing code still works.
2) Add your new muck on top.
Anything else is not acceptable at all.
Thanks,
tglx
Re: [RFC PATCH v3 29/59] KVM: x86: Add option to force LAPIC expiration wait
In reply to [email protected]Quoted 1 line · isaku yamahata
Add an option to skip the IRR check in
kvm_wait_lapic_expire().
Yay for consistency. $Subject says 'Add option to force wait'. Changelog says 'Add option to skip ... check'. Can you make your mind up?
Also the change at hand is not adding an option. It's adding a function argument. You surely can spot the difference between option and function argument, right?
Changelogs are meant to be readable by mere mortals.
Thanks,
tglx
Re: [RFC PATCH v3 30/59] KVM: x86: Add guest_supported_xss placholder
In reply to [email protected]Quoted 7 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Add a per-vcpu placeholder for the support XSS of the guest so that the TDX configuration code doesn't need to hack in manual computation of the supported XSS. KVM XSS enabling is currently being upstreamed, i.e.
guest_supported_xsswill no longer be a placeholder by the time TDX is ready for upstreaming (hopefully).
Yes, hope dies last... Definitely technical useful information for a changelog. There is a reason why notes should go below the --- separator line.
Thanks,
tglx
Re: [RFC PATCH v3 31/59] KVM: x86: Add infrastructure for stolen GPA bits
In reply to [email protected]Quoted 41 lines · isaku yamahata
Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA alising will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.
Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.
To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.
To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.
GFNs (without stolen bits) continue to be used to:
-Specify physical memory by userspace via memslots -Map GPAs to TDP PTEs via RMAP
-Specify dirty tracking and write protection -Look up MTRR types
-Inject async page faultsSince there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN,
kvm_handle_hva()based operations will be applied to both aliases.In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.
Quoted 6 lines
For TLB flushes that are address based, make sure to flush both aliases in the stolen bits case.
Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.
This is a real reasonable and informative changelog. Thanks to Rick for writing this up!
Thanks,
tglx
Re: [RFC PATCH v3 40/59] KVM: VMX: Move NMI/exception handler to common helper
In reply to [email protected]Quoted 1 line · isaku yamahata
From: Sean Christopherson <[email protected]>
Why?
Re: [RFC PATCH v3 41/59] KVM: VMX: Split out guts of EPT violation to common/exposed function
In reply to [email protected]Quoted 1 line · isaku yamahata
From: Sean Christopherson <[email protected]>
Moar why?
Re: [RFC PATCH v3 39/59] KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
In reply to [email protected]Quoted 1 line · isaku yamahata
From: Sean Christopherson <[email protected]>
$subject: s/as param/as function arguments/
param != function argument
It's not rocket science to use the proper terms and to write out words in changelogs and comments instead of using half baken abbreviations.
Precise language matters and especially so for the benefit on non-native speakers and people who are not familiar with a particular corporate slang.
Thanks,
tglx
Re: [RFC PATCH v3 45/59] KVM: VMX: Move setting of EPT MMU masks to common VT-x code
In reply to [email protected]Quoted 1 line · isaku yamahata
From: Sean Christopherson <[email protected]>
Yet another void ...
Re: [RFC PATCH v3 46/59] KVM: VMX: Move register caching logic to common code
In reply to [email protected]Quoted 10 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Move the guts of
vmx_cache_reg()tovt_cache_reg()in preparation for reusing the bulk of the code for TDX, which can access guest state for debug TDs.Use
kvm_x86_ops.cache_reg()inept_update_paging_mode_cr0()rather than trying to exposevt_cache_reg()tovmx.c, even though it means taking a retpoline. The code runs if and only if EPT is enabled but unrestricted guest.
This sentence does not parse because it's not a proper sentence.
Quoted 3 lines
Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.
This one is only significantly better and lacks an explanation what this means for the dumb case.
Thanks,
tglx
Paolo Bonzini
maintainerRe: [RFC PATCH v3 46/59] KVM: VMX: Move register caching logic to common code
Quoted 5 lines · Thomas Gleixner
Quoted 5 lines
Use
kvm_x86_ops.cache_reg()inept_update_paging_mode_cr0()rather than trying to exposevt_cache_reg()tovmx.c, even though it means taking a retpoline. The code runs if and only if EPT is enabled but unrestricted guest.
Quoted 7 lines
This sentence does not parse because it's not a proper sentence.
Quoted 3 lines
Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.
This one is only significantly better and lacks an explanation what this means for the dumb case.
Well, it means a retpoline (see paragraph before). However, I'm not sure why it one wouldn't create a vt.h header with all vt_* functions.
Paolo
Re: [RFC PATCH v3 47/59] KVM: TDX: Define TDCALL exit reason
In reply to [email protected]Quoted 5 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Define the TDCALL exit reason, which is carved out from the VMX exit reason namespace as the TDCALL exit from TDX guest to TDX-SEAM is really just a VM-Exit.
How is this carved out? What's the value of this word salad?
It's simply a new exit reason. Not more, not less. So what?
Quoted 3 lines
Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
I'm pretty sure that it does not take two engineers to add a new exit reason define, but it takes at least two engineers to come up with a convoluted explanation for it.
Thanks,
tglx
Re: [RFC PATCH v3 51/59] KVM: VMX: MOVE GDT and IDT accessors to common code
In reply to [email protected]Quoted 1 line · isaku yamahata
From: Sean Christopherson <[email protected]>
Again: Why?
Re: [RFC PATCH v3 52/59] KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX code
In reply to [email protected]Quoted 1 line · isaku yamahata
From: Sean Christopherson <[email protected]>
Moar missing explanation of the why.
Re: [RFC PATCH v3 49/59] KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
In reply to [email protected]Quoted 4 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Add a macro framework to hide VMX vs. TDX details of VMREAD and VMWRITE so the VMX and TDX can shared common flows, e.g. accessing DTs.
s/shared/share/
Quoted 3 lines
Note, the TDX paths are dead code at this time. There is no great way to deal with the chicken-and-egg scenario of having things in place for TDX without first having TDX.
That's more than obvious and the whole point of building infrastructure in the first place, isn't it?
arch/x86/kvm/vmx/common.h line 27
Quoted 12 lines
arch/x86/kvm/vmx/common.h +12
-0+#ifdef CONFIG_INTEL_TDX_HOST +#define VT_BUILD_VMCS_HELPERS (type, bits, tdbits) \ +static __always_inline type vmread##bits(struct kvm_vcpu *vcpu, \ + unsigned long field) \ +{ \ + if (unlikely(is_td_vcpu(vcpu))) { \ + if (KVM_BUG_ON(!is_debug_td(vcpu), vcpu->kvm)) \ + return 0; \ + return td_vmcs_read##tdbits(to_tdx(vcpu), field); \ + } \ + return vmcs_read##bits(field); \ +} \
New lines exist for a reason to visually separate things and are even possible in macro blocks. This includes the defines.
Aside of that is there any reason why the end of the macro block has to be 3 spaces instead of a tab?
Thanks,
tglx
Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
In reply to [email protected]Quoted 3 lines · isaku yamahata
From: Chao Gao <[email protected]>
$Subject: KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
Which user space are you talking about? This subject line is misleading at best. The unconditional reset is happening when a TDX VM exits because the SEAM firmware enforces this to prevent unformation leaks.
It also does not matter whether this are four or ten MSR. Fact is that the SEAM firmware is buggy because it does not save/restore those MSRs.
So the proper subject line is:
KVM: x86: Add infrastructure to handle MSR corruption by broken TDX firmware
Quoted 10 lines
The TDX module unconditionally reset 4 host MSRs (
MSR_SYSCALL_MASK,MSR_START,MSR_LSTAR,MSR_TSC_AUX) to architectural INIT state on exit from TDX VM to KVM. KVM needs to save their values before TD enter and restore them on exit to userspace.Reuse current
kvm_user_returnmechanism and introduce a function to update cached values and register the user return notifier in this new function.The later patch will use the helper function to save/restore 4 host MSRs.
'The later patch ...' is useless information. Of course there will be a later patch to make use of this which is implied by 'Add infrastructure ...'. Can we please get rid of these useless phrases which have no value at patch submission time and are even more confusing once the pile is merged?
Thanks,
tglx
Paolo Bonzini
maintainerRe: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
In reply to [email protected]Quoted 10 lines · On 11/25/21 01:20, [email protected] wrote:
From: Xiaoyao Li<[email protected]>
Introduce a per-vm variable
initial_tsc_khzto hold the defaulttsc_khzforkvm_arch_vcpu_create().This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.
Signed-off-by Xiaoyao Li<[email protected]>Signed-off-by Isaku Yamahata<[email protected]>
Probably some incorrect squashing happened here, since...
Quoted 5 lines
arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/tdx.c | 2233 +++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 3 +- 3 files changed, 2236 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/vmx/tdx.c
This patch does a bit more than that. :)
Paolo
Paolo Bonzini
maintainerRe: [RFC PATCH v3 28/59] KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
In reply to [email protected]Quoted 8 lines · On 11/25/21 01:20, [email protected] wrote:
From: Sean Christopherson<[email protected]>
Return true for
kvm_vcpu_has_events()if the vCPU has a pending APICv interrupt to support TDX's usage of APICv. Unlike VMX, TDX doesn't have access to vmcs.GUEST_INTR_STATUS and so can't emulate posted interrupts, i.e. needs to generate a posted interrupt and more importantly can't manually move requested interrupts into the vIRR (which it also doesn't have access to).
Does this mean it is impossible to disable APICv on TDX? If so, please add a WARN.
Paolo
Paolo Bonzini
maintainerRe: [RFC PATCH v3 26/59] KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
In reply to Thomas GleixnerQuoted 5 lines · Thomas Gleixner
Quoted 4 lines · isaku yamahata
Add a second
kvm_x86_opshook inkvm_arch_vm_destroy()to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.
Quoted 7 lines
Quoted 4 lines
Note, this knowingly creates a discrepancy in nomenclature for SVM as
svm_vm_teardown()invokesavic_vm_destroy()andsev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.That's just the wrong way around. Fixup SVM first and then add the TDX muck on top. Stop this 'left to a future patch' nonsense. I know for sure that those future patches never materialize.
Or just keep vm_destroy for the "early" destruction, and give a new name to the new hook. It is used to give back the TDCS memory, so perhaps you can call it vm_free?
Paolo
Re: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
In reply to [email protected]Quoted 7 lines · isaku yamahata
From: Xiaoyao Li <[email protected]>
Introduce a per-vm variable
initial_tsc_khzto hold the defaulttsc_khzforkvm_arch_vcpu_create().This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.
So now almost 50 patches after exporting kvm_io_bus_read() I have finally reached the place which makes use of it. But that's just a minor detail compared to this:
Quoted 2 lines
arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/tdx.c | 2233 +++++++++++++++++++++++++++++++
Can you pretty please explain how this massive pile of code is related to the subject line and the change log of this patch?
It takes more than 2000 lines of code to add a new member to struct kvm_arch?
Seriously? This definitely earns an award for the most disconnected changelog ever.
arch/x86/kvm/vmx/tdx.c
Quoted 19 lines
arch/x86/kvm/x86.c | 3 +- 3 files changed, 2236 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/vmx/tdx.carch/x86/include/asm/kvm_host.h +1
-0@@ -1111,6 +1111,7 @@ struct kvm_arch {u64 last_tsc_write;u32 last_tsc_khz;u64 last_tsc_offset; +u32 initial_tsc_khz;u64 cur_tsc_nsec;u64 cur_tsc_write;u64 cur_tsc_offset;arch/x86/kvm/vmx/tdx.c new file +0
-0
< SNIP>
Yes, the above would be the patch which would be expected according to the changelog and the subject line.
And no. Throwing a 2000+ lines patch over the fence which builds up the complete infrastructure for TDX in KVM is not how that works. This patch (aside of the silly changelog) is an unreviewable maze.
So far in this series, there was a continuous build up of infrastructure in reviewable chunks and then you expect a reviewer to digest 2000+ lines at once for no reason and with the most disconneted changelog ever?
This can be done structured and gradually, really.
1) Add the basic infrastructure
2) Add functionality piece by piece
There is no fundamental dependency up to the point where you enable TDX, but there is a fundamental difference of reviewing 2000+ lines of code at once or reviewing a gradual build up of 2000+ lines of code in small pieces with proper changelogs for each of them.
You can argue that my request is unreasonable until you are blue in your face, it's not going to lift my NAK on this.
I've mopped up enough half baken crap in x86/kvm over the years and I have absolutely no interest at all to mop up after you again.
x86/kvm is not a special part of the kernel and neither exempt from general kernel process nor from x86 specific scrunity rules.
That said, I'm stopping the review right here simply because looking at any further changes does not make any sense at all.
Thanks,
tglx
Re: [RFC PATCH v3 26/59] KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
In reply to Paolo BonziniQuoted 10 lines · Paolo Bonzini
Quoted 9 lines · Thomas Gleixner
Quoted 8 lines · isaku yamahata
Add a second
kvm_x86_opshook inkvm_arch_vm_destroy()to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.Note, this knowingly creates a discrepancy in nomenclature for SVM as
svm_vm_teardown()invokesavic_vm_destroy()andsev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.
Quoted 7 lines
Quoted 3 lines
That's just the wrong way around. Fixup SVM first and then add the TDX muck on top. Stop this 'left to a future patch' nonsense. I know for sure that those future patches never materialize.
Or just keep
vm_destroyfor the "early" destruction, and give a new name to the new hook. It is used to give back the TDCS memory, so perhaps you can call itvm_free?
Up to you, but the current approach is bogus. I rather go for a fully symmetric interface and let the various incarnations opt in at the right place. Similar to what cpu hotplug states are implementing.
Thanks,
tglx
Paolo Bonzini
maintainerRe: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
In reply to Thomas GleixnerQuoted 2 lines · Thomas Gleixner
You can argue that my request is unreasonable until you are blue in your face, it's not going to lift my NAK on this.
There's no need for that. I'd be saying the same, and I don't think it's particularly helpful that you made it almost a personal issue.
While in this series there is a separation of changes to existing code vs. new code, what's not clear is _why_ you have all those changes. These are not code cleanups or refactorings that can stand on their own feet; lots of the early patches are actually part of the new functionality. And being in the form of "add an argument here" or "export a function there", it's not really easy (or feasible) to review them without seeing how the new functionality is used, which requires a constant back and forth between early patches and the final 2000 line file.
In some sense, the poor commit messages at the beginning of the series are just a symptom of not having any meat until too late, and then dropping it all at once. There's only so much that you can say about an EXPORT_SYMBOL_GPL, the real thing to talk about is probably the thing that refers to that symbol.
If there are some patches that are actually independent, go ahead and submit them early. But more practically, for the bulk of the changes what you need to do is:
1) incorporate into patch 55 a version of tdx.c that essentially does KVM_BUG_ON or WARN_ON for each function. Temporarily keep the same huge patch that adds the remaining 2000 lines of tdx.c
2) squash the tdx.c stub with patch 44.
3) gather a strace of QEMU starting up a TDX domain.
4) figure out which parts of the code are needed to run until the first ioctl. Make that a first patch.
5) repeat step 4 until you have covered all the code
5) Move the new "KVM: VMX: Add 'main.c' to wrap VMX and TDX" (which also adds the tdx.c stub) as possible in the series.
6) Move each of the new patches as early as possible in the series.
7) Look for candidates for squashing (e.g. commit messages that say it's "used later"; now the use should be very close and the two can be merged). Add to the commit message a note about changes outside VMX.
The resulting series may not be perfect, but it would be a much better starting point for review.
Paolo
Re: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
Paolo,
Quoted 6 lines · Paolo Bonzini
Quoted 2 lines · Thomas Gleixner
You can argue that my request is unreasonable until you are blue in your face, it's not going to lift my NAK on this.
There's no need for that. I'd be saying the same, and I don't think it's particularly helpful that you made it almost a personal issue.
To be entirely clear. It's a personal issue.
Being on the receiving end of a firehose which spills liquid manure for more than 15 years is not a pleasant experience at all.
Being accused to have unreasonable, unactionable and even bizarre review requests (mostly behind the scenes) is not helping either.
I'm dead tired of sitting down for hours explaining basics in lengthy emails over and over just to get yet another experience of being ignored after a couple of months filled with silence.
While I did not do that in this particular context, I read back on the previous incarnations of this crud and found out that you are in the exact same situation.
So what's the difference? Nothing at all.
It's about time that someone speaks up in unambiguous words to make it entirely clear that this is not acceptable at all in the kernel community as a whole and not just because a single person being unreasonable or whatever.
Both of us wasted enough time already to explain how to do it right. There is no point anymore to proliferate any of these political correctness games further.
Thanks,
Thomas
Re: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
In reply to Paolo BonziniPaolo,
Quoted 8 lines · Paolo Bonzini
On 11/25/21 22:05, Thomas Gleixner wrote:
If there are some patches that are actually independent, go ahead and submit them early. But more practically, for the bulk of the changes what you need to do is:1) incorporate into patch 55 a version of
tdx.cthat essentially doesKVM_BUG_ONorWARN_ONfor each function. Temporarily keep the same huge patch that adds the remaining 2000 lines oftdx.c
There is absolutely no reason to populate anything upfront at all.
Why?
Simply because that whole muck cannot work until all pieces are in place.
That's the classic PoC vs. usable code situation. I.e. you know what needs to be done from a PoC point of view and you have to get there by building up the infrastructure piece by piece.
So why would you provide:
handle_A(...) { BUG(); }
..handle_Z(...) { BUG(); }
with all the invocations in the common emulation dispatcher:
handle_stuff(reason)
{
switch(reason) {
case A: return handle_A();
...
case Z: return handle_Z();
default: BUG();
}
};in the first place instead of providing:
1)
handle_stuff(reason)
{
switch(reason) {
default: BUG();
};
}2)
handle_A(...)
{
return do_something_understandable()
}
handle_stuff(reason)
{
switch(reason) {
case A: return handle_A();
default: BUG();
}$Z)
handle_Z(...)
{
return do_something_understandable()
}
handle_stuff(reason)
{
switch(reason) {
case A: return handle_A();
...
case Z: return handle_Z();
default: BUG();
}where each of (1..$Z) is a single patch doing exactly _ONE_ thing at a time and if there is some extra magic required like an additional export for handle_Y() then this export patch is either part of the patch dealing with $Y or just before that.
In both scenarious you cannot boot a TDX guest until you reached $Z, but in the gradual one you and the reviewers have the pleasure of looking at one thing at a time.
No?
Thanks,
tglx
Paolo Bonzini
maintainerRe: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
Quoted 8 lines · Thomas Gleixner
Paolo,
Quoted 5 lines · Paolo Bonzini
On 11/25/21 22:05, Thomas Gleixner wrote:
If there are some patches that are actually independent, go ahead and submit them early. But more practically, for the bulk of the changes what you need to do is:
Quoted 8 lines
Quoted 3 lines
1) incorporate into patch 55 a version of
tdx.cthat essentially doesKVM_BUG_ONorWARN_ONfor each function. Temporarily keep the same huge patch that adds the remaining 2000 lines oftdx.cThere is absolutely no reason to populate anything upfront at all. Why?
Simply because that whole muck cannot work until all pieces are in place.
It can, sort of. It cannot run a complete guest, but it could in principle run a toy guest with a custom userspace, like the ones that make up tools/testing/selftests/kvm. (Note that KVM_BUG_ON marks the VM as bugged but doesn't hang the whole machine).
AMD was working on infrastructure to do this for SEV and SEV-ES.
Quoted 8 lines
So why would you provide:
handle_A(...){BUG(); }
..handle_Z(...){BUG(); }with all the invocations in the common emulation dispatcher:
Quoted 9 lines
handle_stuff(reason)
{
switch(reason) {
case A: return handle_A();
...
case Z: return handle_Z();
default: BUG();
}
};If it's a switch statement that's good, but the common case is more similar to this:
vmx_handle_A(...) { ... }
+tdx_handle_A(...) { ... }
+
+vt_handle_A(...) {
+ if (is_tdx(vcpu->kvm))
+ tdx_handle_A(...);
+ else
+ vmx_handle_A(...);
+}
...
- .handle_A = vmx_handle_A,
+ .handle_A = vt_handle_A,
And you could indeed do it in a single patch, without adding the stub tdx_handle_A upfront. But you would have code that is broken and who knows what the effects would be of calling vmx_handle_A on a TDX virtual machine. It could be an error, or it could be memory corruption.
Quoted 3 lines
In both scenarious you cannot boot a TDX guest until you reached $Z, but in the gradual one you and the reviewers have the pleasure of looking at one thing at a time.
I think both of them are gradual. Not having the stubs might be a little more gradual, but it is a very minor issue for the reviewability of the whole thing.
Paolo
Paolo Bonzini
maintainerRe: [RFC PATCH v3 23/59] KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
In reply to Thomas GleixnerQuoted 10 lines · Thomas Gleixner
Quoted 6 lines · isaku yamahata
Let userspace, or in the case of TDX, KVM itself, enable X2APIC even if X2APIC is not reported as supported in the guest's CPU model. KVM generally does not force specific ordering between
ioctls(), e.g. this forces userspace to configure CPUID before MSRs. And for TDX, vCPUs will always run with X2APIC enabled, e.g. KVM will want/need to enable X2APIC from time zero.This is complete crap. Fix the broken user space and do not add horrible hacks to the kernel.
tl;dr: I agree that it's a userspace issue but "configure CPUID before MSR" is not the issue (in fact QEMU calls KVM_SET_CPUID2 before any call to KVM_SET_MSRS).
We have quite a few other cases in which KVM_GET/SET_MSR is allowed to get/set MSRs in ways that the guests are not allowed to do.
In general, there are several reasons for this:
- simplifying userspace so that it can use the same list of MSRs for all guests (likely, the list that KVM provides with KVM_GET_MSR_INDEX_LIST). For example MSR_TSC_AUX is only exposed to the guest if RDTSCP or
RDPID are available, but the host can always access it. This is usually the reason why host accesses to MSRs override CPUID.
- simplifying userspace so that it does not have to go through the various steps of a state machine; for example, it's okay if userspace goes DISABLED->X2APIC instead of having to do DISABLED->XAPIC->X2APIC.
- allowing userspace to set a reset value, for example overriding the lock bit in MSR_IA32_FEAT_CTL.
- read-only MSRs that are really "CPUID-like", i.e. they give the guest information about processor features (for example the VMX feature MSRs)
- MSRs had some weird limitations that were lifted later by introducing additional MSRs; for example KVM always allows the host to write to the full-width MSR_IA32_PMC0 counters, because they are a saner version of MSR_IA32_PERFCTR0 and there's no reason for userspace to inflict MSR_IA32_PERFCTR0 on userspace.
So the host_initiated check doesn't _necessarily_ count as a horrible hack in the kernel. However, in this case we have a trusted domain without X2APIC. I'm not sure this configuration is clearly bogus. One could imagine special-purpose VMs that don't need interrupts at all. For full guests such as the ones that QEMU runs, I agree with Thomas that userspace must be fixed to enforce x2apic for TDX guests.
Paolo
Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
In reply to Thomas GleixnerQuoted 6 lines · Thomas Gleixner
Quoted 3 lines · isaku yamahata
From: Chao Gao <[email protected]>
$Subject: KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
Which user space are you talking about? This subject line is misleading
Host Ring3.
Quoted 2 lines
at best. The unconditional reset is happening when a TDX VM exits because the SEAM firmware enforces this to prevent unformation leaks.
Yes.
Quoted 2 lines
It also does not matter whether this are four or ten MSR.
Indeed, the number of MSRs doesn't matter.
Quoted 2 lines
Fact is that
the SEAM firmware is buggy because it does not save/restore those MSRs.
It is done deliberately. It gives host a chance to do "lazy" restoration. "lazy" means don't save/restore them on each TD entry/exit but defer restoration to when it is neccesary e.g., when vCPU is scheduled out or when kernel is about to return to Ring3.
Quoted 4 lines
So the proper subject line is:
KVM: x86: Add infrastructure to handle MSR corruption by broken TDX firmware
I rewrote the commit message:
KVM: x86: Allow to update cached values in kvm_user_return_msrs w/o wrmsr
Several MSRs are constant and only used in userspace. But VMs may have different values. KVM uses kvm_set_user_return_msr() to switch to guest's values and leverages user return notifier to restore them when kernel is to return to userspace. In order to save unnecessary wrmsr, KVM also caches the value it wrote to a MSR last time.
TDX module unconditionally resets some of these MSRs to architectural INIT state on TD exit. It makes the cached values in kvm_user_return_msrs are inconsistent with values in hardware. This inconsistency needs to be fixed otherwise, it may mislead kvm_on_user_return() to skip restoring some MSRs to host's values. kvm_set_user_return_msr() can help to correct this case but it is not optimal as it always does a wrmsr. So, introduce a variation of kvm_set_user_return_msr() to update the cached value but skip the wrmsr.
Quoted 9 lines
Quoted 9 lines
The TDX module unconditionally reset 4 host MSRs (
MSR_SYSCALL_MASK,MSR_START,MSR_LSTAR,MSR_TSC_AUX) to architectural INIT state on exit from TDX VM to KVM. KVM needs to save their values before TD enter and restore them on exit to userspace.Reuse current
kvm_user_returnmechanism and introduce a function to update cached values and register the user return notifier in this new function.
Quoted 8 lines
Quoted 2 lines
The later patch will use the helper function to save/restore 4 host MSRs.
'The later patch ...' is useless information. Of course there will be a later patch to make use of this which is implied by 'Add infrastructure ...'. Can we please get rid of these useless phrases which have no value at patch submission time and are even more confusing once the pile is merged?
Of course. Will remove all "later patch" phrases.
Thanks
Chao
Quoted 4 lines
Thanks,
tglx
Paolo Bonzini
maintainerRe: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
Quoted 1 line · Chao Gao
Of course. Will remove all "later patch" phrases.
Chao, see my comment on patch 54, for how to do this. In this case, this patch would be right before the one that enables KVM_RUN on a TDX module.
Paolo
Re: [RFC PATCH v3 16/59] KVM: x86: Add per-VM flag to disable direct IRQ injection
In reply to [email protected]Quoted 6 lines · On Fri, Nov 26, 2021 at 3:44 PM <[email protected]> wrote:
static int
dm_request_for_irq_injection(struct kvm_vcpu *vcpu){ return vcpu->run->request_interrupt_window && + !vcpu->kvm->arch.irq_injection_disallowed && likely(!pic_in_kernel(vcpu->kvm));}
Just judged superficially by the function names, it seems that the logic is better to be put in kvm_cpu_accept_dm_intr() or some deeper function nested in kvm_cpu_accept_dm_intr().
The function name will tell us that the interrupt is not injected because the CPU doesn't accept it. And it will also have an effect that vcpu->run->ready_for_interrupt_injection will always be false which I think is better to have for TDX.
Quoted 4 lines
--
2.25.1
Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
In reply to Chao GaoQuoted 24 lines · Chao Gao
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines · isaku yamahata
From: Chao Gao <[email protected]>
$Subject: KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
Which user space are you talking about? This subject line is misleading
Host Ring3.
Quoted 2 lines
at best. The unconditional reset is happening when a TDX VM exits because the SEAM firmware enforces this to prevent unformation leaks.
Yes.
Quoted 2 lines
It also does not matter whether this are four or ten MSR.
Indeed, the number of MSRs doesn't matter.
Quoted 2 lines
Fact is that
the SEAM firmware is buggy because it does not save/restore those MSRs.
Quoted 8 lines
It is done deliberately. It gives host a chance to do "lazy" restoration. "lazy" means don't save/restore them on each TD entry/exit but defer restoration to when it is neccesary e.g., when vCPU is scheduled out or when kernel is about to return to Ring3.
The TDX module unconditionally reset 4 host MSRs (
MSR_SYSCALL_MASK,MSR_START,MSR_LSTAR,MSR_TSC_AUX) to architectural INIT state on exit from TDX VM to KVM.
I did not find the information in intel-tdx-module-1eas.pdf nor intel-tdx-cpu-architectural-specification.pdf.
Maybe the version I downloaded is outdated.
I guess that the "lazy" restoration mode is not a valid optimization. The SEAM module should restore it to the original value when it tries to reset it to architectural INIT state on exit from TDX VM to KVM since the SEAM module also does it via wrmsr (correct me if not).
If the SEAM module doesn't know "the original value" of the these MSRs, it would be mere an optimization to save an rdmsr in SEAM. But there are a lot of other ways for the host to share the values to SEAM in zero overhead.
Could you provide more information?
Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
Quoted 30 lines · Lai Jiangshan
Quoted 29 lines · Chao Gao
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines · isaku yamahata
From: Chao Gao <[email protected]>
$Subject: KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
Which user space are you talking about? This subject line is misleading
Host Ring3.
Quoted 2 lines
at best. The unconditional reset is happening when a TDX VM exits because the SEAM firmware enforces this to prevent unformation leaks.
Yes.
Quoted 2 lines
It also does not matter whether this are four or ten MSR.
Indeed, the number of MSRs doesn't matter.
Quoted 2 lines
Fact is that
the SEAM firmware is buggy because it does not save/restore those MSRs.It is done deliberately. It gives host a chance to do "lazy" restoration. "lazy" means don't save/restore them on each TD entry/exit but defer restoration to when it is neccesary e.g., when vCPU is scheduled out or when kernel is about to return to Ring3.
Quoted 8 lines
Quoted 3 lines
The TDX module unconditionally reset 4 host MSRs (
MSR_SYSCALL_MASK,MSR_START,MSR_LSTAR,MSR_TSC_AUX) to architectural INIT state on exit from TDX VM to KVM.I did not find the information in intel-tdx-module-1eas.pdf nor intel-tdx-cpu-architectural-specification.pdf.
Maybe the version I downloaded is outdated.
Hi Jiangshan,
Please refer to Table 22.162 MSRs that may be Modified by TDH.VP.ENTER, in section 22.2.40 TDH.VP.ENTER leaf.
Quoted 5 lines
I guess that the "lazy" restoration mode is not a valid optimization. The SEAM module should restore it to the original value when it tries to reset it to architectural INIT state on exit from TDX VM to KVM since the SEAM module also does it via wrmsr (correct me if not).
Correct.
Quoted 3 lines
If the SEAM module doesn't know "the original value" of the these MSRs, it would be mere an optimization to save an rdmsr in SEAM.
Yes. Just a rdmsr is saved in TDX module at the cost of host's restoring a MSR. If restoration (wrmsr) can be done in a lazy fashion or even the MSR isn't used by host, some CPU cycles can be saved.
Quoted 2 lines
But there are a lot of other ways for the host to share the values to SEAM in zero overhead.
I am not sure. Looks it requests a new interface between host and TDX module. I guess one problem is how/when to verify host's inputs in case they are invalid.
Thanks
Chao
Quoted 2 lines
Could you provide more information?
Sean Christopherson
maintainerRe: [RFC PATCH v3 46/59] KVM: VMX: Move register caching logic to common code
In reply to Paolo BonziniQuoted 7 lines · Paolo Bonzini
Quoted 6 lines · Thomas Gleixner
Quoted 5 lines
Use
kvm_x86_ops.cache_reg()inept_update_paging_mode_cr0()rather than trying to exposevt_cache_reg()tovmx.c, even though it means taking a retpoline. The code runs if and only if EPT is enabled but unrestricted guest.This sentence does not parse because it's not a proper sentence.
Heh, supposed to be "... but unrestricted guest is disabled".
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.
This one is only significantly better and lacks an explanation what this means for the dumb case.
Well, it means a retpoline (see paragraph before).
No, the point being made is that, on a CPU that supports Unrestricted Guest (UG), disabling UG without disabling EPT is really, really stupid. UG requires EPT, so disabling EPT _and_ UG is reasonable as there are scenarios where using shadow paging is desirable. But inentionally disabling UG and enabling EPT makes no sense. It forces KVM to emulate non-trivial amounts of guest code and has zero benefits for anything other than testing KVM itself.
Quoted 4 lines
why it one wouldn't create a
vt.hheader with all vt_* functions.Paolo
Sean Christopherson
maintainerRe: [RFC PATCH v3 23/59] KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
In reply to Paolo BonziniQuoted 3 lines · Paolo Bonzini
Quoted 2 lines · Thomas Gleixner
Quoted 1 line · isaku yamahata
Let userspace, or in the case of TDX, KVM itself, enable X2APIC even if
^^^^^^^^^^
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
X2APIC is not reported as supported in the guest's CPU model. KVM generally does not force specific ordering between
ioctls(), e.g. this forces userspace to configure CPUID before MSRs. And for TDX, vCPUs will always run with X2APIC enabled, e.g. KVM will want/need to enable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line
Quoted 1 line
Quoted 1 line
X2APIC from time zero.
^^^^^^^^^^^^^^^^^^^^^
Quoted 7 lines
Quoted 3 lines
This is complete crap. Fix the broken user space and do not add horrible hacks to the kernel.
tl;dr: I agree that it's a userspace issue but "configure CPUID before MSR" is not the issue (in fact QEMU calls
KVM_SET_CPUID2before any call toKVM_SET_MSRS).
Specifically for TDX, it's not a userspace issue. To simplify other checks and to report sane values for KVM_GET_MSRS, KVM forces X2APIC for TDX guests when the vCPU is created, before its exposed to usersepace. The bit about not forcing specific ordering is justification for making the change independent of TDX, i.e. to call out that APIC_BASE is different from every other MSR, and is even inconsistent in its own behavior since illegal transitions are allowed when userspace is stuffing the MSR.
IMO, this patch is valid irrespective of TDX. It's included in the TDX series because TDX support forces the issue.
That said, an alternative for TDX would be do handle this in kvm_lapic_reset() now that the lAPIC RESET flows are consolidated. Back when this patch was first written, that wasn't really an option.
Sean Christopherson
maintainerRe: [RFC PATCH v3 26/59] KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
In reply to Thomas GleixnerQuoted 15 lines · Thomas Gleixner
Quoted 14 lines · Paolo Bonzini
Quoted 12 lines · Thomas Gleixner
Quoted 8 lines · isaku yamahata
Add a second
kvm_x86_opshook inkvm_arch_vm_destroy()to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.Note, this knowingly creates a discrepancy in nomenclature for SVM as
svm_vm_teardown()invokesavic_vm_destroy()andsev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.That's just the wrong way around. Fixup SVM first and then add the TDX muck on top. Stop this 'left to a future patch' nonsense. I know for sure that those future patches never materialize.
Quoted 7 lines
Quoted 3 lines
Or just keep
vm_destroyfor the "early" destruction, and give a new name to the new hook. It is used to give back the TDCS memory, so perhaps you can call itvm_free?Up to you, but the current approach is bogus. I rather go for a fully symmetric interface and let the various incarnations opt in at the right place. Similar to what cpu hotplug states are implementing.
Naming aside, that's what is being done, TDX simply needs two hooks instead of one due to the way KVM handles VM and vCPU destruction. The alternative would be to shove and duplicate what is currently common x86 code into VMX/SVM, which IMO is far worse.
Regarding the naming, I 100% agree SVM should be refactored prior to adding TDX stuff if we choose to go with vm_teardown() and vm_destroy() instead of Paolo's suggestion of vm_destroy() and vm_free(). When this patch/code was originally written, letting SVM become stale was a deliberate choice to reduce conflicts with upstream as we knew the code would live out of tree for quite some time. But that was purely meant to be development "hack", not upstream behavior.
Sean Christopherson
maintainerRe: [RFC PATCH v3 28/59] KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
In reply to Paolo BonziniQuoted 10 lines · Paolo Bonzini
Quoted 8 lines · On 11/25/21 01:20, [email protected] wrote:
From: Sean Christopherson<[email protected]>
Return true for
kvm_vcpu_has_events()if the vCPU has a pending APICv interrupt to support TDX's usage of APICv. Unlike VMX, TDX doesn't have access to vmcs.GUEST_INTR_STATUS and so can't emulate posted interrupts, i.e. needs to generate a posted interrupt and more importantly can't manually move requested interrupts into the vIRR (which it also doesn't have access to).
Quoted 2 lines
Does this mean it is impossible to disable APICv on TDX? If so, please add a WARN.
Yes, APICv is forced.
Rereading this patch, checking only for a pending posted interrupt isn't correct, a pending interrupt that's below the PPR shouldn't be considered a wake event.
A much better approach would be to have vt_sync_pir_to_irr() redirect to a TDX implementation to read the PIR but not update the vIRR, that way common x86 doesn't need to be touched. Hopefully that can be done in a race-free way.
Paolo Bonzini
maintainerRe: [RFC PATCH v3 46/59] KVM: VMX: Move register caching logic to common code
In reply to Sean ChristophersonQuoted 9 lines · Sean Christopherson
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines
Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.
This one is only significantly better and lacks an explanation what this means for the dumb case.
Well, it means a retpoline (see paragraph before).
No, the point being made is that, on a CPU that supports Unrestricted Guest (UG), disabling UG without disabling EPT is really, really stupid.
Yes, I understand that.
Thomas was asking what it means to "Move register caching logic to common code", i.e. what the consequences are. The missing words at the end of the first paragraph didn't make the connection obvious between the extra retpoline and the "dumb case".
Paolo
Sean Christopherson
maintainerRe: [RFC PATCH v3 14/59] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
In reply to Thomas GleixnerQuoted 10 lines · Thomas Gleixner
Quoted 4 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Add a capability to effectively allow userspace to query what VM types are supported by KVM.
I really don't see why this has to be named legacy. There are enough reasonable use cases which are perfectly fine using the non-encrypted muck. Just because there is a new hyped feature does not make anything else legacy.
Yeah, this was brought up in the past. The current proposal is to use KVM_X86_DEFAULT_VM[1], though at one point the plan was to use a generic KVM_VM_TYPE_DEFAULT for all architectures[2], not sure what happened to that idea.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
Sean Christopherson
maintainerRe: [RFC PATCH v3 47/59] KVM: TDX: Define TDCALL exit reason
In reply to Thomas GleixnerQuoted 10 lines · Thomas Gleixner
Quoted 5 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Define the TDCALL exit reason, which is carved out from the VMX exit reason namespace as the TDCALL exit from TDX guest to TDX-SEAM is really just a VM-Exit.
How is this carved out? What's the value of this word salad?
It's simply a new exit reason. Not more, not less. So what?
The changelog is alluding to the fact that KVM should never directly see a TDCALL VM-Exit. For TDX, KVM deals only with "returns" from the TDX-Module. The "carved out" bit is calling out that the transition from SEAM Non-Root (the TDX guest) to SEAM Root (the TDX Module) is actually a VT-x/VMX VM-Exit, e.g. if TDX were somehow implemented without relying on VT-x/VMX, then the TDCALL exit reason wouldn't exist.
Quoted 7 lines
Quoted 3 lines
Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Sean Christopherson <[email protected]>I'm pretty sure that it does not take two engineers to add a new exit reason define, but it takes at least two engineers to come up with a convoluted explanation for it.
Nah, just one ;-)
Sean Christopherson
maintainerRe: [RFC PATCH v3 18/59] KVM: x86: Add flag to mark TSC as immutable (for TDX)
In reply to Thomas GleixnerQuoted 6 lines · Thomas Gleixner
Quoted 3 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
The TSC for TDX1 guests is fixed at TD creation time. Add
tsc_immutableWhat's a TDX1 guest?
The "revision 1.0" version of TDX. Some of these patches use "TDX1" to identify behaviors that may not hold true in future iterations of TDX, and also to highlight things that are dictated by the spec, e.g. some of the guest TSC frequency values. For this patch in particular, there's probably no need to differentiate TDX1 vs. TDX, the qualification was more for cases where KVM needs to define magic values to adhere to the spec, e.g. to make it clear that the magic values aren't made up by KVM.
Quoted 4 lines
Quoted 3 lines
to reflect that the TSC of the guest cannot be changed in any way, and use it to short circuit all paths that lead to one of the myriad TSC adjustment flows.
Quoted 8 lines
I can kinda see the reason for this being valuable on it's own, but in general why does TDX need a gazillion flags to disable tons of different things if _ALL_ these flags are going to be set by for TDX guests anyway?
Seperate flags make only sense when they have a value on their own, i.e. are useful for things outside of TDX. If not they are just useless ballast.
SEV-SNP and TDX have different, but overlapping, restrictions. And SEV-ES also shares most SEV-SNP's restrictions. TDX guests that can be debugged and/or profiled also have different restrictions, though I forget if any of these flags would be affected.
The goal with individual flags is to avoid seemingly arbitrary is_snp_guest() and is_tdx_guest() checks throughout common x86 code, e.g. to avoid confusion over why KVM does X for TDX but Y for SNP. And I personally find it easer to audit KVM behavior with respect to the SNP/TDX specs if the non-obvious restrictions are explicitly set when the VM is created.
For some of the flags, there's also hope that future iterations of TDX will remove some of the restrictions, though that's more of a bonus than a direct justification for adding individual flags.
Sean Christopherson
maintainerRe: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
In reply to Paolo BonziniQuoted 30 lines · Paolo Bonzini
Quoted 2 lines · Thomas Gleixner
You can argue that my request is unreasonable until you are blue in your face, it's not going to lift my NAK on this.
There's no need for that. I'd be saying the same, and I don't think it's particularly helpful that you made it almost a personal issue.
While in this series there is a separation of changes to existing code vs. new code, what's not clear is _why_ you have all those changes. These are not code cleanups or refactorings that can stand on their own feet; lots of the early patches are actually part of the new functionality. And being in the form of "add an argument here" or "export a function there", it's not really easy (or feasible) to review them without seeing how the new functionality is used, which requires a constant back and forth between early patches and the final 2000 line file.
In some sense, the poor commit messages at the beginning of the series are just a symptom of not having any meat until too late, and then dropping it all at once. There's only so much that you can say about an
EXPORT_SYMBOL_GPL, the real thing to talk about is probably the thing that refers to that symbol.If there are some patches that are actually independent, go ahead and submit them early. But more practically, for the bulk of the changes what you need to do is:
1) incorporate into patch 55 a version of
tdx.cthat essentially doesKVM_BUG_ONorWARN_ONfor each function. Temporarily keep the same huge patch that adds the remaining 2000 lines oftdx.c
Quoted 6 lines
2) squash the
tdx.cstub with patch 44.3) gather a strace of QEMU starting up a TDX domain.
4) figure out which parts of the code are needed to run until the first ioctl. Make that a first patch.
Hmm, I don't think this approach will work as well as it did for SEV when applied
at a per-ioctl granuarity, I suspect several patches will end up quite large. I
completely agree with the overall idea, but I'd encourage the TDX folks to have a finer granularity where it makes sense, e.g. things like the x2APIC behavior, immutable TSC, memory management, etc... can probably be sliced up into separate patches.
Quoted 10 lines
5) repeat step 4 until you have covered all the code
5) Move the new "KVM: VMX: Add '
main.c' to wrap VMX and TDX" (which also adds thetdx.cstub) as possible in the series.6) Move each of the new patches as early as possible in the series.
7) Look for candidates for squashing (e.g. commit messages that say it's "used later"; now the use should be very close and the two can be merged). Add to the commit message a note about changes outside VMX.
Generally speaking, I agree. For the flag exposion, I 100% agree that setting the flag in TDX, adding it in x86 is best done in a signal patch, and handling all side effects is best done in a single patch.
But for things like letting debug TDs access registers, I would prefer not to actually squash the two (or more) patches. I agree that two related patches need to be contiguous in the series, but I'd prefer that things with non-trivial changes, especially in common code, are kept separate.
Quoted 4 lines
The resulting series may not be perfect, but it would be a much better starting point for review.
Paolo
Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
In reply to Chao GaoQuoted 9 lines · Chao Gao
Quoted 4 lines
I did not find the information in intel-tdx-module-1eas.pdf nor intel-tdx-cpu-architectural-specification.pdf.
Maybe the version I downloaded is outdated.
Hi Jiangshan,
Please refer to Table 22.162 MSRs that may be Modified by TDH.VP.ENTER, in section 22.2.40 TDH.VP.ENTER leaf.
No file in this link:
https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html
has chapter 22.
Quoted 10 lines
Quoted 6 lines
I guess that the "lazy" restoration mode is not a valid optimization. The SEAM module should restore it to the original value when it tries to reset it to architectural INIT state on exit from TDX VM to KVM since the SEAM module also does it via wrmsr (correct me if not).
Correct.
Quoted 1 line
Quoted 6 lines
Quoted 2 lines
If the SEAM module doesn't know "the original value" of the these MSRs, it would be mere an optimization to save an rdmsr in SEAM.
Yes. Just a rdmsr is saved in TDX module at the cost of host's restoring a MSR. If restoration (wrmsr) can be done in a lazy fashion or even the MSR isn't used by host, some CPU cycles can be saved.
But it adds overall overhead because the wrmsr in TDX module can't be skipped while the unneeded potential overhead of wrmsr is added in user return path.
If TDX module restores the original MSR value, the host hypervisor doesn't need to step in.
I think I'm reviewing the code without the code. It is definitely wrong design to (ab)use the host's user-return-msr mechanism.
Quoted 8 lines
Quoted 3 lines
But there are a lot of other ways for the host to share the values to SEAM in zero overhead.
I am not sure. Looks it requests a new interface between host and TDX module. I guess one problem is how/when to verify host's inputs in case they are invalid.
If the requirement of "lazy restoration" is being added (not seen in the published document yet), you are changing the ABI between the host and the TDX module.
Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
Quoted 7 lines · Lai Jiangshan
Quoted 5 lines · Chao Gao
Quoted 4 lines
I did not find the information in intel-tdx-module-1eas.pdf nor intel-tdx-cpu-architectural-specification.pdf.
Maybe the version I downloaded is outdated.
Quoted 8 lines
Quoted 4 lines
Hi Jiangshan,
Please refer to Table 22.162 MSRs that may be Modified by TDH.VP.ENTER, in section 22.2.40 TDH.VP.ENTER leaf.
No file in this link:
https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html
has chapter 22.
Quoted 14 lines
Quoted 14 lines
Quoted 7 lines
I guess that the "lazy" restoration mode is not a valid optimization. The SEAM module should restore it to the original value when it tries to reset it to architectural INIT state on exit from TDX VM to KVM since the SEAM module also does it via wrmsr (correct me if not).
Correct.
Quoted 3 lines
If the SEAM module doesn't know "the original value" of the these MSRs, it would be mere an optimization to save an rdmsr in SEAM.
Quoted 7 lines
Quoted 3 lines
Yes. Just a rdmsr is saved in TDX module at the cost of host's restoring a MSR. If restoration (wrmsr) can be done in a lazy fashion or even the MSR isn't used by host, some CPU cycles can be saved.
But it adds overall overhead because the wrmsr in TDX module can't be skipped while the unneeded potential overhead of wrmsr is added in user return path.
It is a trade-off. In this way, TDX module needn't save some host MSRs. Of course, host has to restore those MSRs. the overall impact depends on the frequency of restoring: on every exit from TD VM, on vCPU being scheduled out or on exit to userspace. In the last two cases, it is supposed to have less overhead than saving host MSRs and restoring them on every exit from TD VM.
Quoted 6 lines
If TDX module restores the original MSR value, the host hypervisor doesn't need to step in.
I think I'm reviewing the code without the code. It is definitely wrong design to (ab)use the host's user-return-msr mechanism.
I cannot follow.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
But there are a lot of other ways for the host to share the values to SEAM in zero overhead.
Quoted 8 lines
Quoted 4 lines
I am not sure. Looks it requests a new interface between host and TDX module. I guess one problem is how/when to verify host's inputs in case they are invalid.
If the requirement of "lazy restoration" is being added (not seen in the published document yet), you are changing the ABI between the host and the TDX module.
No, it is already documented in public spec.
TDX module spec just says some MSRs are reset to INIT state by TDX module (un)conditionally during TD exit. When to restore these MSRs to host's values is decided by host.
Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space
Quoted 2 lines · Chao Gao
No, it is already documented in public spec.
In intel-tdx-module-1.5-base-spec-348549001.pdf, page 79 "recoverability hist", I think it is a "hint".
Quoted 5 lines
TDX module spec just says some MSRs are reset to INIT state by TDX module (un)conditionally during TD exit. When to restore these MSRs to host's values is decided by host.
Sigh, it is quite a common solution to reset a register to a default value here, for example in VMX, the host GDT.limit and host TSS.limit is reset after VMEXIT, so load_fixmap_gdt() and invalidate_tss_limit() have to be called in host in vmx_vcpu_put().
Off-topic: Is it a good idea to also put load_fixmap_gdt() and invalidate_tss_limit() in user-return-notfier?
Sean Christopherson
maintainerQuoted 1 line · On Wed, Nov 24, 2021, [email protected] wrote:
- drop load/initialization of TDX module
So what's the plan for loading and initializing TDX modules?
Quoted 4 lines · On Tue, 30 Nov 2021 18:51:52 +0000 Sean Christopherson wrote:
Quoted 1 line · On Wed, Nov 24, 2021, [email protected] wrote:
- drop load/initialization of TDX module
So what's the plan for loading and initializing TDX modules?
Hi Sean,
Although I don't quite understand what does Isaku mean here (I thought loading/initializing TDX module was never part of TDX KVM series), for this part we are working internally to improve the quality and finalize the code, but currently I don't have ETA of being able to send patches out, but we are trying to send out asap. Sorry this is what I can say for now : (
Paolo Bonzini
maintainerQuoted 4 lines · Sean Christopherson
Quoted 1 line · On Wed, Nov 24, 2021,[email protected] wrote:
- drop load/initialization of TDX module
So what's the plan for loading and initializing TDX modules?
The latest news I got are that Intel has an EFI application that loads it, so loading it from Linux and updating it at runtime can be punted to later.
Paolo
Quoted 11 lines · Kai Huang
Quoted 4 lines · On Tue, 30 Nov 2021 18:51:52 +0000 Sean Christopherson wrote:
Quoted 1 line · On Wed, Nov 24, 2021, [email protected] wrote:
- drop load/initialization of TDX module
So what's the plan for loading and initializing TDX modules?
Although I don't quite understand what does Isaku mean here (I thought loading/initializing TDX module was never part of TDX KVM series), for this part we are working internally to improve the quality and finalize the code, but currently I don't have ETA of being able to send patches out, but we are trying to send out asap. Sorry this is what I can say for now : (
v1/v2 has it. Anyway The plan is what Kai said. The code will reside in the x86 common directory instead of kvm.
The only dependency between the part of loading/initializing TDX module and TDX KVM is only single function to get the info about the TDX module.
Signature · 1 line
Isaku Yamahata <[email protected]>
Sean Christopherson
maintainerQuoted 9 lines · Isaku Yamahata
Quoted 6 lines · Kai Huang
Quoted 4 lines · On Tue, 30 Nov 2021 18:51:52 +0000 Sean Christopherson wrote:
Quoted 1 line · On Wed, Nov 24, 2021, [email protected] wrote:
- drop load/initialization of TDX module
So what's the plan for loading and initializing TDX modules?
Quoted 7 lines
Quoted 5 lines
Although I don't quite understand what does Isaku mean here (I thought loading/initializing TDX module was never part of TDX KVM series), for this part we are working internally to improve the quality and finalize the code, but currently I don't have ETA of being able to send patches out, but we are trying to send out asap. Sorry this is what I can say for now : (
v1/v2 has it.
No, v1 had support for the old architecture where SEAMLDR was a single ACM, it did not support the new split persistent/non-persistent architcture. v2 didn't have support for either architecture.
Quoted 2 lines
Anyway The plan is what Kai said. The code will reside in the x86 common directory instead of kvm.
But what's the plan at a higher level? Will the kernel load the ACM or is that done by firmware? If it's done by firmware, which entity is responsibile for loading the TDX module? If firmware loads the module, what's the plan for upgrading the module without a reboot? When will the kernel initialize the module, regardless of who loads it?
All of those unanswered questions make it nigh impossible to review the KVM support because the code organization and APIs provided will differ based on how the kernel handles loading and initializing the TDX module.
Re: [RFC PATCH v3 14/59] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
In reply to Sean ChristophersonQuoted 12 lines · Sean Christopherson
Quoted 10 lines · Thomas Gleixner
Quoted 4 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Add a capability to effectively allow userspace to query what VM types are supported by KVM.
I really don't see why this has to be named legacy. There are enough reasonable use cases which are perfectly fine using the non-encrypted muck. Just because there is a new hyped feature does not make anything else legacy.
Quoted 6 lines
Yeah, this was brought up in the past. The current proposal is to use KVM_X86_DEFAULT_VM[1], though at one point the plan was to use a generic
KVM_VM_TYPE_DEFAULTfor all architectures[2], not sure what happened to that idea.[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
Currently <feature>_{unsupported, disallowed} are added and the check is
sprinkled and warn in the corresponding low level tdx code. It helped to detect dubious behavior of guest or qemu.
The other approach is to silently ignore them (SMI, INIT, IRQ etc) without such check. The pros is, the code would be simpler and it's what SEV does today. the cons is, it would bes hard to track down such cases and the user would be confused. For example, when user requests reset/SMI, it's silently ignored. The some check would still be needed.
Any thoughts?
Signature · 1 line
Isaku Yamahata <[email protected]>
Quoted 10 lines · On Wed, 1 Dec 2021 16:05:32 +0100 Paolo Bonzini wrote:
Quoted 4 lines · Sean Christopherson
Quoted 1 line · On Wed, Nov 24, 2021,[email protected] wrote:
- drop load/initialization of TDX module
So what's the plan for loading and initializing TDX modules?
The latest news I got are that Intel has an EFI application that loads it, so loading it from Linux and updating it at runtime can be punted to later.
Yes we are heading this approach.
Quoted 10 lines
Quoted 4 lines
Anyway The plan is what Kai said. The code will reside in the x86 common directory instead of kvm.
But what's the plan at a higher level? Will the kernel load the ACM or is that done by firmware? If it's done by firmware, which entity is responsibile for loading the TDX module? If firmware loads the module, what's the plan for upgrading the module without a reboot? When will the kernel initialize the module, regardless of who loads it?
The UEFI loads both ACM and TDX module before booting into kernel by using UEFI tool. The runtime update is pushed out for future support. One goal of this is to reduce the code size so that it can be reviewed more easily and quickly.
And yes kernel will initialize the TDX module. The direction we are heading is to allow to defer TDX module initialization when TDX is truly needed, i.e. When KVM is loaded with TDX support, or first TD is created. The code will basically still reside in host kernel, provided as functions, etc. And at first stage, KVM will call those functions to initialize TDX when needed.
The advantage of this approach is it provides more flexibility: the TDX module initialization code can be reused by future TDX runtime update, etc. And with only initializing TDX in KVM, the host kernel doesn't need to handle entering VMX operation, etc. It can be introduced later when needed.
Quoted 4 lines
All of those unanswered questions make it nigh impossible to review the KVM support because the code organization and APIs provided will differ based on how the kernel handles loading and initializing the TDX module.
I think theoretically loading/initializing module should be quite independent from KVM series, but yes in practice the APIs matter, but I also don't expect this will reduce the ability to review KVM series a lot as RFC.
Anyway sending out host kernel patches is our top priority now and we are trying to do asap.
Sean Christopherson
maintainerRe: [RFC PATCH v3 14/59] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
In reply to Isaku YamahataQuoted 19 lines · Isaku Yamahata
Quoted 16 lines · Sean Christopherson
Quoted 10 lines · Thomas Gleixner
Quoted 4 lines · isaku yamahata
From: Sean Christopherson <[email protected]>
Add a capability to effectively allow userspace to query what VM types are supported by KVM.
I really don't see why this has to be named legacy. There are enough reasonable use cases which are perfectly fine using the non-encrypted muck. Just because there is a new hyped feature does not make anything else legacy.
Yeah, this was brought up in the past. The current proposal is to use KVM_X86_DEFAULT_VM[1], though at one point the plan was to use a generic
KVM_VM_TYPE_DEFAULTfor all architectures[2], not sure what happened to that idea.
Quoted 6 lines
Quoted 2 lines
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/Currently <feature>_{unsupported, disallowed} are added and the check is
sprinkled and warn in the corresponding low level tdx code. It helped to detect dubious behavior of guest or qemu.
KVM shouldn't log a message or WARN unless the issue is detected at a late sanity check, i.e. where failure indicates a KVM bug. Other than that, I agree that KVM should reject ioctls() that directly violate the rules of a confidential VM with an appropriate error code. I don't think KVM should reject everything though, e.g. if the guest attempts to send an SMI, dropping the request on the floor is the least awful option because we can't communicate an error to the guest without making up our own architecture, and exiting to userspace with -EINVAL from deep in KVM would be both painful to implement and an overreaction since doing so would likely kill the guest.
Quoted 9 lines
The other approach is to silently ignore them (SMI, INIT, IRQ etc) without such check. The pros is, the code would be simpler and it's what SEV does today. the cons is, it would bes hard to track down such cases and the user would be confused. For example, when user requests reset/SMI, it's silently ignored. The some check would still be needed.
Any thoughts?Signature · 1 line
Isaku Yamahata <[email protected]>