Skip to contentSkip to conversation

KVM: X86: TDX support

RFC PATCH In review [email protected] 9 participants 123 messages Nov 25, 2021 – Dec 3, 2021

Subsystems (8)
Documentation x86 KVM perf events +4 more subsystems
lore
Share & tools
lore
#0 lore raw

v3 cover KVM: X86: TDX support

From: Isaku Yamahata <[email protected]>

Changes from v2:
- update based on patch review
- support TDP MMU
- drop non-essential fetures (ftrace etc.) to reduce patch size

TODO:
- integrate vm type patch
- integrate unmapping user space mapping

---
* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform. [1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.

* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.

- 01-13: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 14-30: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 31-38: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 39-54: refactoring KVM
- 55: main patch to add "basic" support for building/running TDX.
- 56-57: TDP MMU support
- 58: support TDX hypercall, GetQuote and SetupEventNotifyInterrupt, that
requires qemu help
- 59: Documentation

* Missing features
Those major features are intentionally missing from this patch series to keep this patch series small. They are addressed as independent patch series.

- qemu gdb stub support
- Large page support
- guest PMU support
- and more

Changes from v1:
- rebase to v5.13
- drop load/initialization of TDX module
- catch up the update of related specifications.
- rework on C-wrapper function to invoke seamcall
- various code clean up

[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAS
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf
[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVF

Chao Gao (1):
KVM: x86: Add a helper function to restore 4 host MSRs on exit to user
space

Isaku Yamahata (9):
x86/mktme: move out MKTME related constatnts/macro to msr-index.h x86/mtrr: mask out keyid bits from variable mtrr mask register
KVM: TDX: Define TDX architectural definitions
KVM: TDX: add a helper function for kvm to call seamcall
KVM: TDX: Add helper functions to print TDX SEAMCALL error
KVM: Add per-VM flag to mark read-only memory as unsupported
KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
KVM: TDX: exit to user space on GET_QUOTE,
SETUP_EVENT_NOTIFY_INTERRUPT
Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)

Kai Huang (3):
KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level
routes
KVM: TDX: Protect private mapping related SEAMCALLs with spinlock KVM, x86/mmu: Support TDX private mapping for TDP MMU

Rick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bits

Sean Christopherson (44):
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Split core of hypercall emulation to helper function
KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO
KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
KVM: Add max_vcpus field in common 'struct kvm'
KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
KVM: x86: Introduce "protected guest" concept and block disallowed
ioctls
KVM: x86: Add per-VM flag to disable direct IRQ injection
KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE
KVM: x86: Add flag to mark TSC as immutable (for TDX)
KVM: Add per-VM flag to disable dirty logging of memslots for TDs
KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()
KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched
behavior
KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: x86: Add guest_supported_xss placholder
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Ignore bits 63 and 62 when checking for "present" SPTEs
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()
KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
KVM: VMX: Move NMI/exception handler to common helper
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Define EPT Violation architectural bits
KVM: VMX: Define VMCS encodings for shared EPT pointer
KVM: VMX: Add 'main.c' to wrap VMX and TDX
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: VMX: Move register caching logic to common code
KVM: TDX: Define TDCALL exit reason
KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h
KVM: VMX: MOVE GDT and IDT accessors to common code
KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX
code
KVM: TDX: Add "basic" support for building and running Trust Domains

Xiaoyao Li (1):
KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

 Documentation/virt/kvm/api.rst        |    9 +-
 Documentation/virt/kvm/intel-tdx.rst  |  359 ++++
 arch/arm64/include/asm/kvm_host.h     |    3 -
 arch/arm64/kvm/arm.c                  |    7 +-
 arch/arm64/kvm/vgic/vgic-init.c       |    6 +-
 arch/x86/events/intel/ds.c            |    1 +
 arch/x86/include/asm/kvm-x86-ops.h    |   11 +
 arch/x86/include/asm/kvm_host.h       |   63 +-
 arch/x86/include/asm/msr-index.h      |   16 +
 arch/x86/include/asm/vmx.h            |    6 +
 arch/x86/include/uapi/asm/kvm.h       |   60 +
 arch/x86/include/uapi/asm/vmx.h       |    7 +-
 arch/x86/kernel/cpu/intel.c           |   14 -
 arch/x86/kernel/cpu/mtrr/mtrr.c       |    9 +
 arch/x86/kvm/Makefile                 |    6 +-
 arch/x86/kvm/ioapic.c                 |    4 +
 arch/x86/kvm/irq_comm.c               |   13 +-
 arch/x86/kvm/lapic.c                  |    7 +-
 arch/x86/kvm/lapic.h                  |    2 +-
 arch/x86/kvm/mmu.h                    |   29 +-
 arch/x86/kvm/mmu/mmu.c                |  667 ++++++-
 arch/x86/kvm/mmu/mmu_internal.h       |   12 +
 arch/x86/kvm/mmu/paging_tmpl.h        |   32 +-
 arch/x86/kvm/mmu/spte.c               |   15 +-
 arch/x86/kvm/mmu/spte.h               |   51 +-
 arch/x86/kvm/mmu/tdp_iter.h           |    2 +-
 arch/x86/kvm/mmu/tdp_mmu.c            |  544 +++++-
 arch/x86/kvm/mmu/tdp_mmu.h            |   15 +-
 arch/x86/kvm/svm/svm.c                |   13 +-
 arch/x86/kvm/vmx/common.h             |  178 ++
 arch/x86/kvm/vmx/main.c               | 1152 ++++++++++++
 arch/x86/kvm/vmx/posted_intr.c        |    6 +
 arch/x86/kvm/vmx/seamcall.h           |  116 ++
 arch/x86/kvm/vmx/tdx.c                | 2437 +++++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.h                |  290 +++
 arch/x86/kvm/vmx/tdx_arch.h           |  239 +++
 arch/x86/kvm/vmx/tdx_errno.h          |  111 ++
 arch/x86/kvm/vmx/tdx_error.c          |   53 +
 arch/x86/kvm/vmx/tdx_ops.h            |  224 +++
 arch/x86/kvm/vmx/tdx_stubs.c          |   50 +
 arch/x86/kvm/vmx/vmenter.S            |  146 ++
 arch/x86/kvm/vmx/vmx.c                |  689 ++-----
 arch/x86/kvm/vmx/x86_ops.h            |  203 ++
 arch/x86/kvm/x86.c                    |  276 ++-
 include/linux/kvm_host.h              |    5 +
 include/uapi/linux/kvm.h              |   59 +
 tools/arch/x86/include/uapi/asm/kvm.h |   55 +
 tools/include/uapi/linux/kvm.h        |    2 +
 virt/kvm/kvm_main.c                   |   34 +-
 49 files changed, 7469 insertions(+), 839 deletions(-)
 create mode 100644 Documentation/virt/kvm/intel-tdx.rst
 create mode 100644 arch/x86/kvm/vmx/common.h
 create mode 100644 arch/x86/kvm/vmx/main.c
 create mode 100644 arch/x86/kvm/vmx/seamcall.h
 create mode 100644 arch/x86/kvm/vmx/tdx.c
 create mode 100644 arch/x86/kvm/vmx/tdx.h
 create mode 100644 arch/x86/kvm/vmx/tdx_arch.h
 create mode 100644 arch/x86/kvm/vmx/tdx_errno.h
 create mode 100644 arch/x86/kvm/vmx/tdx_error.c
 create mode 100644 arch/x86/kvm/vmx/tdx_ops.h
 create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c
 create mode 100644 arch/x86/kvm/vmx/x86_ops.h
Signature · 1 line
2.25.1
#2 lore raw

RFC PATCH v3 2/59 x86/mtrr: mask out keyid bits from variable mtrr mask register

From: Isaku Yamahata <[email protected]>

This is a preparation for TDX support. TDX repurposes high bits of physcial address to private key ID similarly to MKTME. IA32_TME_ACTIVATE.MK_TME_KEYID_BITS has same meaning for both TDX disabled/enable for compatibility.

MTRR calculates mask based on available physical address bits. MKTME repurpose high bit of physical address to key id for key id. CPUID MAX_PA remains same and the bits stolen for key id is controlled IA32_TME_ACTIVATE MSR bit 35:32. Because Key ID bits shouldn't affects memory cachability, MTRR mask should exclude bits repourposed for Key ID. It's OS responsibility to maintain cache coherency. detect_tme @ arch/x86/kernel/cpu/intel.c detects tme and destract it from total usable physical bits. This patch adds same logic needed for MTRR.

---
 arch/x86/kernel/cpu/mtrr/mtrr.c | 9 +++++++++
 1 file changed, 9 insertions(+)
arch/x86/kernel/cpu/mtrr/mtrr.c +9 -0
index 2746cac9d8a9..79eaf6ed20a6 100644
@@ -713,6 +713,15 @@ void __init mtrr_bp_init(void)
 			     boot_cpu_data.x86_stepping == 0x4))
 				phys_addr = 36;
 
+			if (boot_cpu_has(X86_FEATURE_TME)) {
+				u64 tme_activate;
+
+				rdmsrl(MSR_IA32_TME_ACTIVATE, tme_activate);
+				if (TME_ACTIVATE_LOCKED(tme_activate) &&
+				    TME_ACTIVATE_ENABLED(tme_activate)) {
+					phys_addr -= TME_ACTIVATE_KEYID_BITS(tme_activate);
+				}
+			}
 			size_or_mask = SIZE_OR_MASK_BITS(phys_addr);
 			size_and_mask = ~size_or_mask & 0xfffff00000ULL;
 		} else if (boot_cpu_data.x86_vendor == X86_VENDOR_CENTAUR &&
Signature · 1 line
2.25.1
#3 lore raw

RFC PATCH v3 3/59 KVM: TDX: Define TDX architectural definitions

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

Define architectural definitions for KVM to issue TDX SEAMCALLs.

Structures and values that are architecturally defined in the TDX module specifications the chapter of ABI Reference

---
 arch/x86/kvm/vmx/tdx_arch.h | 219 ++++++++++++++++++++++++++++++++++++
 1 file changed, 219 insertions(+)
 create mode 100644 arch/x86/kvm/vmx/tdx_arch.h
arch/x86/kvm/vmx/tdx_arch.h new file +219 -0
new file mode 100644
index 000000000000..f57f9bfb7007
@@ -0,0 +1,219 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* architectural constants/data definitions for TDX SEAMCALLs */
+
+#ifndef __KVM_X86_TDX_ARCH_H
+#define __KVM_X86_TDX_ARCH_H
+
+#include <linux/types.h>
+
+/*
+ * TDX SEAMCALL API function leaves
+ */
+#define TDH_VP_ENTER			0
+#define TDH_MNG_ADDCX			1
+#define TDH_MEM_PAGE_ADD		2
+#define TDH_MEM_SEPT_ADD		3
+#define TDH_VP_ADDCX			4
+#define TDH_MEM_PAGE_AUG		6
+#define TDH_MEM_RANGE_BLOCK		7
+#define TDH_MNG_KEY_CONFIG		8
+#define TDH_MNG_CREATE			9
+#define TDH_VP_CREATE			10
+#define TDH_MNG_RD			11
+#define TDH_MEM_RD			12
+#define TDH_MNG_WR			13
+#define TDH_MEM_WR			14
+#define TDH_MEM_PAGE_DEMOTE		15
+#define TDH_MR_EXTEND			16
+#define TDH_MR_FINALIZE			17
+#define TDH_VP_FLUSH			18
+#define TDH_MNG_VPFLUSHDONE		19
+#define TDH_MNG_KEY_FREEID		20
+#define TDH_MNG_INIT			21
+#define TDH_VP_INIT			22
+#define TDH_MEM_PAGE_PROMOTE		23
+#define TDH_PHYMEM_PAGE_RDMD		24
+#define TDH_MEM_SEPT_RD			25
+#define TDH_VP_RD			26
+#define TDH_MNG_KEY_RECLAIMID		27
+#define TDH_PHYMEM_PAGE_RECLAIM		28
+#define TDH_MEM_PAGE_REMOVE		29
+#define TDH_MEM_SEPT_REMOVE		30
+#define TDH_MEM_TRACK			38
+#define TDH_MEM_RANGE_UNBLOCK		39
+#define TDH_PHYMEM_CACHE_WB		40
+#define TDH_PHYMEM_PAGE_WBINVD		41
+#define TDH_MEM_SEPT_WR			42
+#define TDH_VP_WR			43
+#define TDH_SYS_LP_SHUTDOWN		44
+
+#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO		0x10000
+#define TDG_VP_VMCALL_MAP_GPA				0x10001
+#define TDG_VP_VMCALL_GET_QUOTE				0x10002
+#define TDG_VP_VMCALL_REPORT_FATAL_ERROR		0x10003
+#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT	0x10004
+
+/* TDX control structure (TDR/TDCS/TDVPS) field access codes */
+#define TDX_CLASS_SHIFT			56
+#define TDX_FIELD_MASK			GENMASK_ULL(31, 0)
+
+#define BUILD_TDX_FIELD(class, field)	\
+	(((u64)(class) << TDX_CLASS_SHIFT) | ((u64)(field) & TDX_FIELD_MASK))
+
+/* @field is the VMCS field encoding */
+#define TDVPS_VMCS(field)		BUILD_TDX_FIELD(0, (field))
+
+/*
+ * @offset is the offset (in bytes) from the beginning of the architectural
+ * virtual APIC page.
+ */
+#define TDVPS_APIC(offset)		BUILD_TDX_FIELD(1, (offset))
+
+/* @gpr is the index of a general purpose register, e.g. eax=0 */
+#define TDVPS_GPR(gpr)			BUILD_TDX_FIELD(16, (gpr))
+
+#define TDVPS_DR(dr)			BUILD_TDX_FIELD(17, (0 + (dr)))
+
+enum tdx_guest_other_state {
+	TD_VCPU_XCR0 = 32,
+	TD_VCPU_IWK_ENCKEY0 = 64,
+	TD_VCPU_IWK_ENCKEY1,
+	TD_VCPU_IWK_ENCKEY2,
+	TD_VCPU_IWK_ENCKEY3,
+	TD_VCPU_IWK_INTKEY0 = 68,
+	TD_VCPU_IWK_INTKEY1,
+	TD_VCPU_IWK_FLAGS = 70,
+};
+
+/* @field is any of enum tdx_guest_other_state */
+#define TDVPS_STATE(field)		BUILD_TDX_FIELD(17, (field))
+
+/* @msr is the MSR index */
+#define TDVPS_MSR(msr)			BUILD_TDX_FIELD(19, (msr))
+
+/* Management class fields */
+enum tdx_guest_management {
+	TD_VCPU_PEND_NMI = 11,
+};
+
+/* @field is any of enum tdx_guest_management */
+#define TDVPS_MANAGEMENT(field)		BUILD_TDX_FIELD(32, (field))
+
+enum tdx_tdcs_execution_control {
+	TD_TDCS_EXEC_TSC_OFFSET = 10,
+};
+
+/* @field is any of enum tdx_tdcs_execution_control */
+#define TDCS_EXEC(field)		BUILD_TDX_FIELD(17, (field))
+
+/*
+ * Hard code those values for simplicity and efficiency.  They are constant for
+ * the TDX 1.0.  TDH.SYS.INIT enumerates those values.  On kvm initialization,
+ * do sanity checks so that they are correct values.
+ *
+ * TODO: If they are bumped in future, increase those value.
+ *       (or make them full runtime values)
+ */
+#define TDX_NR_TDCX_PAGES		4
+#define TDX_NR_TDVPX_PAGES		5
+#define TDX_MAX_NR_CPUID_CONFIGS	6
+
+#define TDX_EXTENDMR_CHUNKSIZE		256
+
+struct tdx_cpuid_value {
+	u32 eax;
+	u32 ebx;
+	u32 ecx;
+	u32 edx;
+} __packed;
+
+struct tdx_cpuid_config {
+	u32 leaf;
+	u32 sub_leaf;
+	u32 eax;
+	u32 ebx;
+	u32 ecx;
+	u32 edx;
+} __packed;
+
+struct tdsysinfo_struct {
+	/* TDX-SEAM Module Info */
+	u32 attributes;
+	u32 vendor_id;
+	u32 build_date;
+	u16 build_num;
+	u16 minor_version;
+	u16 major_version;
+	u8 reserved0[14];
+	/* Memory Info */
+	u16 max_tdmrs;
+	u16 max_reserved_per_tdmr;
+	u16 pamt_entry_size;
+	u8 reserved1[10];
+	/* Control Struct Info */
+	u16 tdcs_base_size;
+	u8 reserved2[2];
+	u16 tdvps_base_size;
+	u8 tdvps_xfam_dependent_size;
+	u8 reserved3[9];
+	/* TD Capabilities */
+	u64 attributes_fixed0;
+	u64 attributes_fixed1;
+	u64 xfam_fixed0;
+	u64 xfam_fixed1;
+	u8 reserved4[32];
+	u32 num_cpuid_config;
+	union {
+		struct tdx_cpuid_config cpuid_configs[0];
+		u8 reserved5[892];
+	};
+} __packed __aligned(1024);
+
+#define TDX_TD_ATTRIBUTE_DEBUG		BIT_ULL(0)
+#define TDX_TD_ATTRIBUTE_PKS		BIT_ULL(30)
+#define TDX_TD_ATTRIBUTE_KL		BIT_ULL(31)
+#define TDX_TD_ATTRIBUTE_PERFMON	BIT_ULL(63)
+
+#define TDX_TD_XFAM_LBR			BIT_ULL(15)
+#define TDX_TD_XFAM_AMX			(BIT_ULL(17) | BIT_ULL(18))
+
+/*
+ * TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B.
+ */
+struct td_params {
+	u64 attributes;
+	u64 xfam;
+	u32 max_vcpus;
+	u32 reserved0;
+
+	u64 eptp_controls;
+	u64 exec_controls;
+	u16 tsc_frequency;
+	u8  reserved1[38];
+
+	u64 mrconfigid[6];
+	u64 mrowner[6];
+	u64 mrownerconfig[6];
+	u64 reserved2[4];
+
+	union {
+		struct tdx_cpuid_value cpuid_values[0];
+		u8 reserved3[768];
+	};
+} __packed __aligned(1024);
+
+/* Guest uses MAX_PA for GPAW when set. */
+#define TDX_EXEC_CONTROL_MAX_GPAW      BIT_ULL(0)
+
+/*
+ * TDX requires the frequency to be defined in units of 25MHz, which is the
+ * frequency of the core crystal clock on TDX-capable platforms, i.e. TDX-SEAM
+ * can only program frequencies that are multiples of 25MHz.  The frequency
+ * must be between 1ghz and 10ghz (inclusive).
+ */
+#define TDX_TSC_KHZ_TO_25MHZ(tsc_in_khz)	((tsc_in_khz) / (25 * 1000))
+#define TDX_TSC_25MHZ_TO_KHZ(tsc_in_25mhz)	((tsc_in_25mhz) * (25 * 1000))
+#define TDX_MIN_TSC_FREQUENCY_KHZ		(100 * 1000)
+#define TDX_MAX_TSC_FREQUENCY_KHZ		(10 * 1000 * 1000)
+
+#endif /* __KVM_X86_TDX_ARCH_H */
Signature · 1 line
2.25.1
#1 lore raw

RFC PATCH v3 1/59 x86/mktme: move out MKTME related constatnts/macro to msr-index.h

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

Later the constants/macro will be used by MTRR logic to reduce physical bits to calculate MTRR mask.

---
 arch/x86/include/asm/msr-index.h | 16 ++++++++++++++++
 arch/x86/kernel/cpu/intel.c      | 14 --------------
 2 files changed, 16 insertions(+), 14 deletions(-)
arch/x86/include/asm/msr-index.h +16 -0
index 28b207a786b1..97a41d6238f5 100644
@@ -905,6 +905,22 @@
 /* Geode defined MSRs */
 #define MSR_GEODE_BUSCONT_CONF0		0x00001900
 
+/* MKTME MSRs */
+#define MSR_IA32_TME_ACTIVATE		0x00000982
+
+/* Helpers to access TME_ACTIVATE MSR */
+#define TME_ACTIVATE_LOCKED(x)		(x & 0x1)
+#define TME_ACTIVATE_ENABLED(x)		(x & 0x2)
+
+#define TME_ACTIVATE_POLICY(x)		((x >> 4) & 0xf)	/* Bits 7:4 */
+#define TME_ACTIVATE_KEYID_BITS(x)	((x >> 32) & 0xf)	/* Bits 35:32 */
+
+#define TME_ACTIVATE_POLICY_AES_XTS_128	0
+
+#define TME_ACTIVATE_CRYPTO_ALGS(x)	((x >> 48) & 0xffff)	/* Bits 63:48 */
+#define TME_ACTIVATE_CRYPTO_AES_XTS_128	1
+
+
 /* Intel VT MSRs */
 #define MSR_IA32_VMX_BASIC              0x00000480
 #define MSR_IA32_VMX_PINBASED_CTLS      0x00000481
arch/x86/kernel/cpu/intel.c +0 -14
index 8321c43554a1..df605787137b 100644
@@ -492,20 +492,6 @@ static void srat_detect_node(struct cpuinfo_x86 *c)
 #endif
 }
 
-#define MSR_IA32_TME_ACTIVATE		0x982
-
-/* Helpers to access TME_ACTIVATE MSR */
-#define TME_ACTIVATE_LOCKED(x)		(x & 0x1)
-#define TME_ACTIVATE_ENABLED(x)		(x & 0x2)
-
-#define TME_ACTIVATE_POLICY(x)		((x >> 4) & 0xf)	/* Bits 7:4 */
-#define TME_ACTIVATE_POLICY_AES_XTS_128	0
-
-#define TME_ACTIVATE_KEYID_BITS(x)	((x >> 32) & 0xf)	/* Bits 35:32 */
-
-#define TME_ACTIVATE_CRYPTO_ALGS(x)	((x >> 48) & 0xffff)	/* Bits 63:48 */
-#define TME_ACTIVATE_CRYPTO_AES_XTS_128	1
-
 /* Values for mktme_status (SW only construct) */
 #define MKTME_ENABLED			0
 #define MKTME_DISABLED			1
Signature · 1 line
2.25.1
#5 lore raw

RFC PATCH v3 5/59 KVM: TDX: add a helper function for kvm to call seamcall

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

Add a helper function for kvm to call seamcall and a helper macro to check its return value. The later patches will use them.

---
 arch/x86/kvm/vmx/seamcall.h | 113 ++++++++++++++++++++++++++++++++++++
 1 file changed, 113 insertions(+)
 create mode 100644 arch/x86/kvm/vmx/seamcall.h
arch/x86/kvm/vmx/seamcall.h new file +113 -0
new file mode 100644
index 000000000000..f27e9d27137d
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __KVM_VMX_SEAMCALL_H
+#define __KVM_VMX_SEAMCALL_H
+
+#include <asm/asm.h>
+
+#ifdef CONFIG_INTEL_TDX_HOST
+
+#ifdef __ASSEMBLER__
+
+.macro seamcall
+	.byte 0x66, 0x0f, 0x01, 0xcf
+.endm
+
+#else
+
+/*
+ * TDX extended return:
+ * Some of The "TDX module" SEAMCALLs return extended values (which are function
+ * leaf specific) in registers in addition to the completion status code in
+ %rax.
+ */
+struct tdx_ex_ret {
+	union {
+		struct {
+			u64 rcx;
+			u64 rdx;
+			u64 r8;
+			u64 r9;
+			u64 r10;
+			u64 r11;
+		} regs;
+		/* TDH_MNG_INIT returns CPUID info on error. */
+		struct {
+			u32 leaf;
+			u32 subleaf;
+		} mng_init;
+		/* Functions that walk SEPT */
+		struct {
+			u64 septe;
+			struct {
+				u64 level		:3;
+				u64 sept_reserved_0	:5;
+				u64 state		:8;
+				u64 sept_reserved_1	:48;
+			};
+		} sept_walk;
+		/* TDH_MNG_{RD,WR} return the field value. */
+		struct {
+			u64 field_val;
+		} mng_rdwr;
+		/* TDH_MEM_{RD,WR} return the error info and value. */
+		struct {
+			u64 ext_err_info_1;
+			u64 ext_err_info_2;
+			u64 mem_val;
+		} mem_rdwr;
+		/* TDH_PHYMEM_PAGE_RDMD and TDH_PHYMEM_PAGE_RECLAIM return page metadata. */
+		struct {
+			u64 page_type;
+			u64 owner;
+			u64 page_size;
+		} phymem_page_md;
+	};
+};
+
+static inline u64 seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9, u64 r10,
+			struct tdx_ex_ret *ex)
+{
+	register unsigned long r8_in asm("r8");
+	register unsigned long r9_in asm("r9");
+	register unsigned long r10_in asm("r10");
+	register unsigned long r8_out asm("r8");
+	register unsigned long r9_out asm("r9");
+	register unsigned long r10_out asm("r10");
+	register unsigned long r11_out asm("r11");
+	struct tdx_ex_ret dummy;
+	u64 ret;
+
+	if (!ex)
+		/* The following inline assembly requires non-NULL ex. */
+		ex = &dummy;
+
+	/*
+	 * Because the TDX module is known to be already initialized, seamcall
+	 * instruction should always succeed without exceptions.  Don't check
+	 * the instruction error with CF=1 for the availability of the TDX
+	 * module.
+	 */
+	r8_in = r8;
+	r9_in = r9;
+	r10_in = r10;
+	asm volatile (
+		".byte 0x66, 0x0f, 0x01, 0xcf\n\t"	/* seamcall instruction */
+		: ASM_CALL_CONSTRAINT, "=a"(ret),
+		  "=c"(ex->regs.rcx), "=d"(ex->regs.rdx),
+		  "=r"(r8_out), "=r"(r9_out), "=r"(r10_out), "=r"(r11_out)
+		: "a"(op), "c"(rcx), "d"(rdx),
+		  "r"(r8_in), "r"(r9_in), "r"(r10_in)
+		: "cc", "memory");
+	ex->regs.r8 = r8_out;
+	ex->regs.r9 = r9_out;
+	ex->regs.r10 = r10_out;
+	ex->regs.r11 = r11_out;
+
+	return ret;
+}
+
+#endif /* !__ASSEMBLER__ */
+
+#endif	/* CONFIG_INTEL_TDX_HOST */
+
+#endif /* __KVM_VMX_SEAMCALL_H */
Signature · 1 line
2.25.1
#6 lore raw

RFC PATCH v3 6/59 KVM: TDX: Add C wrapper functions for TDX SEAMCALLs

In reply to [email protected]

From: Sean Christopherson <[email protected]>

TDX SEAMCALL interface is defined in the TDX module specification. Define C wrapper functions for SEAMCALLs which the later patches will use.

---
 arch/x86/kvm/vmx/tdx_ops.h | 210 +++++++++++++++++++++++++++++++++++++
 1 file changed, 210 insertions(+)
 create mode 100644 arch/x86/kvm/vmx/tdx_ops.h
arch/x86/kvm/vmx/tdx_ops.h new file +210 -0
new file mode 100644
index 000000000000..87ed67fd2715
@@ -0,0 +1,210 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* constants/data definitions for TDX SEAMCALLs */
+
+#ifndef __KVM_X86_TDX_OPS_H
+#define __KVM_X86_TDX_OPS_H
+
+#include <linux/compiler.h>
+
+#include <asm/asm.h>
+#include <asm/kvm_host.h>
+
+#include "seamcall.h"
+
+#ifdef CONFIG_INTEL_TDX_HOST
+
+static inline u64 tdh_mng_addcx(hpa_t tdr, hpa_t addr)
+{
+	return seamcall(TDH_MNG_ADDCX, addr, tdr, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mem_page_add(hpa_t tdr, gpa_t gpa, hpa_t hpa, hpa_t source,
+			    struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_PAGE_ADD, gpa, tdr, hpa, source, 0, ex);
+}
+
+static inline u64 tdh_mem_sept_add(hpa_t tdr, gpa_t gpa, int level, hpa_t page,
+			    struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_SEPT_ADD, gpa | level, tdr, page, 0, 0, ex);
+}
+
+static inline u64 tdh_vp_addcx(hpa_t tdvpr, hpa_t addr)
+{
+	return seamcall(TDH_VP_ADDCX, addr, tdvpr, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mem_page_aug(hpa_t tdr, gpa_t gpa, hpa_t hpa,
+			    struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_PAGE_AUG, gpa, tdr, hpa, 0, 0, ex);
+}
+
+static inline u64 tdh_mem_range_block(hpa_t tdr, gpa_t gpa, int level,
+			  struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_RANGE_BLOCK, gpa | level, tdr, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_mng_key_config(hpa_t tdr)
+{
+	return seamcall(TDH_MNG_KEY_CONFIG, tdr, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mng_create(hpa_t tdr, int hkid)
+{
+	return seamcall(TDH_MNG_CREATE, tdr, hkid, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_vp_create(hpa_t tdr, hpa_t tdvpr)
+{
+	return seamcall(TDH_VP_CREATE, tdvpr, tdr, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mng_rd(hpa_t tdr, u64 field, struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MNG_RD, tdr, field, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_mng_wr(hpa_t tdr, u64 field, u64 val, u64 mask,
+			  struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MNG_WR, tdr, field, val, mask, 0, ex);
+}
+
+static inline u64 tdh_mem_rd(hpa_t addr, struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_RD, addr, 0, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_mem_wr(hpa_t addr, u64 val, struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_WR, addr, val, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_mem_page_demote(hpa_t tdr, gpa_t gpa, int level, hpa_t page,
+			       struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_PAGE_DEMOTE, gpa | level, tdr, page, 0, 0, ex);
+}
+
+static inline u64 tdh_mr_extend(hpa_t tdr, gpa_t gpa, struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MR_EXTEND, gpa, tdr, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_mr_finalize(hpa_t tdr)
+{
+	return seamcall(TDH_MR_FINALIZE, tdr, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_vp_flush(hpa_t tdvpr)
+{
+	return seamcall(TDH_VP_FLUSH, tdvpr, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mng_vpflushdone(hpa_t tdr)
+{
+	return seamcall(TDH_MNG_VPFLUSHDONE, tdr, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mng_key_freeid(hpa_t tdr)
+{
+	return seamcall(TDH_MNG_KEY_FREEID, tdr, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mng_init(hpa_t tdr, hpa_t td_params, struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MNG_INIT, tdr, td_params, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_vp_init(hpa_t tdvpr, u64 rcx)
+{
+	return seamcall(TDH_VP_INIT, tdvpr, rcx, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mem_page_promote(hpa_t tdr, gpa_t gpa, int level,
+				struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_PAGE_PROMOTE, gpa | level, tdr, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_phymem_page_rdmd(hpa_t page, struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_PHYMEM_PAGE_RDMD, page, 0, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_mem_sept_rd(hpa_t tdr, gpa_t gpa, int level,
+			   struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_SEPT_RD, gpa | level, tdr, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_vp_rd(hpa_t tdvpr, u64 field, struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_VP_RD, tdvpr, field, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_mng_key_reclaimid(hpa_t tdr)
+{
+	return seamcall(TDH_MNG_KEY_RECLAIMID, tdr, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_phymem_page_reclaim(hpa_t page, struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_PHYMEM_PAGE_RECLAIM, page, 0, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_mem_page_remove(hpa_t tdr, gpa_t gpa, int level,
+				struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_PAGE_REMOVE, gpa | level, tdr, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_mem_sept_remove(hpa_t tdr, gpa_t gpa, int level,
+			       struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_SEPT_REMOVE, gpa | level, tdr, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_sys_lp_shutdown(void)
+{
+	return seamcall(TDH_SYS_LP_SHUTDOWN, 0, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mem_track(hpa_t tdr)
+{
+	return seamcall(TDH_MEM_TRACK, tdr, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mem_range_unblock(hpa_t tdr, gpa_t gpa, int level,
+			    struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_RANGE_UNBLOCK, gpa | level, tdr, 0, 0, 0, ex);
+}
+
+static inline u64 tdh_phymem_cache_wb(bool resume)
+{
+	return seamcall(TDH_PHYMEM_CACHE_WB, resume ? 1 : 0, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_phymem_page_wbinvd(hpa_t page)
+{
+	return seamcall(TDH_PHYMEM_PAGE_WBINVD, page, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mem_sept_wr(hpa_t tdr, gpa_t gpa, int level, u64 val,
+			   struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_MEM_SEPT_WR, gpa | level, tdr, val, 0, 0, ex);
+}
+
+static inline u64 tdh_vp_wr(hpa_t tdvpr, u64 field, u64 val, u64 mask,
+			  struct tdx_ex_ret *ex)
+{
+	return seamcall(TDH_VP_WR, tdvpr, field, val, mask, 0, ex);
+}
+#endif /* CONFIG_INTEL_TDX_HOST */
+
+#endif /* __KVM_X86_TDX_OPS_H */
Signature · 1 line
2.25.1
#7 lore raw

RFC PATCH v3 7/59 KVM: TDX: Add helper functions to print TDX SEAMCALL error

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

Add helper functions to print out errors from the TDX module in an uniform way.

---
 arch/x86/kvm/Makefile        |  1 +
 arch/x86/kvm/vmx/seamcall.h  |  3 ++
 arch/x86/kvm/vmx/tdx_error.c | 53 ++++++++++++++++++++++++++++++++++++
 3 files changed, 57 insertions(+)
 create mode 100644 arch/x86/kvm/vmx/tdx_error.c
arch/x86/kvm/Makefile +1 -0
index 75dfd27b6e8a..c7eceff49e67 100644
@@ -29,6 +29,7 @@ kvm-$(CONFIG_KVM_XEN)	+= xen.o
 kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \
 			   vmx/evmcs.o vmx/nested.o vmx/posted_intr.o
 kvm-intel-$(CONFIG_X86_SGX_KVM)	+= vmx/sgx.o
+kvm-intel-$(CONFIG_INTEL_TDX_HOST)	+= vmx/tdx_error.o
 
 kvm-amd-y		+= svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o
 
arch/x86/kvm/vmx/seamcall.h +3 -0
index f27e9d27137d..5fa948054be6 100644
@@ -106,6 +106,9 @@ static inline u64 seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9, u64 r10,
 	return ret;
 }
 
+void pr_tdx_ex_ret_info(u64 op, u64 error_code, const struct tdx_ex_ret *ex_ret);
+void pr_tdx_error(u64 op, u64 error_code, const struct tdx_ex_ret *ex_ret);
+
 #endif /* !__ASSEMBLER__ */
 
 #endif	/* CONFIG_INTEL_TDX_HOST */
arch/x86/kvm/vmx/tdx_error.c new file +53 -0

53 lines not on this page · raw message

Signature · 1 line
2.25.1
#8 lore raw

RFC PATCH v3 8/59 KVM: Export kvm_io_bus_read for use by TDX for PV MMIO

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Later kvm_intel.ko will use it.

---
 virt/kvm/kvm_main.c | 1 +
 1 file changed, 1 insertion(+)
virt/kvm/kvm_main.c +1 -0
index d31724500501..d8e799eeef8d 100644
@@ -4994,6 +4994,7 @@ int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
 	r = __kvm_io_bus_read(vcpu, bus, &range, val);
 	return r < 0 ? r : 0;
 }
+EXPORT_SYMBOL_GPL(kvm_io_bus_read);
 
 /* Caller must hold slots_lock. */
 int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
Signature · 1 line
2.25.1
#10 lore raw

RFC PATCH v3 9/59 KVM: Enable hardware before doing arch VM initialization

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Swap the order of hardware_enable_all() and kvm_arch_init_vm() to accommodate Intel's TDX, which needs VMX to be enabled during VM init in order to make SEAMCALLs.

This also provides consistent ordering between kvm_create_vm() and kvm_destroy_vm() with respect to calling kvm_arch_destroy_vm() and hardware_disable_all().

---
 virt/kvm/kvm_main.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
virt/kvm/kvm_main.c +8 -8
index d8e799eeef8d..8544e92db2da 100644
@@ -1065,7 +1065,7 @@ static struct kvm *kvm_create_vm(unsigned long type)
 		struct kvm_memslots *slots = kvm_alloc_memslots();
 
 		if (!slots)
-			goto out_err_no_arch_destroy_vm;
+			goto out_err_no_disable;
 		/* Generations must be different for each address space. */
 		slots->generation = i;
 		rcu_assign_pointer(kvm->memslots[i], slots);
@@ -1075,19 +1075,19 @@ static struct kvm *kvm_create_vm(unsigned long type)
 		rcu_assign_pointer(kvm->buses[i],
 			kzalloc(sizeof(struct kvm_io_bus), GFP_KERNEL_ACCOUNT));
 		if (!kvm->buses[i])
-			goto out_err_no_arch_destroy_vm;
+			goto out_err_no_disable;
 	}
 
 	kvm->max_halt_poll_ns = halt_poll_ns;
 
-	r = kvm_arch_init_vm(kvm, type);
-	if (r)
-		goto out_err_no_arch_destroy_vm;
-
 	r = hardware_enable_all();
 	if (r)
 		goto out_err_no_disable;
 
+	r = kvm_arch_init_vm(kvm, type);
+	if (r)
+		goto out_err_no_arch_destroy_vm;
+
 #ifdef CONFIG_HAVE_KVM_IRQFD
 	INIT_HLIST_HEAD(&kvm->irq_ack_notifier_list);
 #endif
@@ -1115,10 +1115,10 @@ static struct kvm *kvm_create_vm(unsigned long type)
 		mmu_notifier_unregister(&kvm->mmu_notifier, current->mm);
 #endif
 out_err_no_mmu_notifier:
-	hardware_disable_all();
-out_err_no_disable:
 	kvm_arch_destroy_vm(kvm);
 out_err_no_arch_destroy_vm:
+	hardware_disable_all();
+out_err_no_disable:
 	WARN_ON_ONCE(!refcount_dec_and_test(&kvm->users_count));
 	for (i = 0; i < KVM_NR_BUSES; i++)
 		kfree(kvm_get_bus(kvm, i));
Signature · 1 line
2.25.1
#12 lore raw

RFC PATCH v3 10/59 KVM: x86: Split core of hypercall emulation to helper function

In reply to [email protected]

From: Sean Christopherson <[email protected]>

By necessity, TDX will use a different register ABI for hypercalls. Break out the core functionality so that it may be reused for TDX.

---
 arch/x86/include/asm/kvm_host.h |  4 +++
 arch/x86/kvm/x86.c              | 55 ++++++++++++++++++++-------------
 2 files changed, 38 insertions(+), 21 deletions(-)
arch/x86/include/asm/kvm_host.h +4 -0
index e5d8700319cc..5b31d8e5fcbf 100644
@@ -1778,6 +1778,10 @@ void kvm_request_apicv_update(struct kvm *kvm, bool activate,
 void __kvm_request_apicv_update(struct kvm *kvm, bool activate,
 				unsigned long bit);
 
+unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr,
+				      unsigned long a0, unsigned long a1,
+				      unsigned long a2, unsigned long a3,
+				      int op_64_bit);
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
 
 int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,
arch/x86/kvm/x86.c +34 -21

77 lines not on this page · raw message

Signature · 1 line
2.25.1
#13 lore raw

RFC PATCH v3 11/59 KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Later kvm_intel.ko will use it.

---
 arch/x86/kvm/x86.c | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/x86.c +1 -0
index 8a4774c5c85d..6f3c77524426 100644
@@ -12584,6 +12584,7 @@ EXPORT_SYMBOL_GPL(kvm_sev_es_string_io);
 
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_entry);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_exit);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_mmio);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_fast_mmio);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_inj_virq);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_page_fault);
Signature · 1 line
2.25.1
#18 lore raw

RFC PATCH v3 14/59 KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add a capability to effectively allow userspace to query what VM types are supported by KVM.

---
 arch/x86/include/asm/kvm-x86-ops.h    | 1 +
 arch/x86/include/asm/kvm_host.h       | 2 ++
 arch/x86/include/uapi/asm/kvm.h       | 4 ++++
 arch/x86/kvm/svm/svm.c                | 6 ++++++
 arch/x86/kvm/vmx/vmx.c                | 6 ++++++
 arch/x86/kvm/x86.c                    | 9 ++++++++-
 include/uapi/linux/kvm.h              | 2 ++
 tools/arch/x86/include/uapi/asm/kvm.h | 4 ++++
 tools/include/uapi/linux/kvm.h        | 2 ++
 9 files changed, 35 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index cefe1d81e2e8..4aa0a1c10b84 100644
@@ -18,6 +18,7 @@ KVM_X86_OP_NULL(hardware_unsetup)
 KVM_X86_OP_NULL(cpu_has_accelerated_tpr)
 KVM_X86_OP(has_emulated_msr)
 KVM_X86_OP(vcpu_after_set_cpuid)
+KVM_X86_OP(is_vm_type_supported)
 KVM_X86_OP(vm_init)
 KVM_X86_OP_NULL(vm_destroy)
 KVM_X86_OP(vcpu_create)
arch/x86/include/asm/kvm_host.h +2 -0
index 5b31d8e5fcbf..cdb908ed7d5b 100644
@@ -1037,6 +1037,7 @@ struct kvm_x86_msr_filter {
 #define APICV_INHIBIT_REASON_BLOCKIRQ	6
 
 struct kvm_arch {
+	unsigned long vm_type;
 	unsigned long n_used_mmu_pages;
 	unsigned long n_requested_mmu_pages;
 	unsigned long n_max_mmu_pages;
@@ -1311,6 +1312,7 @@ struct kvm_x86_ops {
 	bool (*has_emulated_msr)(struct kvm *kvm, u32 index);
 	void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu);
 
+	bool (*is_vm_type_supported)(unsigned long vm_type);
 	unsigned int vm_size;
 	int (*vm_init)(struct kvm *kvm);
 	void (*vm_destroy)(struct kvm *kvm);
arch/x86/include/uapi/asm/kvm.h +4 -0
index 5a776a08f78c..a0805a2a81f8 100644
@@ -508,4 +508,8 @@ struct kvm_pmu_event_filter {
 #define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
 #define   KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
 
+#define KVM_X86_LEGACY_VM	0
+#define KVM_X86_SEV_ES_VM	1
+#define KVM_X86_TDX_VM		2
+
 #endif /* _ASM_X86_KVM_H */
arch/x86/kvm/svm/svm.c +6 -0
index 5630c241d5f6..2ef77d4566a9 100644
@@ -4562,6 +4562,11 @@ static void svm_vm_destroy(struct kvm *kvm)
 	sev_vm_destroy(kvm);
 }
 
+static bool svm_is_vm_type_supported(unsigned long type)
+{
+	return type == KVM_X86_LEGACY_VM;
+}
+
 static int svm_vm_init(struct kvm *kvm)
 {
 	if (!pause_filter_count || !pause_filter_thresh)
@@ -4589,6 +4594,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.vcpu_free = svm_free_vcpu,
 	.vcpu_reset = svm_vcpu_reset,
 
+	.is_vm_type_supported = svm_is_vm_type_supported,
 	.vm_size = sizeof(struct kvm_svm),
 	.vm_init = svm_vm_init,
 	.vm_destroy = svm_vm_destroy,
arch/x86/kvm/vmx/vmx.c +6 -0
index fe4f3ee61db3..f2905e00b063 100644
@@ -6851,6 +6851,11 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu)
 	return err;
 }
 
+static bool vmx_is_vm_type_supported(unsigned long type)
+{
+	return type == KVM_X86_LEGACY_VM;
+}
+
 #define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
 #define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
 
@@ -7505,6 +7510,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = {
 	.cpu_has_accelerated_tpr = report_flexpriority,
 	.has_emulated_msr = vmx_has_emulated_msr,
 
+	.is_vm_type_supported = vmx_is_vm_type_supported,
 	.vm_size = sizeof(struct kvm_vmx),
 	.vm_init = vmx_vm_init,
 
arch/x86/kvm/x86.c +8 -1
index 6f3c77524426..5f4b6f70489b 100644
@@ -4225,6 +4225,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		else
 			r = 0;
 		break;
+	case KVM_CAP_VM_TYPES:
+		r = BIT(KVM_X86_LEGACY_VM);
+		if (static_call(kvm_x86_is_vm_type_supported)(KVM_X86_TDX_VM))
+			r |= BIT(KVM_X86_TDX_VM);
+		break;
 	default:
 		break;
 	}
@@ -11320,9 +11325,11 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	int ret;
 	unsigned long flags;
 
-	if (type)
+	if (!static_call(kvm_x86_is_vm_type_supported)(type))
 		return -EINVAL;
 
+	kvm->arch.vm_type = type;
+
 	ret = kvm_page_track_init(kvm);
 	if (ret)
 		return ret;
include/uapi/linux/kvm.h +2 -0
index 1daa45268de2..bb49e095867e 100644
@@ -1132,6 +1132,8 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_ARM_MTE 205
 #define KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM 206
 
+#define KVM_CAP_VM_TYPES 1000
+
 #ifdef KVM_CAP_IRQ_ROUTING
 
 struct kvm_irq_routing_irqchip {
tools/arch/x86/include/uapi/asm/kvm.h +4 -0
index 2ef1f6513c68..96b0064cff5c 100644
@@ -504,4 +504,8 @@ struct kvm_pmu_event_filter {
 #define KVM_PMU_EVENT_ALLOW 0
 #define KVM_PMU_EVENT_DENY 1
 
+#define KVM_X86_LEGACY_VM	0
+#define KVM_X86_SEV_ES_VM	1
+#define KVM_X86_TDX_VM		2
+
 #endif /* _ASM_X86_KVM_H */
tools/include/uapi/linux/kvm.h +2 -0
index a067410ebea5..e5aadad54ced 100644
@@ -1113,6 +1113,8 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204
 #define KVM_CAP_ARM_MTE 205
 
+#define KVM_CAP_VM_TYPES 1000
+
 #ifdef KVM_CAP_IRQ_ROUTING
 
 struct kvm_irq_routing_irqchip {
Signature · 1 line
2.25.1
#14 lore raw

RFC PATCH v3 12/59 KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Zap only leaf SPTEs when deleting/moving a memslot by default, and add a module param to allow reverting to the old behavior of zapping all SPTEs at all levels and memslots when any memslot is updated.

---
 arch/x86/kvm/mmu/mmu.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +20 -1
index 33794379949e..1e11b14f4f82 100644
@@ -91,6 +91,9 @@ __MODULE_PARM_TYPE(nx_huge_pages_recovery_period_ms, "uint");
 static bool __read_mostly force_flush_and_sync_on_reuse;
 module_param_named(flush_on_reuse, force_flush_and_sync_on_reuse, bool, 0644);
 
+static bool __read_mostly memslot_update_zap_all;
+module_param(memslot_update_zap_all, bool, 0444);
+
 /*
  * When setting this variable to true it enables Two-Dimensional-Paging
  * where the hardware walks 2 page tables:
@@ -5681,11 +5684,27 @@ static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm)
 	return unlikely(!list_empty_careful(&kvm->arch.zapped_obsolete_pages));
 }
 
+static void kvm_mmu_zap_memslot(struct kvm *kvm, struct kvm_memory_slot *slot)
+{
+	/*
+	 * Zapping non-leaf SPTEs, a.k.a. not-last SPTEs, isn't required, worst
+	 * case scenario we'll have unused shadow pages lying around until they
+	 * are recycled due to age or when the VM is destroyed.
+	 */
+	write_lock(&kvm->mmu_lock);
+	slot_handle_level(kvm, slot, kvm_zap_rmapp, PG_LEVEL_4K,
+			  KVM_MAX_HUGEPAGE_LEVEL, true);
+	write_unlock(&kvm->mmu_lock);
+}
+
 static void kvm_mmu_invalidate_zap_pages_in_memslot(struct kvm *kvm,
 			struct kvm_memory_slot *slot,
 			struct kvm_page_track_notifier_node *node)
 {
-	kvm_mmu_zap_all_fast(kvm);
+	if (memslot_update_zap_all)
+		kvm_mmu_zap_all_fast(kvm);
+	else
+		kvm_mmu_zap_memslot(kvm, slot);
 }
 
 void kvm_mmu_init_vm(struct kvm *kvm)
Signature · 1 line
2.25.1
#30 lore raw

RFC PATCH v3 18/59 KVM: x86: Add flag to mark TSC as immutable (for TDX)

In reply to [email protected]

From: Sean Christopherson <[email protected]>

The TSC for TDX1 guests is fixed at TD creation time. Add tsc_immutable to reflect that the TSC of the guest cannot be changed in any way, and use it to short circuit all paths that lead to one of the myriad TSC adjustment flows.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/x86.c              | 35 +++++++++++++++++++++++++--------
 2 files changed, 28 insertions(+), 8 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index e912e1e853ef..f3808672c720 100644
@@ -1122,6 +1122,7 @@ struct kvm_arch {
 	int audit_point;
 	#endif
 
+	bool tsc_immutable;
 	bool backwards_tsc_observed;
 	bool boot_vcpu_runs_old_kvmclock;
 	u32 bsp_vcpu_id;
arch/x86/kvm/x86.c +27 -8
index fefa4602e879..0ebd60846079 100644
@@ -2247,7 +2247,9 @@ static int set_tsc_khz(struct kvm_vcpu *vcpu, u32 user_tsc_khz, bool scale)
 	u64 ratio;
 
 	/* Guest TSC same frequency as host TSC? */
-	if (!scale) {
+	if (!scale || vcpu->kvm->arch.tsc_immutable) {
+		if (scale)
+			pr_warn_ratelimited("Guest TSC immutable, scaling not supported\n");
 		kvm_vcpu_write_tsc_multiplier(vcpu, kvm_default_tsc_scaling_ratio);
 		return 0;
 	}
@@ -2534,6 +2536,9 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)
 	bool matched = false;
 	bool synchronizing = false;
 
+	if (WARN_ON_ONCE(vcpu->kvm->arch.tsc_immutable))
+		return;
+
 	raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
 	offset = kvm_compute_l1_tsc_offset(vcpu, data);
 	ns = get_kvmclock_base_ns();
@@ -2960,6 +2965,10 @@ static int kvm_guest_time_update(struct kvm_vcpu *v)
 	u8 pvclock_flags;
 	bool use_master_clock;
 
+	/* Unable to update guest time if the TSC is immutable. */
+	if (ka->tsc_immutable)
+		return 0;
+
 	kernel_ns = 0;
 	host_tsc = 0;
 
@@ -4372,7 +4381,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		if (tsc_delta < 0)
 			mark_tsc_unstable("KVM discovered backwards TSC");
 
-		if (kvm_check_tsc_unstable()) {
+		if (kvm_check_tsc_unstable() &&
+		    !vcpu->kvm->arch.tsc_immutable) {
 			u64 offset = kvm_compute_l1_tsc_offset(vcpu,
 						vcpu->arch.last_guest_tsc);
 			kvm_vcpu_write_tsc_offset(vcpu, offset);
@@ -4386,7 +4396,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		 * On a host with synchronized TSC, there is no need to update
 		 * kvmclock on vcpu->cpu migration
 		 */
-		if (!vcpu->kvm->arch.use_master_clock || vcpu->cpu == -1)
+		if ((!vcpu->kvm->arch.use_master_clock || vcpu->cpu == -1) &&
+		    !vcpu->kvm->arch.tsc_immutable)
 			kvm_make_request(KVM_REQ_GLOBAL_CLOCK_UPDATE, vcpu);
 		if (vcpu->cpu != cpu)
 			kvm_make_request(KVM_REQ_MIGRATE_TIMER, vcpu);
@@ -5352,10 +5363,11 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 		break;
 	}
 	case KVM_SET_TSC_KHZ: {
-		u32 user_tsc_khz;
+		u32 user_tsc_khz = (u32)arg;
 
 		r = -EINVAL;
-		user_tsc_khz = (u32)arg;
+		if (vcpu->kvm->arch.tsc_immutable)
+			goto out;
 
 		if (kvm_has_tsc_control &&
 		    user_tsc_khz >= kvm_max_guest_tsc_khz)
@@ -10994,9 +11006,12 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
 
 	if (mutex_lock_killable(&vcpu->mutex))
 		return;
-	vcpu_load(vcpu);
-	kvm_synchronize_tsc(vcpu, 0);
-	vcpu_put(vcpu);
+
+	if (!kvm->arch.tsc_immutable) {
+		vcpu_load(vcpu);
+		kvm_synchronize_tsc(vcpu, 0);
+		vcpu_put(vcpu);
+	}
 
 	/* poll control enabled by default */
 	vcpu->arch.msr_kvm_poll_control = 1;
@@ -11253,6 +11268,10 @@ int kvm_arch_hardware_enable(void)
 	if (backwards_tsc) {
 		u64 delta_cyc = max_tsc - local_tsc;
 		list_for_each_entry(kvm, &vm_list, vm_list) {
+			if (vcpu->kvm->arch.tsc_immutable) {
+				pr_warn_ratelimited("Backwards TSC observed and guest with immutable TSC active\n");
+				continue;
+			}
 			kvm->arch.backwards_tsc_observed = true;
 			kvm_for_each_vcpu(i, vcpu, kvm) {
 				vcpu->arch.tsc_offset_adjustment += delta_cyc;
Signature · 1 line
2.25.1
#4 lore raw

RFC PATCH v3 4/59 KVM: TDX: Add TDX "architectural" error codes

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add error codes for the TDX SEAMCALLs both for TDX VMM side and TDX guest side.

TDX SEAMCALL uses bits 31:0 to return more information, so these error codes will only exactly match RAX[63:32]. Error codes for TDG.VP.VMCALL is defined by TDX Guest-Host-Communication interface spec.

---
 arch/x86/kvm/vmx/tdx_errno.h | 111 +++++++++++++++++++++++++++++++++++
 1 file changed, 111 insertions(+)
 create mode 100644 arch/x86/kvm/vmx/tdx_errno.h
arch/x86/kvm/vmx/tdx_errno.h new file +111 -0
new file mode 100644
index 000000000000..395dd3099254
@@ -0,0 +1,111 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* architectural status code for SEAMCALL */
+
+#ifndef __KVM_X86_TDX_ERRNO_H
+#define __KVM_X86_TDX_ERRNO_H
+
+#define TDX_SEAMCALL_STATUS_MASK		0xFFFFFFFF00000000ULL
+
+/*
+ * TDX SEAMCALL Status Codes (returned in RAX)
+ */
+#define TDX_SUCCESS				0x0000000000000000ULL
+#define TDX_NON_RECOVERABLE_VCPU		0x4000000100000000ULL
+#define TDX_NON_RECOVERABLE_TD			0x4000000200000000ULL
+#define TDX_INTERRUPTED_RESUMABLE		0x8000000300000000ULL
+#define TDX_INTERRUPTED_RESTARTABLE		0x8000000400000000ULL
+#define TDX_NON_RECOVERABLE_TD_FATAL		0x4000000500000000ULL
+#define TDX_INVALID_RESUMPTION			0xC000000600000000ULL
+#define TDX_NON_RECOVERABLE_TD_NO_APIC		0xC000000700000000ULL
+#define TDX_OPERAND_INVALID			0xC000010000000000ULL
+#define TDX_OPERAND_ADDR_RANGE_ERROR		0xC000010100000000ULL
+#define TDX_OPERAND_BUSY			0x8000020000000000ULL
+#define TDX_PREVIOUS_TLB_EPOCH_BUSY		0x8000020100000000ULL
+#define TDX_SYS_BUSY				0x8000020200000000ULL
+#define TDX_PAGE_METADATA_INCORRECT		0xC000030000000000ULL
+#define TDX_PAGE_ALREADY_FREE			0x0000030100000000ULL
+#define TDX_PAGE_NOT_OWNED_BY_TD		0xC000030200000000ULL
+#define TDX_PAGE_NOT_FREE			0xC000030300000000ULL
+#define TDX_TD_ASSOCIATED_PAGES_EXIST		0xC000040000000000ULL
+#define TDX_SYSINIT_NOT_PENDING			0xC000050000000000ULL
+#define TDX_SYSINIT_NOT_DONE			0xC000050100000000ULL
+#define TDX_SYSINITLP_NOT_DONE			0xC000050200000000ULL
+#define TDX_SYSINITLP_DONE			0xC000050300000000ULL
+#define TDX_SYS_NOT_READY			0xC000050500000000ULL
+#define TDX_SYS_SHUTDOWN			0xC000050600000000ULL
+#define TDX_SYSCONFIG_NOT_DONE			0xC000050700000000ULL
+#define TDX_TD_NOT_INITIALIZED			0xC000060000000000ULL
+#define TDX_TD_INITIALIZED			0xC000060100000000ULL
+#define TDX_TD_NOT_FINALIZED			0xC000060200000000ULL
+#define TDX_TD_FINALIZED			0xC000060300000000ULL
+#define TDX_TD_FATAL				0xC000060400000000ULL
+#define TDX_TD_NON_DEBUG			0xC000060500000000ULL
+#define TDX_LIFECYCLE_STATE_INCORRECT		0xC000060700000000ULL
+#define TDX_TDCX_NUM_INCORRECT			0xC000061000000000ULL
+#define TDX_VCPU_STATE_INCORRECT		0xC000070000000000ULL
+#define TDX_VCPU_ASSOCIATED			0x8000070100000000ULL
+#define TDX_VCPU_NOT_ASSOCIATED			0x8000070200000000ULL
+#define TDX_TDVPX_NUM_INCORRECT			0xC000070300000000ULL
+#define TDX_NO_VALID_VE_INFO			0xC000070400000000ULL
+#define TDX_MAX_VCPUS_EXCEEDED			0xC000070500000000ULL
+#define TDX_TSC_ROLLBACK			0xC000070600000000ULL
+#define TDX_FIELD_NOT_WRITABLE			0xC000072000000000ULL
+#define TDX_FIELD_NOT_READABLE			0xC000072100000000ULL
+#define TDX_TD_VMCS_FIELD_NOT_INITIALIZED	0xC000073000000000ULL
+#define TDX_KEY_GENERATION_FAILED		0x8000080000000000ULL
+#define TDX_TD_KEYS_NOT_CONFIGURED		0x8000081000000000ULL
+#define TDX_KEY_STATE_INCORRECT			0xC000081100000000ULL
+#define TDX_KEY_CONFIGURED			0x0000081500000000ULL
+#define TDX_WBCACHE_NOT_COMPLETE		0x8000081700000000ULL
+#define TDX_HKID_NOT_FREE			0xC000082000000000ULL
+#define TDX_NO_HKID_READY_TO_WBCACHE		0x0000082100000000ULL
+#define TDX_WBCACHE_RESUME_ERROR		0xC000082300000000ULL
+#define TDX_FLUSHVP_NOT_DONE			0x8000082400000000ULL
+#define TDX_NUM_ACTIVATED_HKIDS_NOT_SUPPORTED	0xC000082500000000ULL
+#define TDX_INCORRECT_CPUID_VALUE		0xC000090000000000ULL
+#define TDX_BOOT_NT4_SET			0xC000090100000000ULL
+#define TDX_INCONSISTENT_CPUID_FIELD		0xC000090200000000ULL
+#define TDX_CPUID_LEAF_1F_FORMAT_UNRECOGNIZED	0xC000090400000000ULL
+#define TDX_INVALID_WBINVD_SCOPE		0xC000090500000000ULL
+#define TDX_INVALID_PKG_ID			0xC000090600000000ULL
+#define TDX_CPUID_LEAF_NOT_SUPPORTED		0xC000090800000000ULL
+#define TDX_SMRR_NOT_LOCKED			0xC000091000000000ULL
+#define TDX_INVALID_SMRR_CONFIGURATION		0xC000091100000000ULL
+#define TDX_SMRR_OVERLAPS_CMR			0xC000091200000000ULL
+#define TDX_SMRR_LOCK_NOT_SUPPORTED		0xC000091300000000ULL
+#define TDX_SMRR_NOT_SUPPORTED			0xC000091400000000ULL
+#define TDX_INCONSISTENT_MSR			0xC000092000000000ULL
+#define TDX_INCORRECT_MSR_VALUE			0xC000092100000000ULL
+#define TDX_SEAMREPORT_NOT_AVAILABLE		0xC000093000000000ULL
+#define TDX_PERF_COUNTERS_ARE_PEBS_ENABLED	0x8000094000000000ULL
+#define TDX_INVALID_TDMR			0xC0000A0000000000ULL
+#define TDX_NON_ORDERED_TDMR			0xC0000A0100000000ULL
+#define TDX_TDMR_OUTSIDE_CMRS			0xC0000A0200000000ULL
+#define TDX_TDMR_ALREADY_INITIALIZED		0x00000A0300000000ULL
+#define TDX_INVALID_PAMT			0xC0000A1000000000ULL
+#define TDX_PAMT_OUTSIDE_CMRS			0xC0000A1100000000ULL
+#define TDX_PAMT_OVERLAP			0xC0000A1200000000ULL
+#define TDX_INVALID_RESERVED_IN_TDMR		0xC0000A2000000000ULL
+#define TDX_NON_ORDERED_RESERVED_IN_TDMR	0xC0000A2100000000ULL
+#define TDX_CMR_LIST_INVALID			0xC0000A2200000000ULL
+#define TDX_EPT_WALK_FAILED			0xC0000B0000000000ULL
+#define TDX_EPT_ENTRY_FREE			0xC0000B0100000000ULL
+#define TDX_EPT_ENTRY_NOT_FREE			0xC0000B0200000000ULL
+#define TDX_EPT_ENTRY_NOT_PRESENT		0xC0000B0300000000ULL
+#define TDX_EPT_ENTRY_NOT_LEAF			0xC0000B0400000000ULL
+#define TDX_EPT_ENTRY_LEAF			0xC0000B0500000000ULL
+#define TDX_GPA_RANGE_NOT_BLOCKED		0xC0000B0600000000ULL
+#define TDX_GPA_RANGE_ALREADY_BLOCKED		0x00000B0700000000ULL
+#define TDX_TLB_TRACKING_NOT_DONE		0xC0000B0800000000ULL
+#define TDX_EPT_INVALID_PROMOTE_CONDITIONS	0xC0000B0900000000ULL
+#define TDX_PAGE_ALREADY_ACCEPTED		0x00000B0A00000000ULL
+#define TDX_PAGE_SIZE_MISMATCH			0xC0000B0B00000000ULL
+
+/*
+ * TDG.VP.VMCALL Status Codes (returned in R10)
+ */
+#define TDG_VP_VMCALL_SUCCESS			0x0000000000000000ULL
+#define TDG_VP_VMCALL_INVALID_OPERAND		0x8000000000000000ULL
+#define TDG_VP_VMCALL_TDREPORT_FAILED		0x8000000000000001ULL
+
+#endif /* __KVM_X86_TDX_ERRNO_H */
Signature · 1 line
2.25.1
#35 lore raw

RFC PATCH v3 21/59 KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level routes

In reply to [email protected]

From: Kai Huang <[email protected]>

Add a flag to let TDX disallow the in-kernel I/O APIC, level triggered routes for a userspace I/O APIC, and anything else that relies on being able to intercept EOIs. TDX-SEAM does not allow intercepting EOI.

Note, technically KVM could partially emulate the I/O APIC by allowing only edge triggered interrupts, but that adds a lot of complexity for basically zero benefit. Ideally KVM wouldn't even allow I/O APIC route reservation, but disabling that is a train wreck for Qemu.

---
 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/ioapic.c           | 4 ++++
 arch/x86/kvm/irq_comm.c         | 9 +++++++--
 arch/x86/kvm/lapic.c            | 3 ++-
 arch/x86/kvm/x86.c              | 6 ++++++
 5 files changed, 20 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index f3808672c720..545b556e420c 100644
@@ -1132,6 +1132,7 @@ struct kvm_arch {
 
 	enum kvm_irqchip_mode irqchip_mode;
 	u8 nr_reserved_ioapic_pins;
+	bool eoi_intercept_unsupported;
 
 	bool disabled_lapic_found;
 
arch/x86/kvm/ioapic.c +4 -0
index 816a82515dcd..39a9031e11b1 100644
@@ -311,6 +311,10 @@ void kvm_arch_post_irq_ack_notifier_list_update(struct kvm *kvm)
 {
 	if (!ioapic_in_kernel(kvm))
 		return;
+
+	if (WARN_ON_ONCE(kvm->arch.eoi_intercept_unsupported))
+		return;
+
 	kvm_make_scan_ioapic_request(kvm);
 }
 
arch/x86/kvm/irq_comm.c +7 -2
index d5b72a08e566..bcfac99db579 100644
@@ -123,7 +123,12 @@ EXPORT_SYMBOL_GPL(kvm_set_msi_irq);
 static inline bool kvm_msi_route_invalid(struct kvm *kvm,
 		struct kvm_kernel_irq_routing_entry *e)
 {
-	return kvm->arch.x2apic_format && (e->msi.address_hi & 0xff);
+	struct msi_msg msg = { .address_lo = e->msi.address_lo,
+			       .address_hi = e->msi.address_hi,
+			       .data = e->msi.data };
+	return  (kvm->arch.eoi_intercept_unsupported &&
+		 msg.arch_data.is_level) ||
+		(kvm->arch.x2apic_format && (msg.address_hi & 0xff));
 }
 
 int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e,
@@ -385,7 +390,7 @@ int kvm_setup_empty_irq_routing(struct kvm *kvm)
 
 void kvm_arch_post_irq_routing_update(struct kvm *kvm)
 {
-	if (!irqchip_split(kvm))
+	if (!irqchip_split(kvm) || kvm->arch.eoi_intercept_unsupported)
 		return;
 	kvm_make_scan_ioapic_request(kvm);
 }
arch/x86/kvm/lapic.c +2 -1
index 759952dd1222..1bfcd325d0d2 100644
@@ -281,7 +281,8 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
 	if (old)
 		call_rcu(&old->rcu, kvm_apic_map_free);
 
-	kvm_make_scan_ioapic_request(kvm);
+	if (!kvm->arch.eoi_intercept_unsupported)
+		kvm_make_scan_ioapic_request(kvm);
 }
 
 static inline void apic_set_spiv(struct kvm_lapic *apic, u32 val)
arch/x86/kvm/x86.c +6 -0
index 535f65b0915d..1573dddd1e43 100644
@@ -6110,6 +6110,9 @@ long kvm_arch_vm_ioctl(struct file *filp,
 			goto create_irqchip_unlock;
 
 		r = -EINVAL;
+		if (kvm->arch.eoi_intercept_unsupported)
+			goto create_irqchip_unlock;
+
 		if (kvm->created_vcpus)
 			goto create_irqchip_unlock;
 
@@ -6140,6 +6143,9 @@ long kvm_arch_vm_ioctl(struct file *filp,
 		u.pit_config.flags = KVM_PIT_SPEAKER_DUMMY;
 		goto create_pit;
 	case KVM_CREATE_PIT2:
+		r = -EINVAL;
+		if (kvm->arch.eoi_intercept_unsupported)
+			goto out;
 		r = -EFAULT;
 		if (copy_from_user(&u.pit_config, argp,
 				   sizeof(struct kvm_pit_config)))
Signature · 1 line
2.25.1
#49 lore raw

RFC PATCH v3 27/59 KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched behavior

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add a flag, KVM_DEBUGREG_AUTO_SWITCHED_GUEST, to skip saving/restoring DRs irrespective of any other flags. TDX-SEAM unconditionally saves and restores guest DRs and reset to architectural INIT state on TD exit. So, KVM needs to save host DRs before TD enter without restoring guest DRs and restore host DRs after TD exit.

Opportunistically convert the KVM_DEBUGREG_* definitions to use BIT().

---
 arch/x86/include/asm/kvm_host.h | 10 ++++++++--
 arch/x86/kvm/x86.c              |  3 ++-
 2 files changed, 10 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +8 -2
index 3bc5417f94ec..a2bc4d9caaef 100644
@@ -526,8 +526,14 @@ struct kvm_pmu {
 struct kvm_pmu_ops;
 
 enum {
-	KVM_DEBUGREG_BP_ENABLED = 1,
-	KVM_DEBUGREG_WONT_EXIT = 2,
+	KVM_DEBUGREG_BP_ENABLED		= BIT(0),
+	KVM_DEBUGREG_WONT_EXIT		= BIT(1),
+	KVM_DEBUGREG_RELOAD		= BIT(2),
+	/*
+	 * Guest debug registers are saved/restored by hardware on exit from
+	 * or enter guest. KVM needn't switch them.
+	 */
+	KVM_DEBUGREG_AUTO_SWITCH	= BIT(3),
 };
 
 struct kvm_mtrr_range {
arch/x86/kvm/x86.c +2 -1
index 88b55ddb22a7..8161475082a7 100644
@@ -9910,7 +9910,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 	if (test_thread_flag(TIF_NEED_FPU_LOAD))
 		switch_fpu_return();
 
-	if (unlikely(vcpu->arch.switch_db_regs)) {
+	if (unlikely(vcpu->arch.switch_db_regs & ~KVM_DEBUGREG_AUTO_SWITCH)) {
 		set_debugreg(0, 7);
 		set_debugreg(vcpu->arch.eff_db[0], 0);
 		set_debugreg(vcpu->arch.eff_db[1], 1);
@@ -9950,6 +9950,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 	 */
 	if (unlikely(vcpu->arch.switch_db_regs & KVM_DEBUGREG_WONT_EXIT)) {
 		WARN_ON(vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP);
+		WARN_ON(vcpu->arch.switch_db_regs & KVM_DEBUGREG_AUTO_SWITCH);
 		static_call(kvm_x86_sync_dirty_debug_regs)(vcpu);
 		kvm_update_dr0123(vcpu);
 		kvm_update_dr7(vcpu);
Signature · 1 line
2.25.1
#44 lore raw

RFC PATCH v3 26/59 KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add a second kvm_x86_ops hook in kvm_arch_vm_destroy() to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.

Note, this knowingly creates a discrepancy in nomenclature for SVM as svm_vm_teardown() invokes avic_vm_destroy() and sev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.

---
 arch/x86/include/asm/kvm-x86-ops.h | 1 +
 arch/x86/include/asm/kvm_host.h    | 1 +
 arch/x86/kvm/svm/svm.c             | 5 +++--
 arch/x86/kvm/vmx/vmx.c             | 7 +++++++
 arch/x86/kvm/x86.c                 | 3 ++-
 5 files changed, 14 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index 30754f2b8a99..1009541fd6c2 100644
@@ -20,6 +20,7 @@ KVM_X86_OP(has_emulated_msr)
 KVM_X86_OP(vcpu_after_set_cpuid)
 KVM_X86_OP(is_vm_type_supported)
 KVM_X86_OP(vm_init)
+KVM_X86_OP_NULL(vm_teardown)
 KVM_X86_OP_NULL(vm_destroy)
 KVM_X86_OP(vcpu_create)
 KVM_X86_OP(vcpu_free)
arch/x86/include/asm/kvm_host.h +1 -0
index 74c3c1629563..3bc5417f94ec 100644
@@ -1321,6 +1321,7 @@ struct kvm_x86_ops {
 	bool (*is_vm_type_supported)(unsigned long vm_type);
 	unsigned int vm_size;
 	int (*vm_init)(struct kvm *kvm);
+	void (*vm_teardown)(struct kvm *kvm);
 	void (*vm_destroy)(struct kvm *kvm);
 
 	/* Create, but do not attach this VCPU */
arch/x86/kvm/svm/svm.c +3 -2
index 2ef77d4566a9..1bf410add13b 100644
@@ -4556,7 +4556,7 @@ static void svm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)
 	sev_vcpu_deliver_sipi_vector(vcpu, vector);
 }
 
-static void svm_vm_destroy(struct kvm *kvm)
+static void svm_vm_teardown(struct kvm *kvm)
 {
 	avic_vm_destroy(kvm);
 	sev_vm_destroy(kvm);
@@ -4597,7 +4597,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.is_vm_type_supported = svm_is_vm_type_supported,
 	.vm_size = sizeof(struct kvm_svm),
 	.vm_init = svm_vm_init,
-	.vm_destroy = svm_vm_destroy,
+	.vm_teardown = svm_vm_teardown,
+	.vm_destroy = NULL,
 
 	.prepare_guest_switch = svm_prepare_guest_switch,
 	.vcpu_load = svm_vcpu_load,
arch/x86/kvm/vmx/vmx.c +7 -0
index f2905e00b063..5e4c6ac9fe69 100644
@@ -6890,6 +6890,11 @@ static int vmx_vm_init(struct kvm *kvm)
 	return 0;
 }
 
+static void vmx_vm_destroy(struct kvm *kvm)
+{
+
+}
+
 static int __init vmx_check_processor_compat(void)
 {
 	struct vmcs_config vmcs_conf;
@@ -7513,6 +7518,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = {
 	.is_vm_type_supported = vmx_is_vm_type_supported,
 	.vm_size = sizeof(struct kvm_vmx),
 	.vm_init = vmx_vm_init,
+	.vm_teardown = NULL,
+	.vm_destroy = vmx_vm_destroy,
 
 	.vcpu_create = vmx_create_vcpu,
 	.vcpu_free = vmx_free_vcpu,
arch/x86/kvm/x86.c +2 -1
index f2091c4b928a..88b55ddb22a7 100644
@@ -11613,7 +11613,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 		__x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0);
 		mutex_unlock(&kvm->slots_lock);
 	}
-	static_call_cond(kvm_x86_vm_destroy)(kvm);
+	static_call_cond(kvm_x86_vm_teardown)(kvm);
 	kvm_free_msr_filter(srcu_dereference_check(kvm->arch.msr_filter, &kvm->srcu, 1));
 	kvm_pic_destroy(kvm);
 	kvm_ioapic_destroy(kvm);
@@ -11624,6 +11624,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 	kvm_page_track_cleanup(kvm);
 	kvm_xen_destroy_vm(kvm);
 	kvm_hv_destroy_vm(kvm);
+	static_call_cond(kvm_x86_vm_destroy)(kvm);
 }
 
 static void memslot_rmap_free(struct kvm_memory_slot *slot)
Signature · 1 line
2.25.1
#53 lore raw

RFC PATCH v3 29/59 KVM: x86: Add option to force LAPIC expiration wait

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add an option to skip the IRR check in kvm_wait_lapic_expire(). This will be used by TDX to wait if there is an outstanding notification for a TD, i.e. a virtual interrupt is being triggered via posted interrupt processing. KVM TDX doesn't emulate PI processing, i.e. there will never be a bit set in IRR/ISR, so the default behavior for APICv of querying the IRR doesn't work as intended.

---
 arch/x86/kvm/lapic.c   | 4 ++--
 arch/x86/kvm/lapic.h   | 2 +-
 arch/x86/kvm/svm/svm.c | 2 +-
 arch/x86/kvm/vmx/vmx.c | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)
arch/x86/kvm/lapic.c +2 -2
index 1bfcd325d0d2..beef195b9d15 100644
@@ -1625,12 +1625,12 @@ static void __kvm_wait_lapic_expire(struct kvm_vcpu *vcpu)
 		__wait_lapic_expire(vcpu, tsc_deadline - guest_tsc);
 }
 
-void kvm_wait_lapic_expire(struct kvm_vcpu *vcpu)
+void kvm_wait_lapic_expire(struct kvm_vcpu *vcpu, bool force_wait)
 {
 	if (lapic_in_kernel(vcpu) &&
 	    vcpu->arch.apic->lapic_timer.expired_tscdeadline &&
 	    vcpu->arch.apic->lapic_timer.timer_advance_ns &&
-	    lapic_timer_int_injected(vcpu))
+	    (force_wait || lapic_timer_int_injected(vcpu)))
 		__kvm_wait_lapic_expire(vcpu);
 }
 EXPORT_SYMBOL_GPL(kvm_wait_lapic_expire);
arch/x86/kvm/lapic.h +1 -1
index 2b44e533fc8d..2a0119ef9e96 100644
@@ -233,7 +233,7 @@ static inline int kvm_lapic_latched_init(struct kvm_vcpu *vcpu)
 
 bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector);
 
-void kvm_wait_lapic_expire(struct kvm_vcpu *vcpu);
+void kvm_wait_lapic_expire(struct kvm_vcpu *vcpu, bool force_wait);
 
 void kvm_bitmap_or_dest_vcpus(struct kvm *kvm, struct kvm_lapic_irq *irq,
 			      unsigned long *vcpu_bitmap);
arch/x86/kvm/svm/svm.c +1 -1
index 1bf410add13b..e33cbf9613d9 100644
@@ -3885,7 +3885,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)
 	clgi();
 	kvm_load_guest_xsave_state(vcpu);
 
-	kvm_wait_lapic_expire(vcpu);
+	kvm_wait_lapic_expire(vcpu, false);
 
 	/*
 	 * If this vCPU has touched SPEC_CTRL, restore the guest's value if
arch/x86/kvm/vmx/vmx.c +1 -1
index 5e4c6ac9fe69..1e20e85a958f 100644
@@ -6647,7 +6647,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)
 	if (enable_preemption_timer)
 		vmx_update_hv_timer(vcpu);
 
-	kvm_wait_lapic_expire(vcpu);
+	kvm_wait_lapic_expire(vcpu, false);
 
 	/*
 	 * If this vCPU has touched SPEC_CTRL, restore the guest's value if
Signature · 1 line
2.25.1
#64 lore raw

RFC PATCH v3 37/59 KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Introduce a helper to directly (pun intended) fault-in a TDP page without having to go through the full page fault path. This allows TDX to get the resulting pfn and also allows the RET_PF_* enums to stay in mmu.c where they belong.

---
 arch/x86/kvm/mmu.h     |  3 +++
 arch/x86/kvm/mmu/mmu.c | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+)
arch/x86/kvm/mmu.h +3 -0
index 9c22dedbb228..168dcd4e0102 100644
@@ -202,6 +202,9 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
 	return vcpu->arch.mmu->page_fault(vcpu, &fault);
 }
 
+kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa,
+			       u32 error_code, int max_level);
+
 /*
  * Currently, we have two sorts of write-protection, a) the first one
  * write-protects guest page to sync the guest modification, b) another one is
arch/x86/kvm/mmu/mmu.c +38 -0
index c3effcbb726e..63d2e3e85c08 100644
@@ -4432,6 +4432,44 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)
 	return direct_page_fault(vcpu, fault);
 }
 
+kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa,
+			       u32 error_code, int max_level)
+{
+	int r;
+	struct kvm_page_fault fault = (struct kvm_page_fault) {
+		.addr = gpa,
+		.error_code = error_code,
+		.exec = error_code & PFERR_FETCH_MASK,
+		.write = error_code & PFERR_WRITE_MASK,
+		.present = error_code & PFERR_PRESENT_MASK,
+		.rsvd = error_code & PFERR_RSVD_MASK,
+		.user = error_code & PFERR_USER_MASK,
+		.prefetch = false,
+		.is_tdp = true,
+		.nx_huge_page_workaround_enabled = is_nx_huge_page_enabled(),
+	};
+
+	if (mmu_topup_memory_caches(vcpu, false))
+		return KVM_PFN_ERR_FAULT;
+
+	/*
+	 * Loop on the page fault path to handle the case where an mmu_notifier
+	 * invalidation triggers RET_PF_RETRY.  In the normal page fault path,
+	 * KVM needs to resume the guest in case the invalidation changed any
+	 * of the page fault properties, i.e. the gpa or error code.  For this
+	 * path, the gpa and error code are fixed by the caller, and the caller
+	 * expects failure if and only if the page fault can't be fixed.
+	 */
+	do {
+		fault.max_level = max_level;
+		fault.req_level = PG_LEVEL_4K;
+		fault.goal_level = PG_LEVEL_4K;
+		r = direct_page_fault(vcpu, &fault);
+	} while (r == RET_PF_RETRY && !is_error_noslot_pfn(fault.pfn));
+	return fault.pfn;
+}
+EXPORT_SYMBOL_GPL(kvm_mmu_map_tdp_page);
+
 static void nonpaging_init_context(struct kvm_mmu *context)
 {
 	context->page_fault = nonpaging_page_fault;
Signature · 1 line
2.25.1
#63 lore raw

RFC PATCH v3 36/59 KVM: x86/mmu: Frame in support for private/inaccessible shadow pages

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add kvm_x86_ops hooks to set/clear private SPTEs, i.e. SEPT entries, and to link/free private shadow pages, i.e. non-leaf SEPT pages.

Because SEAMCALLs are bloody expensive, and because KVM's MMU is already complex enough, TDX's SEPT will mirror KVM's shadow pages instead of replacing them outright. This costs extra memory, but is simpler and far more performant.

Add a separate list for tracking active private shadow pages. Zapping and freeing SEPT entries is subject to very different rules than normal pages/memory, and need to be preserved (along with their shadow page counterparts) when KVM gets trigger happy, e.g. zaps everything during a memslot update.

Zap any aliases of a GPA when mapping in a guest that supports guest private GPAs. This is necessary to avoid integrity failures with TDX due to pointing shared and private GPAs at the same HPA.

Do not prefetch private pages (this should probably be a property of the VM).

TDX needs one more bits in spte for SPTE_PRIVATE_ZAPPED. Steal 1 bit from MMIO generation as 62 bit and re-purpose it for non-present SPTE, SPTE_PRIVATE_ZAPPED.

---
 arch/x86/include/asm/kvm-x86-ops.h |   7 +
 arch/x86/include/asm/kvm_host.h    |  32 +-
 arch/x86/kvm/mmu.h                 |  16 +-
 arch/x86/kvm/mmu/mmu.c             | 478 +++++++++++++++++++++++------
 arch/x86/kvm/mmu/mmu_internal.h    |  13 +-
 arch/x86/kvm/mmu/paging_tmpl.h     |  11 +-
 arch/x86/kvm/mmu/spte.h            |  16 +-
 arch/x86/kvm/x86.c                 |  10 +-
 virt/kvm/kvm_main.c                |   1 +
 9 files changed, 463 insertions(+), 121 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +7 -0
index 1009541fd6c2..ef94a535f452 100644
@@ -20,6 +20,7 @@ KVM_X86_OP(has_emulated_msr)
 KVM_X86_OP(vcpu_after_set_cpuid)
 KVM_X86_OP(is_vm_type_supported)
 KVM_X86_OP(vm_init)
+KVM_X86_OP_NULL(mmu_prezap)
 KVM_X86_OP_NULL(vm_teardown)
 KVM_X86_OP_NULL(vm_destroy)
 KVM_X86_OP(vcpu_create)
@@ -89,6 +90,12 @@ KVM_X86_OP(set_tss_addr)
 KVM_X86_OP(set_identity_map_addr)
 KVM_X86_OP(get_mt_mask)
 KVM_X86_OP(load_mmu_pgd)
+KVM_X86_OP(set_private_spte)
+KVM_X86_OP(drop_private_spte)
+KVM_X86_OP(zap_private_spte)
+KVM_X86_OP(unzap_private_spte)
+KVM_X86_OP(link_private_sp)
+KVM_X86_OP(free_private_sp)
 KVM_X86_OP_NULL(has_wbinvd_exit)
 KVM_X86_OP(get_l2_tsc_offset)
 KVM_X86_OP(get_l2_tsc_multiplier)
arch/x86/include/asm/kvm_host.h +29 -3
index 2dab0899d82c..7ce0641d54dd 100644
@@ -320,7 +320,8 @@ union kvm_mmu_page_role {
 		unsigned smap_andnot_wp:1;
 		unsigned ad_disabled:1;
 		unsigned guest_mode:1;
-		unsigned :6;
+		unsigned private:1;
+		unsigned :5;
 
 		/*
 		 * This is left at the top of the word so that
@@ -427,6 +428,7 @@ struct kvm_mmu {
 			 struct kvm_mmu_page *sp);
 	void (*invlpg)(struct kvm_vcpu *vcpu, gva_t gva, hpa_t root_hpa);
 	hpa_t root_hpa;
+	hpa_t private_root_hpa;
 	gpa_t root_pgd;
 	union kvm_mmu_role mmu_role;
 	u8 root_level;
@@ -463,6 +465,8 @@ struct kvm_mmu {
 
 	struct rsvd_bits_validate guest_rsvd_check;
 
+	bool no_prefetch;
+
 	u64 pdptrs[4]; /* pae */
 };
 
@@ -685,6 +689,7 @@ struct kvm_vcpu_arch {
 	struct kvm_mmu_memory_cache mmu_shadow_page_cache;
 	struct kvm_mmu_memory_cache mmu_gfn_array_cache;
 	struct kvm_mmu_memory_cache mmu_page_header_cache;
+	struct kvm_mmu_memory_cache mmu_private_sp_cache;
 
 	/*
 	 * QEMU userspace and the guest each have their own FPU state.
@@ -1052,6 +1057,7 @@ struct kvm_arch {
 	u8 mmu_valid_gen;
 	struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES];
 	struct list_head active_mmu_pages;
+	struct list_head private_mmu_pages;
 	struct list_head zapped_obsolete_pages;
 	struct list_head lpage_disallowed_mmu_pages;
 	struct kvm_page_track_notifier_node mmu_sp_tracker;
@@ -1236,6 +1242,8 @@ struct kvm_arch {
 	hpa_t	hv_root_tdp;
 	spinlock_t hv_root_tdp_lock;
 #endif
+
+	gfn_t gfn_shared_mask;
 };
 
 struct kvm_vm_stat {
@@ -1328,6 +1336,7 @@ struct kvm_x86_ops {
 	bool (*is_vm_type_supported)(unsigned long vm_type);
 	unsigned int vm_size;
 	int (*vm_init)(struct kvm *kvm);
+	void (*mmu_prezap)(struct kvm *kvm);
 	void (*vm_teardown)(struct kvm *kvm);
 	void (*vm_destroy)(struct kvm *kvm);
 
@@ -1423,6 +1432,17 @@ struct kvm_x86_ops {
 	void (*load_mmu_pgd)(struct kvm_vcpu *vcpu, hpa_t root_hpa,
 			     int root_level);
 
+	void (*set_private_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level,
+				 kvm_pfn_t pfn);
+	void (*drop_private_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level,
+				  kvm_pfn_t pfn);
+	void (*zap_private_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level);
+	void (*unzap_private_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level);
+	int (*link_private_sp)(struct kvm *kvm, gfn_t gfn, enum pg_level level,
+			       void *private_sp);
+	int (*free_private_sp)(struct kvm *kvm, gfn_t gfn, enum pg_level level,
+			       void *private_sp);
+
 	bool (*has_wbinvd_exit)(void);
 
 	u64 (*get_l2_tsc_offset)(struct kvm_vcpu *vcpu);
@@ -1602,7 +1622,8 @@ void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm,
 				   const struct kvm_memory_slot *memslot);
 void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm,
 				   const struct kvm_memory_slot *memslot);
-void kvm_mmu_zap_all(struct kvm *kvm);
+void kvm_mmu_zap_all_active(struct kvm *kvm);
+void kvm_mmu_zap_all_private(struct kvm *kvm);
 void kvm_mmu_invalidate_mmio_sptes(struct kvm *kvm, u64 gen);
 unsigned long kvm_mmu_calculate_default_mmu_pages(struct kvm *kvm);
 void kvm_mmu_change_mmu_pages(struct kvm *kvm, unsigned long kvm_nr_mmu_pages);
@@ -1765,7 +1786,9 @@ static inline int __kvm_irq_line_state(unsigned long *irq_state,
 
 #define KVM_MMU_ROOT_CURRENT		BIT(0)
 #define KVM_MMU_ROOT_PREVIOUS(i)	BIT(1+i)
-#define KVM_MMU_ROOTS_ALL		(~0UL)
+#define KVM_MMU_ROOT_PRIVATE		BIT(1+KVM_MMU_NUM_PREV_ROOTS)
+#define KVM_MMU_ROOTS_ALL		((u32)(~KVM_MMU_ROOT_PRIVATE))
+#define KVM_MMU_ROOTS_ALL_INC_PRIVATE	(KVM_MMU_ROOTS_ALL | KVM_MMU_ROOT_PRIVATE)
 
 int kvm_pic_set_irq(struct kvm_pic *pic, int irq, int irq_source_id, int level);
 void kvm_pic_clear_all(struct kvm_pic *pic, int irq_source_id);
@@ -1973,4 +1996,7 @@ int memslot_rmap_alloc(struct kvm_memory_slot *slot, unsigned long npages);
 #define KVM_CLOCK_VALID_FLAGS						\
 	(KVM_CLOCK_TSC_STABLE | KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC)
 
+bool kvm_is_private_gfn(struct kvm *kvm, gfn_t gfn);
+bool kvm_vcpu_is_private_gfn(struct kvm_vcpu *vcpu, gfn_t gfn);
+
 #endif /* _ASM_X86_KVM_HOST_H */
arch/x86/kvm/mmu.h +5 -11
index 79ccee8bbc38..9c22dedbb228 100644
@@ -355,13 +355,7 @@ static inline void kvm_update_page_stats(struct kvm *kvm, int level, int count)
 
 static inline gfn_t kvm_gfn_stolen_mask(struct kvm *kvm)
 {
-	/* Currently there are no stolen bits in KVM */
-	return 0;
-}
-
-static inline gfn_t vcpu_gfn_stolen_mask(struct kvm_vcpu *vcpu)
-{
-	return kvm_gfn_stolen_mask(vcpu->kvm);
+	return kvm->arch.gfn_shared_mask;
 }
 
 static inline gpa_t kvm_gpa_stolen_mask(struct kvm *kvm)
@@ -369,14 +363,14 @@ static inline gpa_t kvm_gpa_stolen_mask(struct kvm *kvm)
 	return kvm_gfn_stolen_mask(kvm) << PAGE_SHIFT;
 }
 
-static inline gpa_t vcpu_gpa_stolen_mask(struct kvm_vcpu *vcpu)
+static inline gpa_t kvm_gpa_unalias(struct kvm *kvm, gpa_t gpa)
 {
-	return kvm_gpa_stolen_mask(vcpu->kvm);
+	return gpa & ~kvm_gpa_stolen_mask(kvm);
 }
 
-static inline gfn_t vcpu_gpa_to_gfn_unalias(struct kvm_vcpu *vcpu, gpa_t gpa)
+static inline gfn_t kvm_gfn_unalias(struct kvm *kvm, gpa_t gpa)
 {
-	return (gpa >> PAGE_SHIFT) & ~vcpu_gfn_stolen_mask(vcpu);
+	return kvm_gpa_unalias(kvm, gpa) >> PAGE_SHIFT;
 }
 
 #endif
arch/x86/kvm/mmu/mmu.c +382 -96

833 lines not on this page · raw message

arch/x86/kvm/mmu/mmu_internal.h +12 -1
index 90de2d2ebfff..d718a1783112 100644
@@ -49,11 +49,12 @@ struct kvm_mmu_page {
 	 */
 	union kvm_mmu_page_role role;
 	gfn_t gfn;
-	gfn_t gfn_stolen_bits;
 
 	u64 *spt;
 	/* hold the gfn of each spte inside spt */
 	gfn_t *gfns;
+	/* associated private shadow page, e.g. SEPT page */
+	void *private_sp;
 	/* Currently serving as active root */
 	union {
 		int root_count;
@@ -105,6 +106,16 @@ static inline int kvm_mmu_page_as_id(struct kvm_mmu_page *sp)
 	return kvm_mmu_role_as_id(sp->role);
 }
 
+static inline bool is_private_sp(struct kvm_mmu_page *sp)
+{
+	return sp->role.private;
+}
+
+static inline bool is_private_spte(u64 *sptep)
+{
+	return is_private_sp(sptep_to_sp(sptep));
+}
+
 static inline bool kvm_vcpu_ad_need_write_protect(struct kvm_vcpu *vcpu)
 {
 	/*
arch/x86/kvm/mmu/paging_tmpl.h +6 -5
index 80e821996728..3e1759c68912 100644
@@ -25,7 +25,7 @@
 	#define guest_walker guest_walker64
 	#define FNAME(name) paging##64_##name
 	#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK
-	#define PT_LVL_ADDR_MASK(vcpu, lvl) (~vcpu_gpa_stolen_mask(vcpu) & \
+	#define PT_LVL_ADDR_MASK(vcpu, lvl) (~kvm_gpa_stolen_mask(vcpu->kvm) & \
 					     PT64_LVL_ADDR_MASK(lvl))
 	#define PT_LVL_OFFSET_MASK(lvl) PT64_LVL_OFFSET_MASK(lvl)
 	#define PT_INDEX(addr, level) PT64_INDEX(addr, level)
@@ -567,7 +567,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,
 	gfn_t gfn;
 	kvm_pfn_t pfn;
 
-	WARN_ON(gpte & vcpu_gpa_stolen_mask(vcpu));
+	WARN_ON(gpte & kvm_gpa_stolen_mask(vcpu->kvm));
 
 	if (FNAME(prefetch_invalid_gpte)(vcpu, sp, spte, gpte))
 		return false;
@@ -670,7 +670,7 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
 	WARN_ON_ONCE(gw->gfn != base_gfn);
 	direct_access = gw->pte_access;
 
-	WARN_ON(fault->addr & vcpu_gpa_stolen_mask(vcpu));
+	WARN_ON(fault->addr & kvm_gpa_stolen_mask(vcpu->kvm));
 
 	top_level = vcpu->arch.mmu->root_level;
 	if (top_level == PT32E_ROOT_LEVEL)
@@ -700,7 +700,7 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
 			table_gfn = gw->table_gfn[it.level - 2];
 			access = gw->pt_access[it.level - 2];
 			sp = kvm_mmu_get_page(vcpu, table_gfn, fault->addr,
-					      it.level-1, false, access);
+					      it.level-1, false, access, false);
 			/*
 			 * We must synchronize the pagetable before linking it
 			 * because the guest doesn't need to flush tlb when
@@ -757,7 +757,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
 
 		if (!is_shadow_present_pte(*it.sptep)) {
 			sp = kvm_mmu_get_page(vcpu, base_gfn, fault->addr,
-					      it.level - 1, true, direct_access);
+					      it.level - 1, true, direct_access,
+					      false);
 			link_shadow_page(vcpu, it.sptep, sp);
 			if (fault->huge_page_disallowed &&
 			    fault->req_level >= it.level)
arch/x86/kvm/mmu/spte.h +12 -4
index b53b301301dc..43f0d2a773f7 100644
@@ -14,6 +14,9 @@
  */
 #define SPTE_MMU_PRESENT_MASK		BIT_ULL(11)
 
+/* Masks that used to track metadata for not-present SPTEs. */
+#define SPTE_PRIVATE_ZAPPED	BIT_ULL(62)
+
 /*
  * TDP SPTES (more specifically, EPT SPTEs) may not have A/D bits, and may also
  * be restricted to using write-protection (for L2 when CPU dirty logging, i.e.
@@ -95,11 +98,11 @@ static_assert(!(EPT_SPTE_MMU_WRITABLE & SHADOW_ACC_TRACK_SAVED_MASK));
 #undef SHADOW_ACC_TRACK_SAVED_MASK
 
 /*
- * Due to limited space in PTEs, the MMIO generation is a 19 bit subset of
+ * Due to limited space in PTEs, the MMIO generation is a 18 bit subset of
  * the memslots generation and is derived as follows:
  *
  * Bits 0-7 of the MMIO generation are propagated to spte bits 3-10
- * Bits 8-18 of the MMIO generation are propagated to spte bits 52-62
+ * Bits 8-17 of the MMIO generation are propagated to spte bits 52-61
  *
  * The KVM_MEMSLOT_GEN_UPDATE_IN_PROGRESS flag is intentionally not included in
  * the MMIO generation number, as doing so would require stealing a bit from
@@ -113,7 +116,7 @@ static_assert(!(EPT_SPTE_MMU_WRITABLE & SHADOW_ACC_TRACK_SAVED_MASK));
 #define MMIO_SPTE_GEN_LOW_END		10
 
 #define MMIO_SPTE_GEN_HIGH_START	52
-#define MMIO_SPTE_GEN_HIGH_END		62
+#define MMIO_SPTE_GEN_HIGH_END		61
 
 #define MMIO_SPTE_GEN_LOW_MASK		GENMASK_ULL(MMIO_SPTE_GEN_LOW_END, \
 						    MMIO_SPTE_GEN_LOW_START)
@@ -126,7 +129,7 @@ static_assert(!(SPTE_MMU_PRESENT_MASK &
 #define MMIO_SPTE_GEN_HIGH_BITS		(MMIO_SPTE_GEN_HIGH_END - MMIO_SPTE_GEN_HIGH_START + 1)
 
 /* remember to adjust the comment above as well if you change these */
-static_assert(MMIO_SPTE_GEN_LOW_BITS == 8 && MMIO_SPTE_GEN_HIGH_BITS == 11);
+static_assert(MMIO_SPTE_GEN_LOW_BITS == 8 && MMIO_SPTE_GEN_HIGH_BITS == 10);
 
 #define MMIO_SPTE_GEN_LOW_SHIFT		(MMIO_SPTE_GEN_LOW_START - 0)
 #define MMIO_SPTE_GEN_HIGH_SHIFT	(MMIO_SPTE_GEN_HIGH_START - MMIO_SPTE_GEN_LOW_BITS)
@@ -267,6 +270,11 @@ static inline bool is_access_track_spte(u64 spte)
 	return !spte_ad_enabled(spte) && (spte & shadow_acc_track_mask) == 0;
 }
 
+static inline bool is_zapped_private_pte(u64 pte)
+{
+	return !!(pte & SPTE_PRIVATE_ZAPPED);
+}
+
 static inline bool is_large_pte(u64 pte)
 {
 	return pte & PT_PAGE_SIZE_MASK;
arch/x86/kvm/x86.c +9 -1
index c6e56f105673..4dd8ec2641a2 100644
@@ -11436,6 +11436,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 
 	INIT_HLIST_HEAD(&kvm->arch.mask_notifier_list);
 	INIT_LIST_HEAD(&kvm->arch.active_mmu_pages);
+	INIT_LIST_HEAD(&kvm->arch.private_mmu_pages);
 	INIT_LIST_HEAD(&kvm->arch.zapped_obsolete_pages);
 	INIT_LIST_HEAD(&kvm->arch.lpage_disallowed_mmu_pages);
 	INIT_LIST_HEAD(&kvm->arch.assigned_dev_head);
@@ -11872,7 +11873,14 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
 
 void kvm_arch_flush_shadow_all(struct kvm *kvm)
 {
-	kvm_mmu_zap_all(kvm);
+	/*
+	 * kvm_mmu_zap_all_active() zaps both private and shared page tables.
+	 * Before tearing down private page tables, TDX requires TD has started
+	 * to be destroyed (i.e. keyID must have been reclaimed, etc).  Invoke
+	 * kvm_x86_mmu_prezap() for this.
+	 */
+	static_call_cond(kvm_x86_mmu_prezap)(kvm);
+	kvm_mmu_zap_all_active(kvm);
 }
 
 void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
virt/kvm/kvm_main.c +1 -0
index 5bf456734e7d..b56d97d43ad9 100644
@@ -190,6 +190,7 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn)
 
 	return true;
 }
+EXPORT_SYMBOL_GPL(kvm_is_reserved_pfn);
 
 /*
  * Switches to specified vcpu, until a matching vcpu_put()
Signature · 1 line
2.25.1
#65 lore raw

RFC PATCH v3 38/59 KVM: x86/mmu: Allow per-VM override of the TDP max page level

In reply to [email protected]

From: Sean Christopherson <[email protected]>

TODO: This is tentative patch. Support large page and delete this patch.

Allow TDX to effectively disable large pages, as SEPT will initially support only 4k pages.

---
 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/mmu.h              | 2 +-
 arch/x86/kvm/mmu/mmu.c          | 2 ++
 3 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
index 7ce0641d54dd..a2027bc14e41 100644
@@ -1054,6 +1054,7 @@ struct kvm_arch {
 	unsigned long n_requested_mmu_pages;
 	unsigned long n_max_mmu_pages;
 	unsigned int indirect_shadow_pages;
+	int tdp_max_page_level;
 	u8 mmu_valid_gen;
 	struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES];
 	struct list_head active_mmu_pages;
arch/x86/kvm/mmu.h +1 -1
index 168dcd4e0102..2c4b8fde66d9 100644
@@ -191,7 +191,7 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
 		.is_tdp = likely(vcpu->arch.mmu->page_fault == kvm_tdp_page_fault),
 		.nx_huge_page_workaround_enabled = is_nx_huge_page_enabled(),
 
-		.max_level = KVM_MAX_HUGEPAGE_LEVEL,
+		.max_level = vcpu->kvm->arch.tdp_max_page_level,
 		.req_level = PG_LEVEL_4K,
 		.goal_level = PG_LEVEL_4K,
 	};
arch/x86/kvm/mmu/mmu.c +2 -0
index 63d2e3e85c08..7d3830508a44 100644
@@ -6102,6 +6102,8 @@ void kvm_mmu_init_vm(struct kvm *kvm)
 	node->track_write = kvm_mmu_pte_write;
 	node->track_flush_slot = kvm_mmu_invalidate_zap_pages_in_memslot;
 	kvm_page_track_register_notifier(kvm, node);
+
+	kvm->arch.tdp_max_page_level = KVM_MAX_HUGEPAGE_LEVEL;
 }
 
 void kvm_mmu_uninit_vm(struct kvm *kvm)
Signature · 1 line
2.25.1
#60 lore raw

RFC PATCH v3 33/59 KVM: x86/mmu: Ignore bits 63 and 62 when checking for "present" SPTEs

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Ignore bits 63 and 62 when checking for present SPTEs to allow setting said bits in not-present SPTEs. TDX will set bit 63 in "zero" SPTEs to suppress #VEs (TDX-SEAM unconditionally enables EPT Violation #VE), and will use bit 62 to track zapped private SPTEs.

---
 arch/x86/kvm/mmu/paging_tmpl.h |  2 +-
 arch/x86/kvm/mmu/spte.h        | 13 +++++++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu/paging_tmpl.h +1 -1
index 3a515c71e09c..80e821996728 100644
@@ -1102,7 +1102,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)
 		gpa_t pte_gpa;
 		gfn_t gfn;
 
-		if (!sp->spt[i])
+		if (!__is_shadow_present_pte(sp->spt[i]))
 			continue;
 
 		pte_gpa = first_pte_gpa + i * sizeof(pt_element_t);
arch/x86/kvm/mmu/spte.h +13 -0
index cc432f9a966b..56b6dd750fb1 100644
@@ -208,6 +208,19 @@ static inline bool is_mmio_spte(u64 spte)
 	       likely(shadow_mmio_value);
 }
 
+static inline bool __is_shadow_present_pte(u64 pte)
+{
+	/*
+	 * Ignore bits 63 and 62 so that they can be set in SPTEs that are well
+	 * and truly not present.  We can't use the sane/obvious approach of
+	 * querying bits 2:0 (RWX or P) because EPT without A/D bits will clear
+	 * RWX of a "present" SPTE to do access tracking.  Tracking updates can
+	 * be done out of mmu_lock, so even the flushing logic needs to treat
+	 * such SPTEs as present.
+	 */
+	return !!(pte << 2);
+}
+
 static inline bool is_shadow_present_pte(u64 pte)
 {
 	return !!(pte & SPTE_MMU_PRESENT_MASK);
Signature · 1 line
2.25.1
#66 lore raw

RFC PATCH v3 39/59 KVM: VMX: Modify NMI and INTR handlers to take intr_info as param

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Pass intr_info to the NMI and INTR handlers instead of pulling it from vcpu_vmx in preparation for sharing the bulk of the handlers with TDX.

---
 arch/x86/kvm/vmx/vmx.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
arch/x86/kvm/vmx/vmx.c +7 -8
index 1e20e85a958f..f243d5ecf543 100644
@@ -6302,25 +6302,24 @@ static void handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu,
 	kvm_after_interrupt(vcpu);
 }
 
-static void handle_exception_nmi_irqoff(struct vcpu_vmx *vmx)
+static void handle_exception_nmi_irqoff(struct kvm_vcpu *vcpu, u32 intr_info)
 {
 	const unsigned long nmi_entry = (unsigned long)asm_exc_nmi_noist;
-	u32 intr_info = vmx_get_intr_info(&vmx->vcpu);
 
 	/* if exit due to PF check for async PF */
 	if (is_page_fault(intr_info))
-		vmx->vcpu.arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags();
+		vcpu->arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags();
 	/* Handle machine checks before interrupts are enabled */
 	else if (is_machine_check(intr_info))
 		kvm_machine_check();
 	/* We need to handle NMIs before interrupts are enabled */
 	else if (is_nmi(intr_info))
-		handle_interrupt_nmi_irqoff(&vmx->vcpu, nmi_entry);
+		handle_interrupt_nmi_irqoff(vcpu, nmi_entry);
 }
 
-static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu)
+static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu,
+					     u32 intr_info)
 {
-	u32 intr_info = vmx_get_intr_info(vcpu);
 	unsigned int vector = intr_info & INTR_INFO_VECTOR_MASK;
 	gate_desc *desc = (gate_desc *)host_idt_base + vector;
 
@@ -6339,9 +6338,9 @@ static void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu)
 		return;
 
 	if (vmx->exit_reason.basic == EXIT_REASON_EXTERNAL_INTERRUPT)
-		handle_external_interrupt_irqoff(vcpu);
+		handle_external_interrupt_irqoff(vcpu, vmx_get_intr_info(vcpu));
 	else if (vmx->exit_reason.basic == EXIT_REASON_EXCEPTION_NMI)
-		handle_exception_nmi_irqoff(vmx);
+		handle_exception_nmi_irqoff(vcpu, vmx_get_intr_info(vcpu));
 }
 
 /*
Signature · 1 line
2.25.1
#75 lore raw

RFC PATCH v3 45/59 KVM: VMX: Move setting of EPT MMU masks to common VT-x code

In reply to [email protected]

From: Sean Christopherson <[email protected]>

---
 arch/x86/kvm/vmx/main.c | 5 +++++
 arch/x86/kvm/vmx/vmx.c  | 4 ----
 2 files changed, 5 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/main.c +5 -0
index 146dd6a317e5..ef89ed0457d5 100644
@@ -4,6 +4,7 @@
 #include "x86_ops.h"
 #include "vmx.h"
 #include "nested.h"
+#include "mmu.h"
 #include "pmu.h"
 
 static int __init vt_cpu_has_kvm_support(void)
@@ -35,6 +36,10 @@ static __init int vt_hardware_setup(void)
 	if (ret)
 		return ret;
 
+	if (enable_ept)
+		kvm_mmu_set_ept_masks(enable_ept_ad_bits,
+				      cpu_has_vmx_ept_execute_only());
+
 	return 0;
 }
 
arch/x86/kvm/vmx/vmx.c +0 -4
index 404afac897bc..8b2e57de6627 100644
@@ -7546,10 +7546,6 @@ __init int hardware_setup(void)
 
 	set_bit(0, vmx_vpid_bitmap); /* 0 is reserved for host */
 
-	if (enable_ept)
-		kvm_mmu_set_ept_masks(enable_ept_ad_bits,
-				      cpu_has_vmx_ept_execute_only());
-
 	if (!enable_ept)
 		ept_lpage_level = 0;
 	else if (cpu_has_vmx_ept_1g_page())
Signature · 1 line
2.25.1
#70 lore raw

RFC PATCH v3 41/59 KVM: VMX: Split out guts of EPT violation to common/exposed function

In reply to [email protected]

From: Sean Christopherson <[email protected]>

---
 arch/x86/kvm/vmx/common.h | 29 +++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c    | 33 +++++----------------------------
 2 files changed, 34 insertions(+), 28 deletions(-)
arch/x86/kvm/vmx/common.h +29 -0
index 81c73f30d01d..9e5865b05d47 100644
@@ -5,8 +5,11 @@
 #include <linux/kvm_host.h>
 
 #include <asm/traps.h>
+#include <asm/vmx.h>
 
+#include "mmu.h"
 #include "vmcs.h"
+#include "vmx.h"
 #include "x86.h"
 
 extern unsigned long vmx_host_idt_base;
@@ -49,4 +52,30 @@ static inline void vmx_handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu,
 	vmx_handle_interrupt_nmi_irqoff(vcpu, gate_offset(desc));
 }
 
+static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa,
+					     unsigned long exit_qualification)
+{
+	u64 error_code;
+
+	/* Is it a read fault? */
+	error_code = (exit_qualification & EPT_VIOLATION_ACC_READ)
+		     ? PFERR_USER_MASK : 0;
+	/* Is it a write fault? */
+	error_code |= (exit_qualification & EPT_VIOLATION_ACC_WRITE)
+		      ? PFERR_WRITE_MASK : 0;
+	/* Is it a fetch fault? */
+	error_code |= (exit_qualification & EPT_VIOLATION_ACC_INSTR)
+		      ? PFERR_FETCH_MASK : 0;
+	/* ept page table entry is present? */
+	error_code |= (exit_qualification &
+		       (EPT_VIOLATION_READABLE | EPT_VIOLATION_WRITABLE |
+			EPT_VIOLATION_EXECUTABLE))
+		      ? PFERR_PRESENT_MASK : 0;
+
+	error_code |= (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) != 0 ?
+	       PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK;
+
+	return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0);
+}
+
 #endif /* __KVM_X86_VMX_COMMON_H */
arch/x86/kvm/vmx/vmx.c +5 -28
index 4730fd15bea6..92b2c4ac627c 100644
@@ -5230,11 +5230,10 @@ static int handle_task_switch(struct kvm_vcpu *vcpu)
 
 static int handle_ept_violation(struct kvm_vcpu *vcpu)
 {
-	unsigned long exit_qualification;
-	gpa_t gpa;
-	u64 error_code;
+	unsigned long exit_qualification = vmx_get_exit_qual(vcpu);
+	gpa_t gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS);
 
-	exit_qualification = vmx_get_exit_qual(vcpu);
+	trace_kvm_page_fault(gpa, exit_qualification);
 
 	/*
 	 * EPT violation happened while executing iret from NMI,
@@ -5243,31 +5242,9 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)
 	 * AAK134, BY25.
 	 */
 	if (!(to_vmx(vcpu)->idt_vectoring_info & VECTORING_INFO_VALID_MASK) &&
-			enable_vnmi &&
-			(exit_qualification & INTR_INFO_UNBLOCK_NMI))
+	    enable_vnmi && (exit_qualification & INTR_INFO_UNBLOCK_NMI))
 		vmcs_set_bits(GUEST_INTERRUPTIBILITY_INFO, GUEST_INTR_STATE_NMI);
 
-	gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS);
-	trace_kvm_page_fault(gpa, exit_qualification);
-
-	/* Is it a read fault? */
-	error_code = (exit_qualification & EPT_VIOLATION_ACC_READ)
-		     ? PFERR_USER_MASK : 0;
-	/* Is it a write fault? */
-	error_code |= (exit_qualification & EPT_VIOLATION_ACC_WRITE)
-		      ? PFERR_WRITE_MASK : 0;
-	/* Is it a fetch fault? */
-	error_code |= (exit_qualification & EPT_VIOLATION_ACC_INSTR)
-		      ? PFERR_FETCH_MASK : 0;
-	/* ept page table entry is present? */
-	error_code |= (exit_qualification &
-		       (EPT_VIOLATION_READABLE | EPT_VIOLATION_WRITABLE |
-			EPT_VIOLATION_EXECUTABLE))
-		      ? PFERR_PRESENT_MASK : 0;
-
-	error_code |= (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) != 0 ?
-	       PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK;
-
 	vcpu->arch.exit_qualification = exit_qualification;
 
 	/*
@@ -5281,7 +5258,7 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)
 	if (unlikely(allow_smaller_maxphyaddr && kvm_vcpu_is_illegal_gpa(vcpu, gpa)))
 		return kvm_emulate_instruction(vcpu, 0);
 
-	return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0);
+	return __vmx_handle_ept_violation(vcpu, gpa, exit_qualification);
 }
 
 static int handle_ept_misconfig(struct kvm_vcpu *vcpu)
Signature · 1 line
2.25.1
#57 lore raw

RFC PATCH v3 31/59 KVM: x86: Add infrastructure for stolen GPA bits

In reply to [email protected]

From: Rick Edgecombe <[email protected]>

Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA alising will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.

Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.

With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.

To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.

To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.

GFNs (without stolen bits) continue to be used to:
-Specify physical memory by userspace via memslots -Map GPAs to TDP PTEs via RMAP
-Specify dirty tracking and write protection -Look up MTRR types
-Inject async page faults

Since there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN, kvm_handle_hva() based operations will be applied to both aliases.

In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.

For TLB flushes that are address based, make sure to flush both aliases in the stolen bits case.

Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.

---
 arch/x86/kvm/mmu.h              | 27 +++++++++++
 arch/x86/kvm/mmu/mmu.c          | 82 +++++++++++++++++++++++----------
 arch/x86/kvm/mmu/mmu_internal.h |  1 +
 arch/x86/kvm/mmu/paging_tmpl.h  | 25 ++++++----
 4 files changed, 100 insertions(+), 35 deletions(-)
arch/x86/kvm/mmu.h +27 -0
index 9ae6168d381e..583483bb6f71 100644
@@ -351,4 +351,31 @@ static inline void kvm_update_page_stats(struct kvm *kvm, int level, int count)
 {
 	atomic64_add(count, &kvm->stat.pages[level - 1]);
 }
+
+static inline gfn_t kvm_gfn_stolen_mask(struct kvm *kvm)
+{
+	/* Currently there are no stolen bits in KVM */
+	return 0;
+}
+
+static inline gfn_t vcpu_gfn_stolen_mask(struct kvm_vcpu *vcpu)
+{
+	return kvm_gfn_stolen_mask(vcpu->kvm);
+}
+
+static inline gpa_t kvm_gpa_stolen_mask(struct kvm *kvm)
+{
+	return kvm_gfn_stolen_mask(kvm) << PAGE_SHIFT;
+}
+
+static inline gpa_t vcpu_gpa_stolen_mask(struct kvm_vcpu *vcpu)
+{
+	return kvm_gpa_stolen_mask(vcpu->kvm);
+}
+
+static inline gfn_t vcpu_gpa_to_gfn_unalias(struct kvm_vcpu *vcpu, gpa_t gpa)
+{
+	return (gpa >> PAGE_SHIFT) & ~vcpu_gfn_stolen_mask(vcpu);
+}
+
 #endif
arch/x86/kvm/mmu/mmu.c +57 -25

153 lines not on this page · raw message

arch/x86/kvm/mmu/mmu_internal.h +1 -0
index 52c6527b1a06..90de2d2ebfff 100644
@@ -49,6 +49,7 @@ struct kvm_mmu_page {
 	 */
 	union kvm_mmu_page_role role;
 	gfn_t gfn;
+	gfn_t gfn_stolen_bits;
 
 	u64 *spt;
 	/* hold the gfn of each spte inside spt */
arch/x86/kvm/mmu/paging_tmpl.h +15 -10
index f87d36898c44..3a515c71e09c 100644
@@ -25,7 +25,8 @@
 	#define guest_walker guest_walker64
 	#define FNAME(name) paging##64_##name
 	#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK
-	#define PT_LVL_ADDR_MASK(lvl) PT64_LVL_ADDR_MASK(lvl)
+	#define PT_LVL_ADDR_MASK(vcpu, lvl) (~vcpu_gpa_stolen_mask(vcpu) & \
+					     PT64_LVL_ADDR_MASK(lvl))
 	#define PT_LVL_OFFSET_MASK(lvl) PT64_LVL_OFFSET_MASK(lvl)
 	#define PT_INDEX(addr, level) PT64_INDEX(addr, level)
 	#define PT_LEVEL_BITS PT64_LEVEL_BITS
@@ -44,7 +45,7 @@
 	#define guest_walker guest_walker32
 	#define FNAME(name) paging##32_##name
 	#define PT_BASE_ADDR_MASK PT32_BASE_ADDR_MASK
-	#define PT_LVL_ADDR_MASK(lvl) PT32_LVL_ADDR_MASK(lvl)
+	#define PT_LVL_ADDR_MASK(vcpu, lvl) PT32_LVL_ADDR_MASK(lvl)
 	#define PT_LVL_OFFSET_MASK(lvl) PT32_LVL_OFFSET_MASK(lvl)
 	#define PT_INDEX(addr, level) PT32_INDEX(addr, level)
 	#define PT_LEVEL_BITS PT32_LEVEL_BITS
@@ -58,7 +59,7 @@
 	#define guest_walker guest_walkerEPT
 	#define FNAME(name) ept_##name
 	#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK
-	#define PT_LVL_ADDR_MASK(lvl) PT64_LVL_ADDR_MASK(lvl)
+	#define PT_LVL_ADDR_MASK(vcpu, lvl) PT64_LVL_ADDR_MASK(lvl)
 	#define PT_LVL_OFFSET_MASK(lvl) PT64_LVL_OFFSET_MASK(lvl)
 	#define PT_INDEX(addr, level) PT64_INDEX(addr, level)
 	#define PT_LEVEL_BITS PT64_LEVEL_BITS
@@ -75,7 +76,7 @@
 #define PT_GUEST_ACCESSED_MASK (1 << PT_GUEST_ACCESSED_SHIFT)
 
 #define gpte_to_gfn_lvl FNAME(gpte_to_gfn_lvl)
-#define gpte_to_gfn(pte) gpte_to_gfn_lvl((pte), PG_LEVEL_4K)
+#define gpte_to_gfn(vcpu, pte) gpte_to_gfn_lvl(vcpu, pte, PG_LEVEL_4K)
 
 /*
  * The guest_walker structure emulates the behavior of the hardware page
@@ -96,9 +97,9 @@ struct guest_walker {
 	struct x86_exception fault;
 };
 
-static gfn_t gpte_to_gfn_lvl(pt_element_t gpte, int lvl)
+static gfn_t gpte_to_gfn_lvl(struct kvm_vcpu *vcpu, pt_element_t gpte, int lvl)
 {
-	return (gpte & PT_LVL_ADDR_MASK(lvl)) >> PAGE_SHIFT;
+	return (gpte & PT_LVL_ADDR_MASK(vcpu, lvl)) >> PAGE_SHIFT;
 }
 
 static inline void FNAME(protect_clean_gpte)(struct kvm_mmu *mmu, unsigned *access,
@@ -395,7 +396,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,
 		--walker->level;
 
 		index = PT_INDEX(addr, walker->level);
-		table_gfn = gpte_to_gfn(pte);
+		table_gfn = gpte_to_gfn(vcpu, pte);
 		offset    = index * sizeof(pt_element_t);
 		pte_gpa   = gfn_to_gpa(table_gfn) + offset;
 
@@ -461,7 +462,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,
 	if (unlikely(errcode))
 		goto error;
 
-	gfn = gpte_to_gfn_lvl(pte, walker->level);
+	gfn = gpte_to_gfn_lvl(vcpu, pte, walker->level);
 	gfn += (addr & PT_LVL_OFFSET_MASK(walker->level)) >> PAGE_SHIFT;
 
 	if (PTTYPE == 32 && walker->level > PG_LEVEL_4K && is_cpuid_PSE36())
@@ -566,12 +567,14 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,
 	gfn_t gfn;
 	kvm_pfn_t pfn;
 
+	WARN_ON(gpte & vcpu_gpa_stolen_mask(vcpu));
+
 	if (FNAME(prefetch_invalid_gpte)(vcpu, sp, spte, gpte))
 		return false;
 
 	pgprintk("%s: gpte %llx spte %p\n", __func__, (u64)gpte, spte);
 
-	gfn = gpte_to_gfn(gpte);
+	gfn = gpte_to_gfn(vcpu, gpte);
 	pte_access = sp->role.access & FNAME(gpte_access)(gpte);
 	FNAME(protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);
 
@@ -667,6 +670,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,
 	WARN_ON_ONCE(gw->gfn != base_gfn);
 	direct_access = gw->pte_access;
 
+	WARN_ON(fault->addr & vcpu_gpa_stolen_mask(vcpu));
+
 	top_level = vcpu->arch.mmu->root_level;
 	if (top_level == PT32E_ROOT_LEVEL)
 		top_level = PT32_ROOT_LEVEL;
@@ -1111,7 +1116,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)
 			continue;
 		}
 
-		gfn = gpte_to_gfn(gpte);
+		gfn = gpte_to_gfn(vcpu, gpte);
 		pte_access = sp->role.access;
 		pte_access &= FNAME(gpte_access)(gpte);
 		FNAME(protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);
Signature · 1 line
2.25.1
#73 lore raw

RFC PATCH v3 43/59 KVM: VMX: Define VMCS encodings for shared EPT pointer

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add the VMCS field encoding for the shared EPTP, which will be used by TDX to have separate EPT walks for private GPAs (existing EPTP) versus shared GPAs (new shared EPTP).

---
 arch/x86/include/asm/vmx.h | 2 ++
 1 file changed, 2 insertions(+)
arch/x86/include/asm/vmx.h +2 -0
index 132981276a2f..56b3d32941fd 100644
@@ -235,6 +235,8 @@ enum vmcs_field {
 	ENCLS_EXITING_BITMAP_HIGH	= 0x0000202F,
 	TSC_MULTIPLIER                  = 0x00002032,
 	TSC_MULTIPLIER_HIGH             = 0x00002033,
+	SHARED_EPT_POINTER		= 0x0000203C,
+	SHARED_EPT_POINTER_HIGH		= 0x0000203D,
 	GUEST_PHYSICAL_ADDRESS          = 0x00002400,
 	GUEST_PHYSICAL_ADDRESS_HIGH     = 0x00002401,
 	VMCS_LINK_POINTER               = 0x00002800,
Signature · 1 line
2.25.1
#68 lore raw

RFC PATCH v3 40/59 KVM: VMX: Move NMI/exception handler to common helper

In reply to [email protected]

From: Sean Christopherson <[email protected]>

---
 arch/x86/kvm/vmx/common.h | 52 +++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c    | 52 ++++++---------------------------------
 2 files changed, 60 insertions(+), 44 deletions(-)
 create mode 100644 arch/x86/kvm/vmx/common.h
arch/x86/kvm/vmx/common.h new file +52 -0
new file mode 100644
index 000000000000..81c73f30d01d
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __KVM_X86_VMX_COMMON_H
+#define __KVM_X86_VMX_COMMON_H
+
+#include <linux/kvm_host.h>
+
+#include <asm/traps.h>
+
+#include "vmcs.h"
+#include "x86.h"
+
+extern unsigned long vmx_host_idt_base;
+void vmx_do_interrupt_nmi_irqoff(unsigned long entry);
+
+static inline void vmx_handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu,
+				     unsigned long entry)
+{
+	kvm_before_interrupt(vcpu);
+	vmx_do_interrupt_nmi_irqoff(entry);
+	kvm_after_interrupt(vcpu);
+}
+
+static inline void vmx_handle_exception_nmi_irqoff(struct kvm_vcpu *vcpu,
+						   u32 intr_info)
+{
+	const unsigned long nmi_entry = (unsigned long)asm_exc_nmi_noist;
+
+	/* if exit due to PF check for async PF */
+	if (is_page_fault(intr_info))
+		vcpu->arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags();
+	/* Handle machine checks before interrupts are enabled */
+	else if (is_machine_check(intr_info))
+		kvm_machine_check();
+	/* We need to handle NMIs before interrupts are enabled */
+	else if (is_nmi(intr_info))
+		vmx_handle_interrupt_nmi_irqoff(vcpu, nmi_entry);
+}
+
+static inline void vmx_handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu,
+							u32 intr_info)
+{
+	unsigned int vector = intr_info & INTR_INFO_VECTOR_MASK;
+	gate_desc *desc = (gate_desc *)vmx_host_idt_base + vector;
+
+	if (KVM_BUG(!is_external_intr(intr_info), vcpu->kvm,
+	    "KVM: unexpected VM-Exit interrupt info: 0x%x", intr_info))
+		return;
+
+	vmx_handle_interrupt_nmi_irqoff(vcpu, gate_offset(desc));
+}
+
+#endif /* __KVM_X86_VMX_COMMON_H */
arch/x86/kvm/vmx/vmx.c +8 -44

95 lines not on this page · raw message

Signature · 1 line
2.25.1
#72 lore raw

RFC PATCH v3 42/59 KVM: VMX: Define EPT Violation architectural bits

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Define the EPT Violation #VE control bit, #VE info VMCS fields, and the suppress #VE bit for EPT entries.

---
 arch/x86/include/asm/vmx.h | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/include/asm/vmx.h +4 -0
index 035dfdafa2c1..132981276a2f 100644
@@ -78,6 +78,7 @@ struct vmcs {
 #define SECONDARY_EXEC_ENCLS_EXITING		VMCS_CONTROL_BIT(ENCLS_EXITING)
 #define SECONDARY_EXEC_RDSEED_EXITING		VMCS_CONTROL_BIT(RDSEED_EXITING)
 #define SECONDARY_EXEC_ENABLE_PML               VMCS_CONTROL_BIT(PAGE_MOD_LOGGING)
+#define SECONDARY_EXEC_EPT_VIOLATION_VE		VMCS_CONTROL_BIT(EPT_VIOLATION_VE)
 #define SECONDARY_EXEC_PT_CONCEAL_VMX		VMCS_CONTROL_BIT(PT_CONCEAL_VMX)
 #define SECONDARY_EXEC_XSAVES			VMCS_CONTROL_BIT(XSAVES)
 #define SECONDARY_EXEC_MODE_BASED_EPT_EXEC	VMCS_CONTROL_BIT(MODE_BASED_EPT_EXEC)
@@ -226,6 +227,8 @@ enum vmcs_field {
 	VMREAD_BITMAP_HIGH              = 0x00002027,
 	VMWRITE_BITMAP                  = 0x00002028,
 	VMWRITE_BITMAP_HIGH             = 0x00002029,
+	VE_INFO_ADDRESS                 = 0x0000202A,
+	VE_INFO_ADDRESS_HIGH            = 0x0000202B,
 	XSS_EXIT_BITMAP                 = 0x0000202C,
 	XSS_EXIT_BITMAP_HIGH            = 0x0000202D,
 	ENCLS_EXITING_BITMAP		= 0x0000202E,
@@ -509,6 +512,7 @@ enum vmcs_field {
 #define VMX_EPT_IPAT_BIT    			(1ull << 6)
 #define VMX_EPT_ACCESS_BIT			(1ull << 8)
 #define VMX_EPT_DIRTY_BIT			(1ull << 9)
+#define VMX_EPT_SUPPRESS_VE_BIT			(1ull << 63)
 #define VMX_EPT_RWX_MASK                        (VMX_EPT_READABLE_MASK |       \
 						 VMX_EPT_WRITABLE_MASK |       \
 						 VMX_EPT_EXECUTABLE_MASK)
Signature · 1 line
2.25.1
#74 lore raw

RFC PATCH v3 44/59 KVM: VMX: Add 'main.c' to wrap VMX and TDX

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Wrap the VMX kvm_x86_ops hooks in preparation of adding TDX, which can coexist with VMX, i.e. KVM can run both VMs and TDs. Use 'vt' for the naming scheme as a nod to VT-x and as a concatenation of VmxTdx.

---
 arch/x86/kvm/Makefile      |   2 +-
 arch/x86/kvm/vmx/main.c    | 729 +++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c     | 463 ++++++++---------------
 arch/x86/kvm/vmx/x86_ops.h | 124 +++++++
 4 files changed, 1000 insertions(+), 318 deletions(-)
 create mode 100644 arch/x86/kvm/vmx/main.c
 create mode 100644 arch/x86/kvm/vmx/x86_ops.h
arch/x86/kvm/Makefile +1 -1
index c7eceff49e67..d28f990bd81d 100644
@@ -27,7 +27,7 @@ kvm-$(CONFIG_X86_64) += mmu/tdp_iter.o mmu/tdp_mmu.o
 kvm-$(CONFIG_KVM_XEN)	+= xen.o
 
 kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \
-			   vmx/evmcs.o vmx/nested.o vmx/posted_intr.o
+			   vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o
 kvm-intel-$(CONFIG_X86_SGX_KVM)	+= vmx/sgx.o
 kvm-intel-$(CONFIG_INTEL_TDX_HOST)	+= vmx/tdx_error.o
 
arch/x86/kvm/vmx/main.c new file +729 -0

729 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.c +146 -317

1,017 lines not on this page · raw message

arch/x86/kvm/vmx/x86_ops.h new file +124 -0

124 lines not on this page · raw message

Signature · 1 line
2.25.1
#82 lore raw

RFC PATCH v3 47/59 KVM: TDX: Define TDCALL exit reason

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Define the TDCALL exit reason, which is carved out from the VMX exit reason namespace as the TDCALL exit from TDX guest to TDX-SEAM is really just a VM-Exit.

---
 arch/x86/include/uapi/asm/vmx.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/include/uapi/asm/vmx.h +3 -1
index 946d761adbd3..ba5908dfc7c0 100644
@@ -91,6 +91,7 @@
 #define EXIT_REASON_UMWAIT              67
 #define EXIT_REASON_TPAUSE              68
 #define EXIT_REASON_BUS_LOCK            74
+#define EXIT_REASON_TDCALL              77
 
 #define VMX_EXIT_REASONS \
 	{ EXIT_REASON_EXCEPTION_NMI,         "EXCEPTION_NMI" }, \
@@ -153,7 +154,8 @@
 	{ EXIT_REASON_XRSTORS,               "XRSTORS" }, \
 	{ EXIT_REASON_UMWAIT,                "UMWAIT" }, \
 	{ EXIT_REASON_TPAUSE,                "TPAUSE" }, \
-	{ EXIT_REASON_BUS_LOCK,              "BUS_LOCK" }
+	{ EXIT_REASON_BUS_LOCK,              "BUS_LOCK" }, \
+	{ EXIT_REASON_TDCALL,                "TDCALL" }
 
 #define VMX_EXIT_REASON_FLAGS \
 	{ VMX_EXIT_REASONS_FAILED_VMENTRY,	"FAILED_VMENTRY" }
Signature · 1 line
2.25.1
#77 lore raw

RFC PATCH v3 46/59 KVM: VMX: Move register caching logic to common code

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Move the guts of vmx_cache_reg() to vt_cache_reg() in preparation for reusing the bulk of the code for TDX, which can access guest state for debug TDs.

Use kvm_x86_ops.cache_reg() in ept_update_paging_mode_cr0() rather than trying to expose vt_cache_reg() to vmx.c, even though it means taking a retpoline. The code runs if and only if EPT is enabled but unrestricted guest. Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.

---
 arch/x86/kvm/vmx/main.c    | 44 ++++++++++++++++++++++++++++++++++---
 arch/x86/kvm/vmx/vmx.c     | 45 +-------------------------------------
 arch/x86/kvm/vmx/x86_ops.h |  1 +
 3 files changed, 43 insertions(+), 47 deletions(-)
arch/x86/kvm/vmx/main.c +41 -3
index ef89ed0457d5..a0a8cc2fd600 100644
@@ -339,9 +339,47 @@ static void vt_sync_dirty_debug_regs(struct kvm_vcpu *vcpu)
 	vmx_sync_dirty_debug_regs(vcpu);
 }
 
-static void vt_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)
-{
-	vmx_cache_reg(vcpu, reg);
+void vt_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)
+{
+	unsigned long guest_owned_bits;
+
+	kvm_register_mark_available(vcpu, reg);
+
+	switch (reg) {
+	case VCPU_REGS_RSP:
+		vcpu->arch.regs[VCPU_REGS_RSP] = vmcs_readl(GUEST_RSP);
+		break;
+	case VCPU_REGS_RIP:
+		vcpu->arch.regs[VCPU_REGS_RIP] = vmcs_readl(GUEST_RIP);
+		break;
+	case VCPU_EXREG_PDPTR:
+		if (enable_ept)
+			ept_save_pdptrs(vcpu);
+		break;
+	case VCPU_EXREG_CR0:
+		guest_owned_bits = vcpu->arch.cr0_guest_owned_bits;
+
+		vcpu->arch.cr0 &= ~guest_owned_bits;
+		vcpu->arch.cr0 |= vmcs_readl(GUEST_CR0) & guest_owned_bits;
+		break;
+	case VCPU_EXREG_CR3:
+		/*
+		 * When intercepting CR3 loads, e.g. for shadowing paging, KVM's
+		 * CR3 is loaded into hardware, not the guest's CR3.
+		 */
+		if (!(exec_controls_get(to_vmx(vcpu)) & CPU_BASED_CR3_LOAD_EXITING))
+			vcpu->arch.cr3 = vmcs_readl(GUEST_CR3);
+		break;
+	case VCPU_EXREG_CR4:
+		guest_owned_bits = vcpu->arch.cr4_guest_owned_bits;
+
+		vcpu->arch.cr4 &= ~guest_owned_bits;
+		vcpu->arch.cr4 |= vmcs_readl(GUEST_CR4) & guest_owned_bits;
+		break;
+	default:
+		KVM_BUG_ON(1, vcpu->kvm);
+		break;
+	}
 }
 
 static unsigned long vt_get_rflags(struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/vmx.c +1 -44

57 lines not on this page · raw message

arch/x86/kvm/vmx/x86_ops.h +1 -0
index f04b30f1b72d..c49d6f9f36fd 100644
@@ -9,6 +9,7 @@
 #include "x86.h"
 
 extern struct kvm_x86_ops vt_x86_ops __initdata;
+void vt_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg);
 
 __init int vmx_disabled_by_bios(void);
 int __init vmx_check_processor_compat(void);
Signature · 1 line
2.25.1
#85 lore raw

RFC PATCH v3 48/59 KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Stub in kvm_tdx, vcpu_tdx, their various accessors, and VMCS helpers. The VMCS helpers, which rely on the stubs, will be used by preparatory patches to move VMX functions for accessing VMCS state to common code.

---
 arch/x86/kvm/vmx/tdx.h | 169 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 169 insertions(+)
 create mode 100644 arch/x86/kvm/vmx/tdx.h
arch/x86/kvm/vmx/tdx.h new file +169 -0
new file mode 100644
index 000000000000..31412ed8049f
@@ -0,0 +1,169 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __KVM_X86_TDX_H
+#define __KVM_X86_TDX_H
+
+#include <linux/list.h>
+#include <linux/kvm_host.h>
+
+#include "tdx_errno.h"
+#include "tdx_arch.h"
+#include "tdx_ops.h"
+
+#ifdef CONFIG_INTEL_TDX_HOST
+
+struct tdx_td_page {
+	unsigned long va;
+	hpa_t pa;
+	bool added;
+};
+
+struct kvm_tdx {
+	struct kvm kvm;
+
+	struct tdx_td_page tdr;
+	struct tdx_td_page tdcs[TDX_NR_TDCX_PAGES];
+};
+
+struct vcpu_tdx {
+	struct kvm_vcpu	vcpu;
+
+	struct tdx_td_page tdvpr;
+	struct tdx_td_page tdvpx[TDX_NR_TDVPX_PAGES];
+};
+
+static inline bool is_td(struct kvm *kvm)
+{
+	return kvm->arch.vm_type == KVM_X86_TDX_VM;
+}
+
+static inline bool is_td_vcpu(struct kvm_vcpu *vcpu)
+{
+	return is_td(vcpu->kvm);
+}
+
+static inline bool is_debug_td(struct kvm_vcpu *vcpu)
+{
+	return !vcpu->arch.guest_state_protected;
+}
+
+static inline struct kvm_tdx *to_kvm_tdx(struct kvm *kvm)
+{
+	return container_of(kvm, struct kvm_tdx, kvm);
+}
+
+static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu)
+{
+	return container_of(vcpu, struct vcpu_tdx, vcpu);
+}
+
+static __always_inline void tdvps_vmcs_check(u32 field, u8 bits)
+{
+	BUILD_BUG_ON_MSG(__builtin_constant_p(field) && (field) & 0x1,
+			 "Read/Write to TD VMCS *_HIGH fields not supported");
+
+	BUILD_BUG_ON(bits != 16 && bits != 32 && bits != 64);
+
+	BUILD_BUG_ON_MSG(bits != 64 && __builtin_constant_p(field) &&
+			 (((field) & 0x6000) == 0x2000 ||
+			  ((field) & 0x6000) == 0x6000),
+			 "Invalid TD VMCS access for 64-bit field");
+	BUILD_BUG_ON_MSG(bits != 32 && __builtin_constant_p(field) &&
+			 ((field) & 0x6000) == 0x4000,
+			 "Invalid TD VMCS access for 32-bit field");
+	BUILD_BUG_ON_MSG(bits != 16 && __builtin_constant_p(field) &&
+			 ((field) & 0x6000) == 0x0000,
+			 "Invalid TD VMCS access for 16-bit field");
+}
+
+static __always_inline void tdvps_gpr_check(u64 field, u8 bits)
+{
+	BUILD_BUG_ON_MSG(__builtin_constant_p(field) && (field) >= NR_VCPU_REGS,
+			 "Invalid TD guest GPR index");
+}
+
+static __always_inline void tdvps_apic_check(u64 field, u8 bits) {}
+static __always_inline void tdvps_dr_check(u64 field, u8 bits) {}
+static __always_inline void tdvps_state_check(u64 field, u8 bits) {}
+static __always_inline void tdvps_msr_check(u64 field, u8 bits) {}
+static __always_inline void tdvps_management_check(u64 field, u8 bits) {}
+
+#define TDX_BUILD_TDVPS_ACCESSORS(bits, uclass, lclass)			       \
+static __always_inline u##bits td_##lclass##_read##bits(struct vcpu_tdx *tdx,  \
+							u32 field)	       \
+{									       \
+	struct tdx_ex_ret ex_ret;					       \
+	u64 err;							       \
+									       \
+	tdvps_##lclass##_check(field, bits);				       \
+	err = tdh_vp_rd(tdx->tdvpr.pa, TDVPS_##uclass(field), &ex_ret);        \
+	if (unlikely(err)) {						       \
+		pr_err("TDH_VP_RD["#uclass".0x%x] failed: 0x%llx\n",	       \
+		       field, err);					       \
+		return 0;						       \
+	}								       \
+	return (u##bits)ex_ret.regs.r8;					       \
+}									       \
+static __always_inline void td_##lclass##_write##bits(struct vcpu_tdx *tdx,    \
+						      u32 field, u##bits val)  \
+{									       \
+	struct tdx_ex_ret ex_ret;					       \
+	u64 err;							       \
+									       \
+	tdvps_##lclass##_check(field, bits);				       \
+	err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), val,	       \
+		      GENMASK_ULL(bits - 1, 0), &ex_ret);		       \
+	if (unlikely(err))						       \
+		pr_err("TDH_VP_WR["#uclass".0x%x] = 0x%llx failed: 0x%llx\n",  \
+		       field, (u64)val, err);				       \
+}									       \
+static __always_inline void td_##lclass##_setbit##bits(struct vcpu_tdx *tdx,   \
+						       u32 field, u64 bit)     \
+{									       \
+	struct tdx_ex_ret ex_ret;					       \
+	u64 err;							       \
+									       \
+	tdvps_##lclass##_check(field, bits);				       \
+	err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), bit, bit,        \
+			&ex_ret);					       \
+	if (unlikely(err))						       \
+		pr_err("TDH_VP_WR["#uclass".0x%x] |= 0x%llx failed: 0x%llx\n", \
+		       field, bit, err);				       \
+}									       \
+static __always_inline void td_##lclass##_clearbit##bits(struct vcpu_tdx *tdx, \
+							 u32 field, u64 bit)   \
+{									       \
+	struct tdx_ex_ret ex_ret;					       \
+	u64 err;							       \
+									       \
+	tdvps_##lclass##_check(field, bits);				       \
+	err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), 0, bit,	       \
+			&ex_ret);					       \
+	if (unlikely(err))						       \
+		pr_err("TDH_VP_WR["#uclass".0x%x] &= ~0x%llx failed: 0x%llx\n", \
+		       field, bit, err);				       \
+}
+
+TDX_BUILD_TDVPS_ACCESSORS(16, VMCS, vmcs);
+TDX_BUILD_TDVPS_ACCESSORS(32, VMCS, vmcs);
+TDX_BUILD_TDVPS_ACCESSORS(64, VMCS, vmcs);
+
+TDX_BUILD_TDVPS_ACCESSORS(64, APIC, apic);
+TDX_BUILD_TDVPS_ACCESSORS(64, GPR, gpr);
+TDX_BUILD_TDVPS_ACCESSORS(64, DR, dr);
+TDX_BUILD_TDVPS_ACCESSORS(64, STATE, state);
+TDX_BUILD_TDVPS_ACCESSORS(64, MSR, msr);
+TDX_BUILD_TDVPS_ACCESSORS(8, MANAGEMENT, management);
+
+#else
+struct kvm_tdx;
+struct vcpu_tdx;
+
+static inline bool is_td(struct kvm *kvm) { return false; }
+static inline bool is_td_vcpu(struct kvm_vcpu *vcpu) { return false; }
+static inline bool is_debug_td(struct kvm_vcpu *vcpu) { return false; }
+static inline struct kvm_tdx *to_kvm_tdx(struct kvm *kvm) { return NULL; }
+static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) { return NULL; }
+
+#endif /* CONFIG_INTEL_TDX_HOST */
+
+#endif /* __KVM_X86_TDX_H */
Signature · 1 line
2.25.1
#50 lore raw

RFC PATCH v3 28/59 KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Return true for kvm_vcpu_has_events() if the vCPU has a pending APICv interrupt to support TDX's usage of APICv. Unlike VMX, TDX doesn't have access to vmcs.GUEST_INTR_STATUS and so can't emulate posted interrupts, i.e. needs to generate a posted interrupt and more importantly can't manually move requested interrupts into the vIRR (which it also doesn't have access to).

---
 arch/x86/kvm/x86.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +3 -1
index 8161475082a7..c6e56f105673 100644
@@ -11914,7 +11914,9 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu)
 
 	if (kvm_arch_interrupt_allowed(vcpu) &&
 	    (kvm_cpu_has_interrupt(vcpu) ||
-	    kvm_guest_apic_has_interrupt(vcpu)))
+	     kvm_guest_apic_has_interrupt(vcpu) ||
+	     (vcpu->arch.apicv_active &&
+	      kvm_x86_ops.dy_apicv_has_pending_interrupt(vcpu))))
 		return true;
 
 	if (kvm_hv_has_stimer_pending(vcpu))
Signature · 1 line
2.25.1
#91 lore raw

RFC PATCH v3 52/59 KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX code

In reply to [email protected]

From: Sean Christopherson <[email protected]>

---
 arch/x86/kvm/vmx/common.h | 14 ++++++++++++++
 arch/x86/kvm/vmx/vmx.c    | 10 +---------
 2 files changed, 15 insertions(+), 9 deletions(-)
arch/x86/kvm/vmx/common.h +14 -0
index e45d2d222168..684cd3add46b 100644
@@ -120,6 +120,20 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa,
 	return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0);
 }
 
+static inline u32 __vmx_get_interrupt_shadow(struct kvm_vcpu *vcpu)
+{
+	u32 interruptibility;
+	int ret = 0;
+
+	interruptibility = vmread32(vcpu, GUEST_INTERRUPTIBILITY_INFO);
+	if (interruptibility & GUEST_INTR_STATE_STI)
+		ret |= KVM_X86_SHADOW_INT_STI;
+	if (interruptibility & GUEST_INTR_STATE_MOV_SS)
+		ret |= KVM_X86_SHADOW_INT_MOV_SS;
+
+	return ret;
+}
+
 static inline u32 vmx_encode_ar_bytes(struct kvm_segment *var)
 {
 	u32 ar;
arch/x86/kvm/vmx/vmx.c +1 -9
index a644b9627f9d..6f38e0d2e1b6 100644
@@ -1365,15 +1365,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
 
 u32 vmx_get_interrupt_shadow(struct kvm_vcpu *vcpu)
 {
-	u32 interruptibility = vmcs_read32(GUEST_INTERRUPTIBILITY_INFO);
-	int ret = 0;
-
-	if (interruptibility & GUEST_INTR_STATE_STI)
-		ret |= KVM_X86_SHADOW_INT_STI;
-	if (interruptibility & GUEST_INTR_STATE_MOV_SS)
-		ret |= KVM_X86_SHADOW_INT_MOV_SS;
-
-	return ret;
+	return __vmx_get_interrupt_shadow(vcpu);
 }
 
 void vmx_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask)
Signature · 1 line
2.25.1
#93 lore raw

RFC PATCH v3 53/59 KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

In reply to [email protected]

From: Chao Gao <[email protected]>

The TDX module unconditionally reset 4 host MSRs (MSR_SYSCALL_MASK, MSR_START, MSR_LSTAR, MSR_TSC_AUX) to architectural INIT state on exit from TDX VM to KVM. KVM needs to save their values before TD enter and restore them on exit to userspace.

Reuse current kvm_user_return mechanism and introduce a function to update cached values and register the user return notifier in this new function.

The later patch will use the helper function to save/restore 4 host MSRs.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/x86.c              | 25 ++++++++++++++++++++-----
 2 files changed, 21 insertions(+), 5 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index a2027bc14e41..17d6e4bcf84b 100644
@@ -1910,6 +1910,7 @@ int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low,
 int kvm_add_user_return_msr(u32 msr);
 int kvm_find_user_return_msr(u32 msr);
 int kvm_set_user_return_msr(unsigned index, u64 val, u64 mask);
+void kvm_user_return_update_cache(unsigned int index, u64 val);
 
 static inline bool kvm_is_supported_user_return_msr(u32 msr)
 {
arch/x86/kvm/x86.c +20 -5
index 4dd8ec2641a2..09da7cbedb5f 100644
@@ -418,6 +418,15 @@ static void kvm_user_return_msr_cpu_online(void)
 	}
 }
 
+static void kvm_user_return_register_notifier(struct kvm_user_return_msrs *msrs)
+{
+	if (!msrs->registered) {
+		msrs->urn.on_user_return = kvm_on_user_return;
+		user_return_notifier_register(&msrs->urn);
+		msrs->registered = true;
+	}
+}
+
 int kvm_set_user_return_msr(unsigned slot, u64 value, u64 mask)
 {
 	unsigned int cpu = smp_processor_id();
@@ -432,15 +441,21 @@ int kvm_set_user_return_msr(unsigned slot, u64 value, u64 mask)
 		return 1;
 
 	msrs->values[slot].curr = value;
-	if (!msrs->registered) {
-		msrs->urn.on_user_return = kvm_on_user_return;
-		user_return_notifier_register(&msrs->urn);
-		msrs->registered = true;
-	}
+	kvm_user_return_register_notifier(msrs);
 	return 0;
 }
 EXPORT_SYMBOL_GPL(kvm_set_user_return_msr);
 
+/* Update the cache, "curr", and register the notifier */
+void kvm_user_return_update_cache(unsigned int slot, u64 value)
+{
+	struct kvm_user_return_msrs *msrs = this_cpu_ptr(user_return_msrs);
+
+	msrs->values[slot].curr = value;
+	kvm_user_return_register_notifier(msrs);
+}
+EXPORT_SYMBOL_GPL(kvm_user_return_update_cache);
+
 static void drop_user_return_notifiers(void)
 {
 	unsigned int cpu = smp_processor_id();
Signature · 1 line
2.25.1
#89 lore raw

RFC PATCH v3 51/59 KVM: VMX: MOVE GDT and IDT accessors to common code

In reply to [email protected]

From: Sean Christopherson <[email protected]>

---
 arch/x86/kvm/vmx/main.c    |  7 +++++--
 arch/x86/kvm/vmx/vmx.c     | 12 ------------
 arch/x86/kvm/vmx/x86_ops.h |  2 --
 3 files changed, 5 insertions(+), 16 deletions(-)
arch/x86/kvm/vmx/main.c +5 -2
index a0a8cc2fd600..4d6bf1f56641 100644
@@ -3,6 +3,7 @@
 
 #include "x86_ops.h"
 #include "vmx.h"
+#include "common.h"
 #include "nested.h"
 #include "mmu.h"
 #include "pmu.h"
@@ -311,7 +312,8 @@ static int vt_set_efer(struct kvm_vcpu *vcpu, u64 efer)
 
 static void vt_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
 {
-	vmx_get_idt(vcpu, dt);
+	dt->size = vmread32(vcpu, GUEST_IDTR_LIMIT);
+	dt->address = vmreadl(vcpu, GUEST_IDTR_BASE);
 }
 
 static void vt_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
@@ -321,7 +323,8 @@ static void vt_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
 
 static void vt_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
 {
-	vmx_get_gdt(vcpu, dt);
+	dt->size = vmread32(vcpu, GUEST_GDTR_LIMIT);
+	dt->address = vmreadl(vcpu, GUEST_GDTR_BASE);
 }
 
 static void vt_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
arch/x86/kvm/vmx/vmx.c +0 -12
index 6f98d1b2a498..a644b9627f9d 100644
@@ -3227,24 +3227,12 @@ void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l)
 	*l = (ar >> 13) & 1;
 }
 
-void vmx_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
-{
-	dt->size = vmcs_read32(GUEST_IDTR_LIMIT);
-	dt->address = vmcs_readl(GUEST_IDTR_BASE);
-}
-
 void vmx_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
 {
 	vmcs_write32(GUEST_IDTR_LIMIT, dt->size);
 	vmcs_writel(GUEST_IDTR_BASE, dt->address);
 }
 
-void vmx_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
-{
-	dt->size = vmcs_read32(GUEST_GDTR_LIMIT);
-	dt->address = vmcs_readl(GUEST_GDTR_BASE);
-}
-
 void vmx_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)
 {
 	vmcs_write32(GUEST_GDTR_LIMIT, dt->size);
arch/x86/kvm/vmx/x86_ops.h +0 -2
index c49d6f9f36fd..fec22bef05b7 100644
@@ -66,9 +66,7 @@ void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level);
 void vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4);
 bool vmx_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4);
 int vmx_set_efer(struct kvm_vcpu *vcpu, u64 efer);
-void vmx_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt);
 void vmx_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt);
-void vmx_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt);
 void vmx_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt);
 void vmx_set_dr7(struct kvm_vcpu *vcpu, unsigned long val);
 void vmx_sync_dirty_debug_regs(struct kvm_vcpu *vcpu);
Signature · 1 line
2.25.1
#88 lore raw

RFC PATCH v3 50/59 KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Move the AR_BYTES helpers to common.h so that future patches can reuse them to decode/encode AR for TDX.

---
 arch/x86/kvm/vmx/common.h | 41 ++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c    | 47 ++++-----------------------------------
 2 files changed, 45 insertions(+), 43 deletions(-)
arch/x86/kvm/vmx/common.h +41 -0
index d37ef4dd9d90..e45d2d222168 100644
@@ -4,6 +4,7 @@
 
 #include <linux/kvm_host.h>
 
+#include <asm/kvm.h>
 #include <asm/traps.h>
 #include <asm/vmx.h>
 
@@ -119,4 +120,44 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa,
 	return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0);
 }
 
+static inline u32 vmx_encode_ar_bytes(struct kvm_segment *var)
+{
+	u32 ar;
+
+	if (var->unusable || !var->present)
+		ar = 1 << 16;
+	else {
+		ar = var->type & 15;
+		ar |= (var->s & 1) << 4;
+		ar |= (var->dpl & 3) << 5;
+		ar |= (var->present & 1) << 7;
+		ar |= (var->avl & 1) << 12;
+		ar |= (var->l & 1) << 13;
+		ar |= (var->db & 1) << 14;
+		ar |= (var->g & 1) << 15;
+	}
+
+	return ar;
+}
+
+static inline void vmx_decode_ar_bytes(u32 ar, struct kvm_segment *var)
+{
+	var->unusable = (ar >> 16) & 1;
+	var->type = ar & 15;
+	var->s = (ar >> 4) & 1;
+	var->dpl = (ar >> 5) & 3;
+	/*
+	 * Some userspaces do not preserve unusable property. Since usable
+	 * segment has to be present according to VMX spec we can use present
+	 * property to amend userspace bug by making unusable segment always
+	 * nonpresent. vmx_encode_ar_bytes() already marks nonpresent
+	 * segment as unusable.
+	 */
+	var->present = !var->unusable;
+	var->avl = (ar >> 12) & 1;
+	var->l = (ar >> 13) & 1;
+	var->db = (ar >> 14) & 1;
+	var->g = (ar >> 15) & 1;
+}
+
 #endif /* __KVM_X86_VMX_COMMON_H */
arch/x86/kvm/vmx/vmx.c +4 -43

89 lines not on this page · raw message

Signature · 1 line
2.25.1
#86 lore raw

RFC PATCH v3 49/59 KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add a macro framework to hide VMX vs. TDX details of VMREAD and VMWRITE so the VMX and TDX can shared common flows, e.g. accessing DTs.

Note, the TDX paths are dead code at this time. There is no great way to deal with the chicken-and-egg scenario of having things in place for TDX without first having TDX.

---
 arch/x86/kvm/vmx/common.h | 41 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
arch/x86/kvm/vmx/common.h +41 -0
index 9e5865b05d47..d37ef4dd9d90 100644
@@ -11,6 +11,47 @@
 #include "vmcs.h"
 #include "vmx.h"
 #include "x86.h"
+#include "tdx.h"
+
+#ifdef CONFIG_INTEL_TDX_HOST
+#define VT_BUILD_VMCS_HELPERS(type, bits, tdbits)			   \
+static __always_inline type vmread##bits(struct kvm_vcpu *vcpu,		   \
+					 unsigned long field)		   \
+{									   \
+	if (unlikely(is_td_vcpu(vcpu))) {				   \
+		if (KVM_BUG_ON(!is_debug_td(vcpu), vcpu->kvm))		   \
+			return 0;					   \
+		return td_vmcs_read##tdbits(to_tdx(vcpu), field);	   \
+	}								   \
+	return vmcs_read##bits(field);					   \
+}									   \
+static __always_inline void vmwrite##bits(struct kvm_vcpu *vcpu,	   \
+					  unsigned long field, type value) \
+{									   \
+	if (unlikely(is_td_vcpu(vcpu))) {				   \
+		if (KVM_BUG_ON(!is_debug_td(vcpu), vcpu->kvm))		   \
+			return;						   \
+		return td_vmcs_write##tdbits(to_tdx(vcpu), field, value);  \
+	}								   \
+	vmcs_write##bits(field, value);					   \
+}
+#else
+#define VT_BUILD_VMCS_HELPERS(type, bits, tdbits)			   \
+static __always_inline type vmread##bits(struct kvm_vcpu *vcpu,		   \
+					 unsigned long field)		   \
+{									   \
+	return vmcs_read##bits(field);					   \
+}									   \
+static __always_inline void vmwrite##bits(struct kvm_vcpu *vcpu,	   \
+					  unsigned long field, type value) \
+{									   \
+	vmcs_write##bits(field, value);					   \
+}
+#endif /* CONFIG_INTEL_TDX_HOST */
+VT_BUILD_VMCS_HELPERS(u16, 16, 16);
+VT_BUILD_VMCS_HELPERS(u32, 32, 32);
+VT_BUILD_VMCS_HELPERS(u64, 64, 64);
+VT_BUILD_VMCS_HELPERS(unsigned long, l, 64);
 
 extern unsigned long vmx_host_idt_base;
 void vmx_do_interrupt_nmi_irqoff(unsigned long entry);
Signature · 1 line
2.25.1
#41 lore raw

RFC PATCH v3 24/59 KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add hooks to cache and flush GPRs and invoke them from KVM_GET_REGS and KVM_SET_REGS respecitively. TDX will use the hooks to read/write GPRs from TDX-SEAM on-demand (for debug TDs).

---
 arch/x86/include/asm/kvm-x86-ops.h | 2 ++
 arch/x86/include/asm/kvm_host.h    | 2 ++
 arch/x86/kvm/x86.c                 | 4 ++++
 3 files changed, 8 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +2 -0
index 4aa0a1c10b84..30754f2b8a99 100644
@@ -45,6 +45,8 @@ KVM_X86_OP(get_gdt)
 KVM_X86_OP(set_gdt)
 KVM_X86_OP(sync_dirty_debug_regs)
 KVM_X86_OP(set_dr7)
+KVM_X86_OP_NULL(cache_gprs)
+KVM_X86_OP_NULL(flush_gprs)
 KVM_X86_OP(cache_reg)
 KVM_X86_OP(get_rflags)
 KVM_X86_OP(set_rflags)
arch/x86/include/asm/kvm_host.h +2 -0
index 5ed07f31459e..f12eec3f05ab 100644
@@ -1352,6 +1352,8 @@ struct kvm_x86_ops {
 	void (*set_gdt)(struct kvm_vcpu *vcpu, struct desc_ptr *dt);
 	void (*sync_dirty_debug_regs)(struct kvm_vcpu *vcpu);
 	void (*set_dr7)(struct kvm_vcpu *vcpu, unsigned long value);
+	void (*cache_gprs)(struct kvm_vcpu *vcpu);
+	void (*flush_gprs)(struct kvm_vcpu *vcpu);
 	void (*cache_reg)(struct kvm_vcpu *vcpu, enum kvm_reg reg);
 	unsigned long (*get_rflags)(struct kvm_vcpu *vcpu);
 	void (*set_rflags)(struct kvm_vcpu *vcpu, unsigned long rflags);
arch/x86/kvm/x86.c +4 -0
index 0221ef691a15..2e0bccbd3c44 100644
@@ -10293,6 +10293,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 
 static void __get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
 {
+	static_call_cond(kvm_x86_cache_gprs)(vcpu);
+
 	if (vcpu->arch.emulate_regs_need_sync_to_vcpu) {
 		/*
 		 * We are here if userspace calls get_regs() in the middle of
@@ -10367,6 +10369,8 @@ static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
 
 	vcpu->arch.exception.pending = false;
 
+	static_call_cond(kvm_x86_flush_gprs);
+
 	kvm_make_request(KVM_REQ_EVENT, vcpu);
 }
 
Signature · 1 line
2.25.1
#42 lore raw

RFC PATCH v3 25/59 KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP

In reply to [email protected]

From: Sean Christopherson <[email protected]>

---
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/x86.c              | 12 ++++++++++++
 2 files changed, 14 insertions(+)
arch/x86/include/asm/kvm_host.h +2 -0
index f12eec3f05ab..74c3c1629563 100644
@@ -1483,7 +1483,9 @@ struct kvm_x86_ops {
 	int (*leave_smm)(struct kvm_vcpu *vcpu, const char *smstate);
 	void (*enable_smi_window)(struct kvm_vcpu *vcpu);
 
+	int (*mem_enc_op_dev)(void __user *argp);
 	int (*mem_enc_op)(struct kvm *kvm, void __user *argp);
+	int (*mem_enc_op_vcpu)(struct kvm_vcpu *vcpu, void __user *argp);
 	int (*mem_enc_reg_region)(struct kvm *kvm, struct kvm_enc_region *argp);
 	int (*mem_enc_unreg_region)(struct kvm *kvm, struct kvm_enc_region *argp);
 	int (*vm_copy_enc_context_from)(struct kvm *kvm, unsigned int source_fd);
arch/x86/kvm/x86.c +12 -0
index 2e0bccbd3c44..f2091c4b928a 100644
@@ -4339,6 +4339,12 @@ long kvm_arch_dev_ioctl(struct file *filp,
 	case KVM_GET_SUPPORTED_HV_CPUID:
 		r = kvm_ioctl_get_supported_hv_cpuid(NULL, argp);
 		break;
+	case KVM_MEMORY_ENCRYPT_OP:
+		r = -EINVAL;
+		if (!kvm_x86_ops.mem_enc_op_dev)
+			goto out;
+		r = kvm_x86_ops.mem_enc_op_dev(argp);
+		break;
 	default:
 		r = -EINVAL;
 		break;
@@ -5516,6 +5522,12 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 	case KVM_SET_DEVICE_ATTR:
 		r = kvm_vcpu_ioctl_device_attr(vcpu, ioctl, argp);
 		break;
+	case KVM_MEMORY_ENCRYPT_OP:
+		r = -EINVAL;
+		if (!kvm_x86_ops.mem_enc_op_vcpu)
+			goto out;
+		r = kvm_x86_ops.mem_enc_op_vcpu(vcpu, argp);
+		break;
 	default:
 		r = -EINVAL;
 	}
Signature · 1 line
2.25.1
#59 lore raw

RFC PATCH v3 32/59 KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Explicity check for an MMIO spte in the fast page fault flow. TDX will use a not-present entry for MMIO sptes, which can be mistaken for an access-tracked spte since both have SPTE_SPECIAL_MASK set.

---
 arch/x86/kvm/mmu/mmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +1 -1
index 5305d1d9a976..01569913f1f0 100644
@@ -3186,7 +3186,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)
 			break;
 
 		sp = sptep_to_sp(sptep);
-		if (!is_last_spte(spte, sp->role.level))
+		if (!is_last_spte(spte, sp->role.level) || is_mmio_spte(spte))
 			break;
 
 		/*
Signature · 1 line
2.25.1
#61 lore raw

RFC PATCH v3 34/59 KVM: x86/mmu: Allow non-zero init value for shadow PTE

In reply to [email protected]

From: Sean Christopherson <[email protected]>

TDX will run with EPT violation #VEs enabled, which means KVM needs to set the "suppress #VE" bit in unused PTEs to avoid unintentionally reflecting not-present EPT violations into the guest.

---
 arch/x86/kvm/mmu.h      |  1 +
 arch/x86/kvm/mmu/mmu.c  | 50 +++++++++++++++++++++++++++++++++++------
 arch/x86/kvm/mmu/spte.c | 10 +++++++++
 arch/x86/kvm/mmu/spte.h |  2 ++
 4 files changed, 56 insertions(+), 7 deletions(-)
arch/x86/kvm/mmu.h +1 -0
index 583483bb6f71..79ccee8bbc38 100644
@@ -66,6 +66,7 @@ static __always_inline u64 rsvd_bits(int s, int e)
 
 void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask);
 void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only);
+void kvm_mmu_set_spte_init_value(u64 init_value);
 
 void kvm_init_mmu(struct kvm_vcpu *vcpu);
 void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr0,
arch/x86/kvm/mmu/mmu.c +43 -7

87 lines not on this page · raw message

arch/x86/kvm/mmu/spte.c +10 -0
index 0c76c45fdb68..bb45e71eb105 100644
@@ -34,6 +34,7 @@ u64 __read_mostly shadow_mmio_access_mask;
 u64 __read_mostly shadow_present_mask;
 u64 __read_mostly shadow_me_mask;
 u64 __read_mostly shadow_acc_track_mask;
+u64 __read_mostly shadow_init_value;
 
 u64 __read_mostly shadow_nonpresent_or_rsvd_mask;
 u64 __read_mostly shadow_nonpresent_or_rsvd_lower_gfn_mask;
@@ -221,6 +222,14 @@ u64 kvm_mmu_changed_pte_notifier_make_spte(u64 old_spte, kvm_pfn_t new_pfn)
 	return new_spte;
 }
 
+void kvm_mmu_set_spte_init_value(u64 init_value)
+{
+	if (WARN_ON(!IS_ENABLED(CONFIG_X86_64) && init_value))
+		init_value = 0;
+	shadow_init_value = init_value;
+}
+EXPORT_SYMBOL_GPL(kvm_mmu_set_spte_init_value);
+
 static u8 kvm_get_shadow_phys_bits(void)
 {
 	/*
@@ -365,6 +374,7 @@ void kvm_mmu_reset_all_pte_masks(void)
 	shadow_present_mask	= PT_PRESENT_MASK;
 	shadow_acc_track_mask	= 0;
 	shadow_me_mask		= sme_me_mask;
+	shadow_init_value	= 0;
 
 	shadow_host_writable_mask = DEFAULT_SPTE_HOST_WRITEABLE;
 	shadow_mmu_writable_mask  = DEFAULT_SPTE_MMU_WRITEABLE;
arch/x86/kvm/mmu/spte.h +2 -0
index 56b6dd750fb1..b53b301301dc 100644
@@ -146,6 +146,8 @@ extern u64 __read_mostly shadow_mmio_access_mask;
 extern u64 __read_mostly shadow_present_mask;
 extern u64 __read_mostly shadow_me_mask;
 
+extern u64 __read_mostly shadow_init_value;
+
 /*
  * SPTEs in MMUs without A/D bits are marked with SPTE_TDP_AD_DISABLED_MASK;
  * shadow_acc_track_mask is the set of bits to be cleared in non-accessed
Signature · 1 line
2.25.1
#62 lore raw

RFC PATCH v3 35/59 KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Return the old SPTE when clearing a SPTE and push the "old SPTE present" check to the caller. Private shadow page support will use the old SPTE in rmap_remove() to determine whether or not there is a linked private shadow page.

---
 arch/x86/kvm/mmu/mmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +1 -1
index 510991c2e94e..f1bd7d952bfe 100644
@@ -616,7 +616,7 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte)
  * state bits, it is used to clear the last level sptep.
  * Returns the old PTE.
  */
-static int mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep)
+static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep)
 {
 	kvm_pfn_t pfn;
 	u64 old_spte = *sptep;
Signature · 1 line
2.25.1
#55 lore raw

RFC PATCH v3 30/59 KVM: x86: Add guest_supported_xss placholder

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add a per-vcpu placeholder for the support XSS of the guest so that the TDX configuration code doesn't need to hack in manual computation of the supported XSS. KVM XSS enabling is currently being upstreamed, i.e. guest_supported_xss will no longer be a placeholder by the time TDX is ready for upstreaming (hopefully).

---
 arch/x86/include/asm/kvm_host.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/kvm_host.h +1 -0
index a2bc4d9caaef..2dab0899d82c 100644
@@ -701,6 +701,7 @@ struct kvm_vcpu_arch {
 
 	u64 xcr0;
 	u64 guest_supported_xcr0;
+	u64 guest_supported_xss;
 
 	struct kvm_pio_request pio;
 	void *pio_data;
Signature · 1 line
2.25.1
#112 lore raw

RFC PATCH v3 57/59 KVM, x86/mmu: Support TDX private mapping for TDP MMU

In reply to [email protected]

From: Kai Huang <[email protected]>

Frame in TDX private mapping support to support running TD with TDP MMU.

Similar to legacy MMU, use private mapping related kvm_x86_ops hooks in __handle_changed_spte()/handle_removed_tdp_mmu_page() to support creating/removing private mapping. And support temporarily blocking private mapping upon receiving MMU notifier and later on unblocking it upon EPT violation, rather than completely removing the private page, because currently page migration for private page is not supported, therefore the page cannot be really removed from TD upon MMU notifier.

Similar to legacy MMU, zap aliasing mapping (truly remove the page) upon EPT violation. And only leaf page is zapped, but not intermediate page tables.

Support 4K page only at this stage. 2M page support can be done in future patches.

A key change to TDP MMU to support TD guest is, the read_lock() is changed to write_lock() for page fault on private GPA in direct_page_fault(), while for fault on shared GPA, read_lock() is still used. This is because for TD guest, at given time for given GFN, only one type of mapping (either private, or shared) can be valid, but not both, otherwise it may cause machine check and data lose. As a result, aliasing mapping is zapped in TDP MMU fault handler. In this case, running multiple fault threads with both private and shared address concurrently may end up with having both private and shared mapping for given GPN. Consider below case: vcpu 0 is accessing using private GPA, and vcpu 1 is accessing the shared GPA (i.e. right after MAP_GPA).

	vcpu 0 				vcpu 1

(fault with private GPA) (fault with shared GPA)
zap shared mapping
zap private mapping setup shared mapping
setup private mapping

This may end up with having both private and shared mappings. Perhaps it is arguable whether above case is valid, but for security, just don't allow running multiple fault threads concurrently.

---
 arch/x86/kvm/mmu/mmu.c      |  63 ++++-
 arch/x86/kvm/mmu/spte.h     |  20 +-
 arch/x86/kvm/mmu/tdp_iter.h |   2 +-
 arch/x86/kvm/mmu/tdp_mmu.c  | 544 ++++++++++++++++++++++++++++++++----
 arch/x86/kvm/mmu/tdp_mmu.h  |  15 +-
 5 files changed, 567 insertions(+), 77 deletions(-)
arch/x86/kvm/mmu/mmu.c +57 -6
index f8109002da0d..1e13949ac62a 100644
@@ -3692,7 +3692,11 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu)
 		goto out_unlock;
 
 	if (is_tdp_mmu_enabled(vcpu->kvm)) {
-		root = kvm_tdp_mmu_get_vcpu_root_hpa(vcpu);
+		if (gfn_shared && !VALID_PAGE(mmu->private_root_hpa)) {
+			root = kvm_tdp_mmu_get_vcpu_root_hpa(vcpu, true);
+			mmu->private_root_hpa = root;
+		}
+		root = kvm_tdp_mmu_get_vcpu_root_hpa(vcpu, false);
 		mmu->root_hpa = root;
 	} else if (shadow_root_level >= PT64_ROOT_4LEVEL) {
 		if (gfn_shared && !VALID_PAGE(vcpu->arch.mmu->private_root_hpa)) {
@@ -4348,7 +4352,33 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault
 
 	r = RET_PF_RETRY;
 
-	if (is_tdp_mmu_fault)
+	/*
+	 * Unfortunately, when running TD, TDP MMU cannot always run multiple
+	 * fault threads concurrently.  TDX has two page tables supporting
+	 * private and shared mapping simultaneously, but at given time, only
+	 * one mapping can be valid for given GFN, otherwise it may cause
+	 * machine check and data lose.  Therefore, TDP MMU fault handler zaps
+	 * aliasing mapping.
+	 *
+	 * Running fault threads for both private and shared GPA concurrently
+	 * can potentially end up with having both private and shared mapping
+	 * for one GPA.  For instance, vcpu 0 is accessing using private GPA,
+	 * and vcpu 1 is accessing using shared GPA (i.e. right after MAP_GPA):
+	 *
+	 *		vcpu 0				vcpu 1
+	 *	(fault with private GPA)	(fault with shared GPA)
+	 *
+	 *	zap shared mapping
+	 *					zap priavte mapping
+	 *					setup shared mapping
+	 *	setup private mapping
+	 *
+	 * This can be prevented by only allowing one type of fault (private
+	 * or shared) to run concurrently.  Choose to let private fault to run
+	 * concurrently, because for TD, most pages should be private.
+	 */
+	if (is_tdp_mmu_fault && kvm_is_private_gfn(vcpu->kvm,
+				fault->addr >> PAGE_SHIFT))
 		read_lock(&vcpu->kvm->mmu_lock);
 	else
 		write_lock(&vcpu->kvm->mmu_lock);
@@ -4365,7 +4395,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault
 		r = __direct_map(vcpu, fault);
 
 out_unlock:
-	if (is_tdp_mmu_fault)
+	if (is_tdp_mmu_fault && kvm_is_private_gfn(vcpu->kvm,
+				fault->addr >> PAGE_SHIFT))
 		read_unlock(&vcpu->kvm->mmu_lock);
 	else
 		write_unlock(&vcpu->kvm->mmu_lock);
@@ -6057,6 +6088,10 @@ static void kvm_mmu_zap_all_fast(struct kvm *kvm)
 
 	write_unlock(&kvm->mmu_lock);
 
+	/*
+	 * For now private root is never invalidate during VM is running,
+	 * so this can only happen for shared roots.
+	 */
 	if (is_tdp_mmu_enabled(kvm)) {
 		read_lock(&kvm->mmu_lock);
 		kvm_tdp_mmu_zap_invalidated_roots(kvm);
@@ -6162,7 +6197,8 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
 	if (is_tdp_mmu_enabled(kvm)) {
 		for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++)
 			flush = kvm_tdp_mmu_zap_gfn_range(kvm, i, gfn_start,
-							  gfn_end, flush);
+							  gfn_end, flush,
+							  false);
 	}
 
 	if (flush)
@@ -6195,6 +6231,11 @@ void kvm_mmu_slot_remove_write_access(struct kvm *kvm,
 		write_unlock(&kvm->mmu_lock);
 	}
 
+	/*
+	 * For now this can only happen for non-TD VM, because TD private
+	 * mapping doesn't support write protection.  kvm_tdp_mmu_wrprot_slot()
+	 * will give a WARN() if it hits for TD.
+	 */
 	if (is_tdp_mmu_enabled(kvm)) {
 		read_lock(&kvm->mmu_lock);
 		flush |= kvm_tdp_mmu_wrprot_slot(kvm, memslot, start_level);
@@ -6277,6 +6318,11 @@ void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm,
 		write_unlock(&kvm->mmu_lock);
 	}
 
+	/*
+	 * This should only be reachable in case of log-dirty, wihch TD private
+	 * mapping doesn't support so far.  kvm_tdp_mmu_zap_collapsible_sptes()
+	 * internally gives a WARN() when it hits.
+	 */
 	if (is_tdp_mmu_enabled(kvm)) {
 		read_lock(&kvm->mmu_lock);
 		flush = kvm_tdp_mmu_zap_collapsible_sptes(kvm, slot, flush);
@@ -6316,6 +6362,7 @@ void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm,
 		write_unlock(&kvm->mmu_lock);
 	}
 
+	/* See comments in kvm_mmu_slot_remove_write_access() */
 	if (is_tdp_mmu_enabled(kvm)) {
 		read_lock(&kvm->mmu_lock);
 		flush |= kvm_tdp_mmu_clear_dirty_slot(kvm, memslot);
@@ -6350,8 +6397,12 @@ static void __kvm_mmu_zap_all(struct kvm *kvm, struct list_head *mmu_pages)
 	}
 	kvm_mmu_commit_zap_page(kvm, &invalid_list);
 
-	if (is_tdp_mmu_enabled(kvm))
-		kvm_tdp_mmu_zap_all(kvm);
+	if (is_tdp_mmu_enabled(kvm)) {
+		bool zap_private =
+			(mmu_pages == &kvm->arch.private_mmu_pages) ?
+			true : false;
+		kvm_tdp_mmu_zap_all(kvm, zap_private);
+	}
 
 	write_unlock(&kvm->mmu_lock);
 }
arch/x86/kvm/mmu/spte.h +17 -3
index 43f0d2a773f7..3108d46e0dd3 100644
@@ -173,7 +173,9 @@ extern u64 __read_mostly shadow_nonpresent_or_rsvd_mask;
  * If a thread running without exclusive control of the MMU lock must perform a
  * multi-part operation on an SPTE, it can set the SPTE to REMOVED_SPTE as a
  * non-present intermediate value. Other threads which encounter this value
- * should not modify the SPTE.
+ * should not modify the SPTE.  When TDX is enabled, shadow_init_value, which
+ * is "suppress #VE" bit set, is also set to removed SPTE, because TDX module
+ * always enables "EPT violation #VE".
  *
  * Use a semi-arbitrary value that doesn't set RWX bits, i.e. is not-present on
  * bot AMD and Intel CPUs, and doesn't set PFN bits, i.e. doesn't create a L1TF
@@ -183,12 +185,24 @@ extern u64 __read_mostly shadow_nonpresent_or_rsvd_mask;
  */
 #define REMOVED_SPTE	0x5a0ULL
 
-/* Removed SPTEs must not be misconstrued as shadow present PTEs. */
+/*
+ * Removed SPTEs must not be misconstrued as shadow present PTEs, and
+ * temporarily blocked private PTEs.
+ */
 static_assert(!(REMOVED_SPTE & SPTE_MMU_PRESENT_MASK));
+static_assert(!(REMOVED_SPTE & SPTE_PRIVATE_ZAPPED));
+
+/*
+ * See above comment around REMOVED_SPTE.  SHADOW_REMOVED_SPTE is the actual
+ * intermediate value set to the removed SPET.  When TDX is enabled, it sets
+ * the "suppress #VE" bit, otherwise it's REMOVED_SPTE.
+ */
+extern u64 __read_mostly shadow_init_value;
+#define SHADOW_REMOVED_SPTE	(shadow_init_value | REMOVED_SPTE)
 
 static inline bool is_removed_spte(u64 spte)
 {
-	return spte == REMOVED_SPTE;
+	return spte == SHADOW_REMOVED_SPTE;
 }
 
 /*
arch/x86/kvm/mmu/tdp_iter.h +1 -1
index b1748b988d3a..52089728d583 100644
@@ -28,7 +28,7 @@ struct tdp_iter {
 	tdp_ptep_t pt_path[PT64_ROOT_MAX_LEVEL];
 	/* A pointer to the current SPTE */
 	tdp_ptep_t sptep;
-	/* The lowest GFN mapped by the current SPTE */
+	/* The lowest GFN (stolen bits included) mapped by the current SPTE */
 	gfn_t gfn;
 	/* The level of the root page given to the iterator */
 	int root_level;
arch/x86/kvm/mmu/tdp_mmu.c +483 -61

972 lines not on this page · raw message

arch/x86/kvm/mmu/tdp_mmu.h +9 -6
index 476b133544dd..0a975d30909e 100644
@@ -5,7 +5,7 @@
 
 #include <linux/kvm_host.h>
 
-hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu);
+hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu, bool private);
 
 __must_check static inline bool kvm_tdp_mmu_get_root(struct kvm *kvm,
 						     struct kvm_mmu_page *root)
@@ -20,11 +20,14 @@ void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root,
 			  bool shared);
 
 bool __kvm_tdp_mmu_zap_gfn_range(struct kvm *kvm, int as_id, gfn_t start,
-				 gfn_t end, bool can_yield, bool flush);
+				 gfn_t end, bool can_yield, bool flush,
+				 bool zap_private);
 static inline bool kvm_tdp_mmu_zap_gfn_range(struct kvm *kvm, int as_id,
-					     gfn_t start, gfn_t end, bool flush)
+					     gfn_t start, gfn_t end, bool flush,
+					     bool zap_private)
 {
-	return __kvm_tdp_mmu_zap_gfn_range(kvm, as_id, start, end, true, flush);
+	return __kvm_tdp_mmu_zap_gfn_range(kvm, as_id, start, end, true, flush,
+			zap_private);
 }
 static inline bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp)
 {
@@ -41,10 +44,10 @@ static inline bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp)
 	 */
 	lockdep_assert_held_write(&kvm->mmu_lock);
 	return __kvm_tdp_mmu_zap_gfn_range(kvm, kvm_mmu_page_as_id(sp),
-					   sp->gfn, end, false, false);
+					   sp->gfn, end, false, false, false);
 }
 
-void kvm_tdp_mmu_zap_all(struct kvm *kvm);
+void kvm_tdp_mmu_zap_all(struct kvm *kvm, bool zap_private);
 void kvm_tdp_mmu_invalidate_all_roots(struct kvm *kvm);
 void kvm_tdp_mmu_zap_invalidated_roots(struct kvm *kvm);
 
Signature · 1 line
2.25.1
#111 lore raw

RFC PATCH v3 56/59 KVM: TDX: Protect private mapping related SEAMCALLs with spinlock

In reply to [email protected]

From: Kai Huang <[email protected]>

Unlike legacy MMU, TDP MMU runs page fault handler concurrently. However, TDX private mapping related SEAMCALLs cannot run concurrently even they operate on different pages, because they need to acquire exclusive access to secure EPT table. Protect those SEAMCALLs with spinlock, so they won't run concurrently even under TDP MMU.

---
 arch/x86/kvm/vmx/tdx.c | 111 +++++++++++++++++++++++++++++++++++++----
 arch/x86/kvm/vmx/tdx.h |   7 +++
 2 files changed, 108 insertions(+), 10 deletions(-)
arch/x86/kvm/vmx/tdx.c +101 -10
index 64b2841064c4..53fc01f3bab1 100644
@@ -519,6 +519,8 @@ int tdx_vm_init(struct kvm *kvm)
 		tdx_add_td_page(&kvm_tdx->tdcs[i]);
 	}
 
+	spin_lock_init(&kvm_tdx->seamcall_lock);
+
 	/*
 	 * Note, TDH_MNG_INIT cannot be invoked here.  TDH_MNG_INIT requires a dedicated
 	 * ioctl() to define the configure CPUID values for the TD.
@@ -1187,8 +1189,8 @@ static void tdx_measure_page(struct kvm_tdx *kvm_tdx, hpa_t gpa)
 	}
 }
 
-static void tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn,
-				      enum pg_level level, kvm_pfn_t pfn)
+static void __tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn,
+					enum pg_level level, kvm_pfn_t pfn)
 {
 	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
 	hpa_t hpa = pfn << PAGE_SHIFT;
@@ -1215,6 +1217,15 @@ static void tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn,
 		return;
 	}
 
+	/*
+	 * In case of TDP MMU, fault handler can run concurrently.  Note
+	 * 'source_pa' is a TD scope variable, meaning if there are multiple
+	 * threads reaching here with all needing to access 'source_pa', it
+	 * will break.  However fortunately this won't happen, because below
+	 * TDH_MEM_PAGE_ADD code path is only used when VM is being created
+	 * before it is running, using KVM_TDX_INIT_MEM_REGION ioctl (which
+	 * always uses vcpu 0's page table and protected by vcpu->mutex).
+	 */
 	WARN_ON(kvm_tdx->source_pa == INVALID_PAGE);
 	source_pa = kvm_tdx->source_pa & ~KVM_TDX_MEASURE_MEMORY_REGION;
 
@@ -1227,8 +1238,25 @@ static void tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn,
 	kvm_tdx->source_pa = INVALID_PAGE;
 }
 
-static void tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_level level,
-				       kvm_pfn_t pfn)
+static void tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn,
+				      enum pg_level level, kvm_pfn_t pfn)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+
+	/*
+	 * Only TDP MMU needs to use spinlock, however for simplicity,
+	 * just always use spinlock for seamcall, regardless whether
+	 * legacy MMU or TDP MMU is being used.  For legacy MMU it
+	 * should not have noticeable performance impact since taking
+	 * spinlock w/o needing to wait should be fast.
+	 */
+	spin_lock(&kvm_tdx->seamcall_lock);
+	__tdx_sept_set_private_spte(kvm, gfn, level, pfn);
+	spin_unlock(&kvm_tdx->seamcall_lock);
+}
+
+static void __tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_level level,
+					kvm_pfn_t pfn)
 {
 	int tdx_level = pg_level_to_tdx_sept_level(level);
 	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
@@ -1262,8 +1290,19 @@ static void tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_level
 	put_page(pfn_to_page(pfn));
 }
 
-static int tdx_sept_link_private_sp(struct kvm *kvm, gfn_t gfn,
-				    enum pg_level level, void *sept_page)
+static void tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_level level,
+				       kvm_pfn_t pfn)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+
+	/* See comment in tdx_sept_set_private_spte() */
+	spin_lock(&kvm_tdx->seamcall_lock);
+	__tdx_sept_drop_private_spte(kvm, gfn, level, pfn);
+	spin_unlock(&kvm_tdx->seamcall_lock);
+}
+
+static int __tdx_sept_link_private_sp(struct kvm *kvm, gfn_t gfn,
+				      enum pg_level level, void *sept_page)
 {
 	int tdx_level = pg_level_to_tdx_sept_level(level);
 	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
@@ -1281,7 +1320,22 @@ static int tdx_sept_link_private_sp(struct kvm *kvm, gfn_t gfn,
 	return 0;
 }
 
-static void tdx_sept_zap_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_level level)
+static int tdx_sept_link_private_sp(struct kvm *kvm, gfn_t gfn,
+				    enum pg_level level, void *sept_page)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	int ret;
+
+	/* See comment in tdx_sept_set_private_spte() */
+	spin_lock(&kvm_tdx->seamcall_lock);
+	ret = __tdx_sept_link_private_sp(kvm, gfn, level, sept_page);
+	spin_unlock(&kvm_tdx->seamcall_lock);
+
+	return ret;
+}
+
+static void __tdx_sept_zap_private_spte(struct kvm *kvm, gfn_t gfn,
+					enum pg_level level)
 {
 	int tdx_level = pg_level_to_tdx_sept_level(level);
 	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
@@ -1294,7 +1348,19 @@ static void tdx_sept_zap_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_level
 		pr_tdx_error(TDH_MEM_RANGE_BLOCK, err, &ex_ret);
 }
 
-static void tdx_sept_unzap_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_level level)
+static void tdx_sept_zap_private_spte(struct kvm *kvm, gfn_t gfn,
+				      enum pg_level level)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+
+	/* See comment in tdx_sept_set_private_spte() */
+	spin_lock(&kvm_tdx->seamcall_lock);
+	__tdx_sept_zap_private_spte(kvm, gfn, level);
+	spin_unlock(&kvm_tdx->seamcall_lock);
+}
+
+static void __tdx_sept_unzap_private_spte(struct kvm *kvm, gfn_t gfn,
+					  enum pg_level level)
 {
 	int tdx_level = pg_level_to_tdx_sept_level(level);
 	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
@@ -1307,8 +1373,19 @@ static void tdx_sept_unzap_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_leve
 		pr_tdx_error(TDH_MEM_RANGE_UNBLOCK, err, &ex_ret);
 }
 
-static int tdx_sept_free_private_sp(struct kvm *kvm, gfn_t gfn, enum pg_level level,
-				    void *sept_page)
+static void tdx_sept_unzap_private_spte(struct kvm *kvm, gfn_t gfn,
+					enum pg_level level)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+
+	/* See comment in tdx_sept_set_private_spte() */
+	spin_lock(&kvm_tdx->seamcall_lock);
+	__tdx_sept_unzap_private_spte(kvm, gfn, level);
+	spin_unlock(&kvm_tdx->seamcall_lock);
+}
+
+static int __tdx_sept_free_private_sp(struct kvm *kvm, gfn_t gfn, enum pg_level level,
+				      void *sept_page)
 {
 	/*
 	 * free_private_sp() is (obviously) called when a shadow page is being
@@ -1320,6 +1397,20 @@ static int tdx_sept_free_private_sp(struct kvm *kvm, gfn_t gfn, enum pg_level le
 	return tdx_reclaim_page((unsigned long)sept_page, __pa(sept_page));
 }
 
+static int tdx_sept_free_private_sp(struct kvm *kvm, gfn_t gfn, enum pg_level level,
+				    void *sept_page)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	int ret;
+
+	/* See comment in tdx_sept_set_private_spte() */
+	spin_lock(&kvm_tdx->seamcall_lock);
+	ret = __tdx_sept_free_private_sp(kvm, gfn, level, sept_page);
+	spin_unlock(&kvm_tdx->seamcall_lock);
+
+	return ret;
+}
+
 static int tdx_sept_tlb_remote_flush(struct kvm *kvm)
 {
 	struct kvm_tdx *kvm_tdx;
arch/x86/kvm/vmx/tdx.h +7 -0
index 39853a260e06..3b1a1e2878c2 100644
@@ -37,6 +37,13 @@ struct kvm_tdx {
 	hpa_t source_pa;
 
 	u64 tsc_offset;
+
+	/*
+	 * Lock to prevent seamcalls from running concurrently
+	 * when TDP MMU is enabled, because TDP fault handler
+	 * runs concurrently.
+	 */
+	spinlock_t seamcall_lock;
 };
 
 union tdx_exit_reason {
Signature · 1 line
2.25.1
#110 lore raw

RFC PATCH v3 55/59 KVM: TDX: Add "basic" support for building and running Trust Domains

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add what is effectively a TDX-specific ioctl for initializing the guest Trust Domain. Implement the functionality as a subcommand of KVM_MEMORY_ENCRYPT_OP, analogous to how the ioctl is used by SVM to manage SEV guests.

For easy compatibility with future versions of TDX-SEAM, add a KVM-defined struct, tdx_capabilities, to track requirements/capabilities for the overall system, and define a global instance to serve as the canonical reference.

---
 arch/x86/events/intel/ds.c            |   1 +
 arch/x86/include/uapi/asm/kvm.h       |  56 ++++
 arch/x86/include/uapi/asm/vmx.h       |   3 +-
 arch/x86/kvm/Makefile                 |   5 +-
 arch/x86/kvm/mmu.h                    |   2 +-
 arch/x86/kvm/mmu/mmu.c                |   1 +
 arch/x86/kvm/mmu/spte.c               |   5 +-
 arch/x86/kvm/vmx/common.h             |   1 +
 arch/x86/kvm/vmx/main.c               | 403 +++++++++++++++++++++++++-
 arch/x86/kvm/vmx/posted_intr.c        |   6 +
 arch/x86/kvm/vmx/tdx.h                | 114 ++++++++
 arch/x86/kvm/vmx/tdx_arch.h           |  24 +-
 arch/x86/kvm/vmx/tdx_ops.h            |  14 +
 arch/x86/kvm/vmx/tdx_stubs.c          |  50 ++++
 arch/x86/kvm/vmx/vmenter.S            | 146 ++++++++++
 arch/x86/kvm/vmx/vmx.c                |  39 ---
 arch/x86/kvm/vmx/x86_ops.h            |  80 +++++
 arch/x86/kvm/x86.c                    |   8 +-
 tools/arch/x86/include/uapi/asm/kvm.h |  51 ++++
 19 files changed, 946 insertions(+), 63 deletions(-)
 create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c
arch/x86/events/intel/ds.c +1 -0
index 2e215369df4a..b6c556225fd2 100644
@@ -2248,3 +2248,4 @@ void perf_restore_debug_store(void)
 
 	wrmsrl(MSR_IA32_DS_AREA, (unsigned long)ds);
 }
+EXPORT_SYMBOL_GPL(perf_restore_debug_store);
arch/x86/include/uapi/asm/kvm.h +56 -0

60 lines not on this page · raw message

arch/x86/include/uapi/asm/vmx.h +2 -1
index ba5908dfc7c0..79843a0143d2 100644
@@ -32,8 +32,9 @@
 #define EXIT_REASON_EXCEPTION_NMI       0
 #define EXIT_REASON_EXTERNAL_INTERRUPT  1
 #define EXIT_REASON_TRIPLE_FAULT        2
-#define EXIT_REASON_INIT_SIGNAL			3
+#define EXIT_REASON_INIT_SIGNAL         3
 #define EXIT_REASON_SIPI_SIGNAL         4
+#define EXIT_REASON_OTHER_SMI           6
 
 #define EXIT_REASON_INTERRUPT_WINDOW    7
 #define EXIT_REASON_NMI_WINDOW          8
arch/x86/kvm/Makefile +4 -1
index d28f990bd81d..51b2d5fdaeed 100644
@@ -29,7 +29,10 @@ kvm-$(CONFIG_KVM_XEN)	+= xen.o
 kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \
 			   vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o
 kvm-intel-$(CONFIG_X86_SGX_KVM)	+= vmx/sgx.o
-kvm-intel-$(CONFIG_INTEL_TDX_HOST)	+= vmx/tdx_error.o
+kvm-intel-$(CONFIG_INTEL_TDX_HOST)	+= vmx/tdx_error.o vmx/tdx.o
+ifneq ($(CONFIG_INTEL_TDX_HOST),y)
+kvm-intel-y				+= vmx/tdx_stubs.o
+endif
 
 kvm-amd-y		+= svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o
 
arch/x86/kvm/mmu.h +1 -1
index 2c4b8fde66d9..0bcf583216a1 100644
@@ -65,7 +65,7 @@ static __always_inline u64 rsvd_bits(int s, int e)
 }
 
 void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask);
-void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only);
+void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only, u64 init_value);
 void kvm_mmu_set_spte_init_value(u64 init_value);
 
 void kvm_init_mmu(struct kvm_vcpu *vcpu);
arch/x86/kvm/mmu/mmu.c +1 -0
index 7d3830508a44..f8109002da0d 100644
@@ -5461,6 +5461,7 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu)
 out:
 	return r;
 }
+EXPORT_SYMBOL_GPL(kvm_mmu_load);
 
 static void __kvm_mmu_unload(struct kvm_vcpu *vcpu, u32 roots_to_free)
 {
arch/x86/kvm/mmu/spte.c +3 -2
index bb45e71eb105..75784ac9d91a 100644
@@ -312,14 +312,15 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask)
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_set_mmio_spte_mask);
 
-void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only)
+void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only, u64 init_value)
 {
 	shadow_user_mask	= VMX_EPT_READABLE_MASK;
 	shadow_accessed_mask	= has_ad_bits ? VMX_EPT_ACCESS_BIT : 0ull;
 	shadow_dirty_mask	= has_ad_bits ? VMX_EPT_DIRTY_BIT : 0ull;
 	shadow_nx_mask		= 0ull;
 	shadow_x_mask		= VMX_EPT_EXECUTABLE_MASK;
-	shadow_present_mask	= has_exec_only ? 0ull : VMX_EPT_READABLE_MASK;
+	shadow_present_mask	=
+		(has_exec_only ? 0ull : VMX_EPT_READABLE_MASK) | init_value;
 	shadow_acc_track_mask	= VMX_EPT_RWX_MASK;
 	shadow_me_mask		= 0ull;
 
arch/x86/kvm/vmx/common.h +1 -0
index 684cd3add46b..18b939e363e2 100644
@@ -9,6 +9,7 @@
 #include <asm/vmx.h>
 
 #include "mmu.h"
+#include "tdx.h"
 #include "vmcs.h"
 #include "vmx.h"
 #include "x86.h"
arch/x86/kvm/vmx/main.c +390 -13

919 lines not on this page · raw message

arch/x86/kvm/vmx/posted_intr.c +6 -0
index 5f81ef092bd4..d71808358feb 100644
@@ -6,6 +6,7 @@
 
 #include "lapic.h"
 #include "posted_intr.h"
+#include "tdx.h"
 #include "trace.h"
 #include "vmx.h"
 
@@ -18,6 +19,11 @@ static DEFINE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock);
 
 static inline struct pi_desc *vcpu_to_pi_desc(struct kvm_vcpu *vcpu)
 {
+#ifdef CONFIG_INTEL_TDX_HOST
+	if (is_td_vcpu(vcpu))
+		return &(to_tdx(vcpu)->pi_desc);
+#endif
+
 	return &(to_vmx(vcpu)->pi_desc);
 }
 
arch/x86/kvm/vmx/tdx.h +114 -0

159 lines not on this page · raw message

arch/x86/kvm/vmx/tdx_arch.h +22 -2
index f57f9bfb7007..7d1483a23714 100644
@@ -54,11 +54,21 @@
 #define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT	0x10004
 
 /* TDX control structure (TDR/TDCS/TDVPS) field access codes */
+#define TDX_NON_ARCH			BIT_ULL(63)
 #define TDX_CLASS_SHIFT			56
 #define TDX_FIELD_MASK			GENMASK_ULL(31, 0)
 
-#define BUILD_TDX_FIELD(class, field)	\
-	(((u64)(class) << TDX_CLASS_SHIFT) | ((u64)(field) & TDX_FIELD_MASK))
+#define __BUILD_TDX_FIELD(non_arch, class, field)	\
+	(((non_arch) ? TDX_NON_ARCH : 0) |		\
+	 ((u64)(class) << TDX_CLASS_SHIFT) |		\
+	 ((u64)(field) & TDX_FIELD_MASK))
+
+#define BUILD_TDX_FIELD(class, field)			\
+	__BUILD_TDX_FIELD(false, (class), (field))
+
+#define BUILD_TDX_FIELD_NON_ARCH(class, field)		\
+	__BUILD_TDX_FIELD(true, (class), (field))
+
 
 /* @field is the VMCS field encoding */
 #define TDVPS_VMCS(field)		BUILD_TDX_FIELD(0, (field))
@@ -83,10 +93,20 @@ enum tdx_guest_other_state {
 	TD_VCPU_IWK_INTKEY0 = 68,
 	TD_VCPU_IWK_INTKEY1,
 	TD_VCPU_IWK_FLAGS = 70,
+	TD_VCPU_STATE_DETAILS_NON_ARCH = 0x100,
+};
+
+union tdx_vcpu_state_details {
+	struct {
+		u64 vmxip	: 1;
+		u64 reserved	: 63;
+	};
+	u64 full;
 };
 
 /* @field is any of enum tdx_guest_other_state */
 #define TDVPS_STATE(field)		BUILD_TDX_FIELD(17, (field))
+#define TDVPS_STATE_NON_ARCH(field)	BUILD_TDX_FIELD_NON_ARCH(17, field)
 
 /* @msr is the MSR index */
 #define TDVPS_MSR(msr)			BUILD_TDX_FIELD(19, (msr))
arch/x86/kvm/vmx/tdx_ops.h +14 -0
index 87ed67fd2715..f40c46eaff4c 100644
@@ -8,36 +8,48 @@
 
 #include <asm/asm.h>
 #include <asm/kvm_host.h>
+#include <asm/cacheflush.h>
 
 #include "seamcall.h"
+#include "tdx_arch.h"
 
 #ifdef CONFIG_INTEL_TDX_HOST
 
+static inline void tdx_clflush_page(hpa_t addr)
+{
+	clflush_cache_range(__va(addr), PAGE_SIZE);
+}
+
 static inline u64 tdh_mng_addcx(hpa_t tdr, hpa_t addr)
 {
+	tdx_clflush_page(addr);
 	return seamcall(TDH_MNG_ADDCX, addr, tdr, 0, 0, 0, NULL);
 }
 
 static inline u64 tdh_mem_page_add(hpa_t tdr, gpa_t gpa, hpa_t hpa, hpa_t source,
 			    struct tdx_ex_ret *ex)
 {
+	tdx_clflush_page(hpa);
 	return seamcall(TDH_MEM_PAGE_ADD, gpa, tdr, hpa, source, 0, ex);
 }
 
 static inline u64 tdh_mem_sept_add(hpa_t tdr, gpa_t gpa, int level, hpa_t page,
 			    struct tdx_ex_ret *ex)
 {
+	tdx_clflush_page(page);
 	return seamcall(TDH_MEM_SEPT_ADD, gpa | level, tdr, page, 0, 0, ex);
 }
 
 static inline u64 tdh_vp_addcx(hpa_t tdvpr, hpa_t addr)
 {
+	tdx_clflush_page(addr);
 	return seamcall(TDH_VP_ADDCX, addr, tdvpr, 0, 0, 0, NULL);
 }
 
 static inline u64 tdh_mem_page_aug(hpa_t tdr, gpa_t gpa, hpa_t hpa,
 			    struct tdx_ex_ret *ex)
 {
+	tdx_clflush_page(hpa);
 	return seamcall(TDH_MEM_PAGE_AUG, gpa, tdr, hpa, 0, 0, ex);
 }
 
@@ -54,11 +66,13 @@ static inline u64 tdh_mng_key_config(hpa_t tdr)
 
 static inline u64 tdh_mng_create(hpa_t tdr, int hkid)
 {
+	tdx_clflush_page(tdr);
 	return seamcall(TDH_MNG_CREATE, tdr, hkid, 0, 0, 0, NULL);
 }
 
 static inline u64 tdh_vp_create(hpa_t tdr, hpa_t tdvpr)
 {
+	tdx_clflush_page(tdvpr);
 	return seamcall(TDH_VP_CREATE, tdvpr, tdr, 0, 0, 0, NULL);
 }
 
arch/x86/kvm/vmx/tdx_stubs.c new file +50 -0

50 lines not on this page · raw message

arch/x86/kvm/vmx/vmenter.S +146 -0

161 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.c +0 -39
index 6f38e0d2e1b6..4b7d6fe63d58 100644
@@ -3752,45 +3752,6 @@ void vmx_msr_filter_changed(struct kvm_vcpu *vcpu)
 	pt_update_intercept_for_msr(vcpu);
 }
 
-static inline bool kvm_vcpu_trigger_posted_interrupt(struct kvm_vcpu *vcpu,
-						     bool nested)
-{
-#ifdef CONFIG_SMP
-	int pi_vec = nested ? POSTED_INTR_NESTED_VECTOR : POSTED_INTR_VECTOR;
-
-	if (vcpu->mode == IN_GUEST_MODE) {
-		/*
-		 * The vector of interrupt to be delivered to vcpu had
-		 * been set in PIR before this function.
-		 *
-		 * Following cases will be reached in this block, and
-		 * we always send a notification event in all cases as
-		 * explained below.
-		 *
-		 * Case 1: vcpu keeps in non-root mode. Sending a
-		 * notification event posts the interrupt to vcpu.
-		 *
-		 * Case 2: vcpu exits to root mode and is still
-		 * runnable. PIR will be synced to vIRR before the
-		 * next vcpu entry. Sending a notification event in
-		 * this case has no effect, as vcpu is not in root
-		 * mode.
-		 *
-		 * Case 3: vcpu exits to root mode and is blocked.
-		 * vcpu_block() has already synced PIR to vIRR and
-		 * never blocks vcpu if vIRR is not cleared. Therefore,
-		 * a blocked vcpu here does not wait for any requested
-		 * interrupts in PIR, and sending a notification event
-		 * which has no effect is safe here.
-		 */
-
-		apic->send_IPI_mask(get_cpu_mask(vcpu->cpu), pi_vec);
-		return true;
-	}
-#endif
-	return false;
-}
-
 static int vmx_deliver_nested_posted_interrupt(struct kvm_vcpu *vcpu,
 						int vector)
 {
arch/x86/kvm/vmx/x86_ops.h +80 -0

90 lines not on this page · raw message

arch/x86/kvm/x86.c +4 -4
index 447f0d5b53c7..da75530d75e9 100644
@@ -292,6 +292,7 @@ const struct kvm_stats_header kvm_vcpu_stats_header = {
 };
 
 u64 __read_mostly host_xcr0;
+EXPORT_SYMBOL_GPL(host_xcr0);
 u64 __read_mostly supported_xcr0;
 EXPORT_SYMBOL_GPL(supported_xcr0);
 
@@ -2265,9 +2266,7 @@ static int set_tsc_khz(struct kvm_vcpu *vcpu, u32 user_tsc_khz, bool scale)
 	u64 ratio;
 
 	/* Guest TSC same frequency as host TSC? */
-	if (!scale || vcpu->kvm->arch.tsc_immutable) {
-		if (scale)
-			pr_warn_ratelimited("Guest TSC immutable, scaling not supported\n");
+	if (!scale) {
 		kvm_vcpu_write_tsc_multiplier(vcpu, kvm_default_tsc_scaling_ratio);
 		return 0;
 	}
@@ -10740,7 +10739,8 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
 {
 	int ret;
 
-	if (vcpu->arch.guest_state_protected)
+	if (vcpu->arch.guest_state_protected ||
+	    vcpu->kvm->arch.vm_type == KVM_X86_TDX_VM)
 		return -EINVAL;
 
 	vcpu_load(vcpu);
tools/arch/x86/include/uapi/asm/kvm.h +51 -0

55 lines not on this page · raw message

Signature · 1 line
2.25.1
#113 lore raw

RFC PATCH v3 58/59 KVM: TDX: exit to user space on GET_QUOTE, SETUP_EVENT_NOTIFY_INTERRUPT

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

GET_QUOTE, SETUP_EVENT_NOTIFY_INTERRUPT TDG.VP.VMCALL requires user space to handle them on behalf of kvm kernel module.

Introduce new kvm exit, KVM_EXIT_TDX, and when GET_QUOTE and SETUP_EVENT_NOTIFY_INTERRUPT is called by TD guest, transfer the execution to user space as KVM exit to the user space. TDG_VP_VMCALL_INVALID_OPERAND is set as default return value to avoid random value. User space should update R10 if necessary.

---
 arch/x86/kvm/vmx/tdx.c   | 113 +++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/kvm.h |  57 ++++++++++++++++++++
 2 files changed, 170 insertions(+)
arch/x86/kvm/vmx/tdx.c +113 -0
index 53fc01f3bab1..a87db46477cc 100644
@@ -152,6 +152,18 @@ BUILD_TDVMCALL_ACCESSORS(p2, r13);
 BUILD_TDVMCALL_ACCESSORS(p3, r14);
 BUILD_TDVMCALL_ACCESSORS(p4, r15);
 
+#define TDX_VMCALL_REG_MASK_RBX	BIT_ULL(2)
+#define TDX_VMCALL_REG_MASK_RDX	BIT_ULL(3)
+#define TDX_VMCALL_REG_MASK_RBP	BIT_ULL(5)
+#define TDX_VMCALL_REG_MASK_RSI	BIT_ULL(6)
+#define TDX_VMCALL_REG_MASK_RDI	BIT_ULL(7)
+#define TDX_VMCALL_REG_MASK_R8	BIT_ULL(8)
+#define TDX_VMCALL_REG_MASK_R9	BIT_ULL(9)
+#define TDX_VMCALL_REG_MASK_R12	BIT_ULL(12)
+#define TDX_VMCALL_REG_MASK_R13	BIT_ULL(13)
+#define TDX_VMCALL_REG_MASK_R14	BIT_ULL(14)
+#define TDX_VMCALL_REG_MASK_R15	BIT_ULL(15)
+
 static __always_inline unsigned long tdvmcall_exit_type(struct kvm_vcpu *vcpu)
 {
 	return kvm_r10_read(vcpu);
@@ -1122,6 +1134,91 @@ static int tdx_map_gpa(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
+static int tdx_complete_vp_vmcall(struct kvm_vcpu *vcpu)
+{
+	struct kvm_tdx_vmcall *tdx_vmcall = &vcpu->run->tdx.u.vmcall;
+	__u64 reg_mask;
+
+	tdvmcall_set_return_code(vcpu, tdx_vmcall->status_code);
+	tdvmcall_set_return_val(vcpu, tdx_vmcall->out_r11);
+
+	reg_mask = kvm_rcx_read(vcpu);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R12)
+		kvm_r12_write(vcpu, tdx_vmcall->out_r12);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R13)
+		kvm_r13_write(vcpu, tdx_vmcall->out_r13);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R14)
+		kvm_r14_write(vcpu, tdx_vmcall->out_r14);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R15)
+		kvm_r15_write(vcpu, tdx_vmcall->out_r15);
+	if (reg_mask & TDX_VMCALL_REG_MASK_RBX)
+		kvm_rbx_write(vcpu, tdx_vmcall->out_rbx);
+	if (reg_mask & TDX_VMCALL_REG_MASK_RDI)
+		kvm_rdi_write(vcpu, tdx_vmcall->out_rdi);
+	if (reg_mask & TDX_VMCALL_REG_MASK_RSI)
+		kvm_rsi_write(vcpu, tdx_vmcall->out_rsi);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R8)
+		kvm_r8_write(vcpu, tdx_vmcall->out_r8);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R9)
+		kvm_r9_write(vcpu, tdx_vmcall->out_r9);
+	if (reg_mask & TDX_VMCALL_REG_MASK_RDX)
+		kvm_rdx_write(vcpu, tdx_vmcall->out_rdx);
+
+	return 1;
+}
+
+static int tdx_vp_vmcall_to_user(struct kvm_vcpu *vcpu)
+{
+	struct kvm_tdx_vmcall *tdx_vmcall = &vcpu->run->tdx.u.vmcall;
+	__u64 reg_mask;
+
+	vcpu->arch.complete_userspace_io = tdx_complete_vp_vmcall;
+	memset(tdx_vmcall, 0, sizeof(*tdx_vmcall));
+
+	vcpu->run->exit_reason = KVM_EXIT_TDX;
+	vcpu->run->tdx.type = KVM_EXIT_TDX_VMCALL;
+	tdx_vmcall->type = tdvmcall_exit_type(vcpu);
+	tdx_vmcall->subfunction = tdvmcall_exit_reason(vcpu);
+
+	reg_mask = kvm_rcx_read(vcpu);
+	tdx_vmcall->reg_mask = reg_mask;
+	if (reg_mask & TDX_VMCALL_REG_MASK_R12)
+		tdx_vmcall->in_r12 = kvm_r12_read(vcpu);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R13)
+		tdx_vmcall->in_r13 = kvm_r13_read(vcpu);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R14)
+		tdx_vmcall->in_r14 = kvm_r14_read(vcpu);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R15)
+		tdx_vmcall->in_r15 = kvm_r15_read(vcpu);
+	if (reg_mask & TDX_VMCALL_REG_MASK_RBX)
+		tdx_vmcall->in_rbx = kvm_rbx_read(vcpu);
+	if (reg_mask & TDX_VMCALL_REG_MASK_RDI)
+		tdx_vmcall->in_rdi = kvm_rdi_read(vcpu);
+	if (reg_mask & TDX_VMCALL_REG_MASK_RSI)
+		tdx_vmcall->in_rsi = kvm_rsi_read(vcpu);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R8)
+		tdx_vmcall->in_r8 = kvm_r8_read(vcpu);
+	if (reg_mask & TDX_VMCALL_REG_MASK_R9)
+		tdx_vmcall->in_r9 = kvm_r9_read(vcpu);
+	if (reg_mask & TDX_VMCALL_REG_MASK_RDX)
+		tdx_vmcall->in_rdx = kvm_rdx_read(vcpu);
+
+	/* notify userspace to handle the request */
+	return 0;
+}
+
+static int tdx_get_quote(struct kvm_vcpu *vcpu)
+{
+	gpa_t gpa = tdvmcall_p1_read(vcpu);
+
+	if (!IS_ALIGNED(gpa, PAGE_SIZE)) {
+		tdvmcall_set_return_code(vcpu, TDG_VP_VMCALL_INVALID_OPERAND);
+		return 1;
+	}
+
+	return tdx_vp_vmcall_to_user(vcpu);
+}
+
 static int tdx_report_fatal_error(struct kvm_vcpu *vcpu)
 {
 	vcpu->run->exit_reason = KVM_EXIT_SYSTEM_EVENT;
@@ -1130,6 +1227,18 @@ static int tdx_report_fatal_error(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
+static int tdx_setup_event_notify_interrupt(struct kvm_vcpu *vcpu)
+{
+	u64 vector = tdvmcall_p1_read(vcpu);
+
+	if (!(vector >= 32 && vector <= 255)) {
+		tdvmcall_set_return_code(vcpu, TDG_VP_VMCALL_INVALID_OPERAND);
+		return 1;
+	}
+
+	return tdx_vp_vmcall_to_user(vcpu);
+}
+
 static int handle_tdvmcall(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_tdx *tdx = to_tdx(vcpu);
@@ -1158,8 +1267,12 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu)
 		return tdx_emulate_mmio(vcpu);
 	case TDG_VP_VMCALL_MAP_GPA:
 		return tdx_map_gpa(vcpu);
+	case TDG_VP_VMCALL_GET_QUOTE:
+		return tdx_get_quote(vcpu);
 	case TDG_VP_VMCALL_REPORT_FATAL_ERROR:
 		return tdx_report_fatal_error(vcpu);
+	case TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT:
+		return tdx_setup_event_notify_interrupt(vcpu);
 	default:
 		break;
 	}
include/uapi/linux/kvm.h +57 -0

75 lines not on this page · raw message

Signature · 1 line
2.25.1
#102 lore raw

RFC PATCH v3 54/59 KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

In reply to [email protected]

From: Xiaoyao Li <[email protected]>

Introduce a per-vm variable initial_tsc_khz to hold the default tsc_khz for kvm_arch_vcpu_create().

This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.

---
 arch/x86/include/asm/kvm_host.h |    1 +
 arch/x86/kvm/vmx/tdx.c          | 2233 +++++++++++++++++++++++++++++++
 arch/x86/kvm/x86.c              |    3 +-
 3 files changed, 2236 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kvm/vmx/tdx.c
arch/x86/include/asm/kvm_host.h +1 -0
index 17d6e4bcf84b..f10c7c2830e5 100644
@@ -1111,6 +1111,7 @@ struct kvm_arch {
 	u64 last_tsc_write;
 	u32 last_tsc_khz;
 	u64 last_tsc_offset;
+	u32 initial_tsc_khz;
 	u64 cur_tsc_nsec;
 	u64 cur_tsc_write;
 	u64 cur_tsc_offset;
arch/x86/kvm/vmx/tdx.c new file +2233 -0

2,233 lines not on this page · raw message

arch/x86/kvm/x86.c +2 -1
index 09da7cbedb5f..447f0d5b53c7 100644
@@ -11019,7 +11019,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 	vcpu->arch.msr_platform_info = MSR_PLATFORM_INFO_CPUID_FAULT;
 	kvm_vcpu_mtrr_init(vcpu);
 	vcpu_load(vcpu);
-	kvm_set_tsc_khz(vcpu, max_tsc_khz);
+	kvm_set_tsc_khz(vcpu, vcpu->kvm->arch.initial_tsc_khz);
 	kvm_vcpu_reset(vcpu, false);
 	kvm_init_mmu(vcpu);
 	vcpu_put(vcpu);
@@ -11473,6 +11473,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
 
 	kvm->arch.guest_can_read_msr_platform_info = true;
+	kvm->arch.initial_tsc_khz = max_tsc_khz;
 
 #if IS_ENABLED(CONFIG_HYPERV)
 	spin_lock_init(&kvm->arch.hv_root_tdp_lock);
Signature · 1 line
2.25.1
#36 lore raw

RFC PATCH v3 22/59 KVM: x86: add per-VM flags to disable SMI/INIT/SIPI

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

Add a flag to let TDX disallow to inject interrupt with delivery mode of SMI/INIT/SIPI. add a check to reject SMI/INIT interrupt delivery mode.

---
 arch/x86/include/asm/kvm_host.h | 2 ++
 arch/x86/kvm/irq_comm.c         | 4 ++++
 arch/x86/kvm/x86.c              | 3 +--
 3 files changed, 7 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
index 545b556e420c..5ed07f31459e 100644
@@ -1133,6 +1133,8 @@ struct kvm_arch {
 	enum kvm_irqchip_mode irqchip_mode;
 	u8 nr_reserved_ioapic_pins;
 	bool eoi_intercept_unsupported;
+	bool smm_unsupported;
+	bool init_sipi_unsupported;
 
 	bool disabled_lapic_found;
 
arch/x86/kvm/irq_comm.c +4 -0
index bcfac99db579..396ccf086bdd 100644
@@ -128,6 +128,10 @@ static inline bool kvm_msi_route_invalid(struct kvm *kvm,
 			       .data = e->msi.data };
 	return  (kvm->arch.eoi_intercept_unsupported &&
 		 msg.arch_data.is_level) ||
+		(kvm->arch.smm_unsupported &&
+		 msg.arch_data.delivery_mode == APIC_DELIVERY_MODE_SMI) ||
+		(kvm->arch.init_sipi_unsupported &&
+		 msg.arch_data.delivery_mode == APIC_DELIVERY_MODE_INIT) ||
 		(kvm->arch.x2apic_format && (msg.address_hi & 0xff));
 }
 
arch/x86/kvm/x86.c +1 -2
index 1573dddd1e43..f2b6a3f89e9e 100644
@@ -4553,8 +4553,7 @@ static int kvm_vcpu_ioctl_nmi(struct kvm_vcpu *vcpu)
 
 static int kvm_vcpu_ioctl_smi(struct kvm_vcpu *vcpu)
 {
-	/* TODO: use more precise flag */
-	if (vcpu->arch.guest_state_protected)
+	if (vcpu->kvm->arch.smm_unsupported)
 		return -EINVAL;
 
 	kvm_make_request(KVM_REQ_SMI, vcpu);
Signature · 1 line
2.25.1
#37 lore raw

RFC PATCH v3 23/59 KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Let userspace, or in the case of TDX, KVM itself, enable X2APIC even if X2APIC is not reported as supported in the guest's CPU model. KVM generally does not force specific ordering between ioctls(), e.g. this forces userspace to configure CPUID before MSRs. And for TDX, vCPUs will always run with X2APIC enabled, e.g. KVM will want/need to enable X2APIC from time zero.

---
 arch/x86/kvm/x86.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +5 -2
index f2b6a3f89e9e..0221ef691a15 100644
@@ -466,8 +466,11 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
 	enum lapic_mode old_mode = kvm_get_apic_mode(vcpu);
 	enum lapic_mode new_mode = kvm_apic_mode(msr_info->data);
-	u64 reserved_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu) | 0x2ff |
-		(guest_cpuid_has(vcpu, X86_FEATURE_X2APIC) ? 0 : X2APIC_ENABLE);
+	u64 reserved_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu) | 0x2ff;
+
+	if (!msr_info->host_initiated &&
+	    !guest_cpuid_has(vcpu, X86_FEATURE_X2APIC))
+		reserved_bits |= X2APIC_ENABLE;
 
 	if ((msr_info->data & reserved_bits) != 0 || new_mode == LAPIC_MODE_INVALID)
 		return 1;
Signature · 1 line
2.25.1
#114 lore raw

RFC PATCH v3 59/59 Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

Add a documentation to Intel Trusted Docmain Extensions(TDX) support.

---
 Documentation/virt/kvm/api.rst       |   9 +-
 Documentation/virt/kvm/intel-tdx.rst | 359 +++++++++++++++++++++++++++
 2 files changed, 367 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/virt/kvm/intel-tdx.rst
Documentation/virt/kvm/api.rst +8 -1
index aeeb071c7688..72341707d98f 100644
@@ -1366,6 +1366,9 @@ It is recommended to use this API instead of the KVM_SET_MEMORY_REGION ioctl.
 The KVM_SET_MEMORY_REGION does not allow fine grained control over memory
 allocation and is deprecated.
 
+For TDX guest, deleting/moving memory region loses guest memory contents.
+Read only region isn't supported.  Only as-id 0 is supported.
+
 
 4.36 KVM_SET_TSS_ADDR
 ---------------------
@@ -4489,7 +4492,7 @@ H_GET_CPU_CHARACTERISTICS hypercall.
 
 :Capability: basic
 :Architectures: x86
-:Type: vm
+:Type: vm ioctl, vcpu ioctl
 :Parameters: an opaque platform specific structure (in/out)
 :Returns: 0 on success; -1 on error
 
@@ -4501,6 +4504,10 @@ Currently, this ioctl is used for issuing Secure Encrypted Virtualization
 (SEV) commands on AMD Processors. The SEV commands are defined in
 Documentation/virt/kvm/amd-memory-encryption.rst.
 
+Currently, this ioctl is used for issuing Trusted Domain Extensions
+(TDX) commands on Intel Processors. The TDX commands are defined in
+Documentation/virt/kvm/intel-tdx.rst.
+
 4.111 KVM_MEMORY_ENCRYPT_REG_REGION
 -----------------------------------
 
Documentation/virt/kvm/intel-tdx.rst new file +359 -0

359 lines not on this page · raw message

Signature · 1 line
2.25.1
#34 lore raw

RFC PATCH v3 20/59 KVM: Add per-VM flag to disable dirty logging of memslots for TDs

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add a flag for TDX to mark dirty logging as unsupported.

---
 include/linux/kvm_host.h | 1 +
 virt/kvm/kvm_main.c      | 5 ++++-
 2 files changed, 5 insertions(+), 1 deletion(-)
include/linux/kvm_host.h +1 -0
index d59debe99212..08284807d48e 100644
@@ -624,6 +624,7 @@ struct kvm {
 	struct notifier_block pm_notifier;
 #endif
 	char stats_id[KVM_STATS_NAME_SIZE];
+	bool dirty_log_unsupported;
 #ifdef __KVM_HAVE_READONLY_MEM
 	bool readonly_mem_unsupported;
 #endif
virt/kvm/kvm_main.c +4 -1
index aaa6d69265ea..5bf456734e7d 100644
@@ -1425,7 +1425,10 @@ static void update_memslots(struct kvm_memslots *slots,
 static int check_memory_region_flags(struct kvm *kvm,
 				     const struct kvm_userspace_memory_region *mem)
 {
-	u32 valid_flags = KVM_MEM_LOG_DIRTY_PAGES;
+	u32 valid_flags = 0;
+
+	if (!kvm->dirty_log_unsupported)
+		valid_flags |= KVM_MEM_LOG_DIRTY_PAGES;
 
 #ifdef __KVM_HAVE_READONLY_MEM
 	if (!kvm->readonly_mem_unsupported)
Signature · 1 line
2.25.1
#33 lore raw

RFC PATCH v3 19/59 KVM: Add per-VM flag to mark read-only memory as unsupported

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

Add a flag for TDX to flag RO memory as unsupported and propagate it to KVM_MEM_READONLY to allow reporting RO memory as unsupported on a per-VM basis. TDX1 doesn't expose permission bits to the VMM in the SEPT tables, i.e. doesn't support read-only private memory.

---
 arch/x86/kvm/x86.c       | 4 +++-
 include/linux/kvm_host.h | 3 +++
 virt/kvm/kvm_main.c      | 8 +++++---
 3 files changed, 11 insertions(+), 4 deletions(-)
arch/x86/kvm/x86.c +3 -1
index 0ebd60846079..535f65b0915d 100644
@@ -4121,7 +4121,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_ASYNC_PF_INT:
 	case KVM_CAP_GET_TSC_KHZ:
 	case KVM_CAP_KVMCLOCK_CTRL:
-	case KVM_CAP_READONLY_MEM:
 	case KVM_CAP_HYPERV_TIME:
 	case KVM_CAP_IOAPIC_POLARITY_IGNORED:
 	case KVM_CAP_TSC_DEADLINE_TIMER:
@@ -4239,6 +4238,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		if (static_call(kvm_x86_is_vm_type_supported)(KVM_X86_TDX_VM))
 			r |= BIT(KVM_X86_TDX_VM);
 		break;
+	case KVM_CAP_READONLY_MEM:
+		r = kvm && kvm->readonly_mem_unsupported ? 0 : 1;
+		break;
 	default:
 		break;
 	}
include/linux/kvm_host.h +3 -0
index 625de21ceb43..d59debe99212 100644
@@ -624,6 +624,9 @@ struct kvm {
 	struct notifier_block pm_notifier;
 #endif
 	char stats_id[KVM_STATS_NAME_SIZE];
+#ifdef __KVM_HAVE_READONLY_MEM
+	bool readonly_mem_unsupported;
+#endif
 };
 
 #define kvm_err(fmt, ...) \
virt/kvm/kvm_main.c +5 -3
index 4a1dbf0b1d3d..aaa6d69265ea 100644
@@ -1422,12 +1422,14 @@ static void update_memslots(struct kvm_memslots *slots,
 	}
 }
 
-static int check_memory_region_flags(const struct kvm_userspace_memory_region *mem)
+static int check_memory_region_flags(struct kvm *kvm,
+				     const struct kvm_userspace_memory_region *mem)
 {
 	u32 valid_flags = KVM_MEM_LOG_DIRTY_PAGES;
 
 #ifdef __KVM_HAVE_READONLY_MEM
-	valid_flags |= KVM_MEM_READONLY;
+	if (!kvm->readonly_mem_unsupported)
+		valid_flags |= KVM_MEM_READONLY;
 #endif
 
 	if (mem->flags & ~valid_flags)
@@ -1665,7 +1667,7 @@ int __kvm_set_memory_region(struct kvm *kvm,
 	int as_id, id;
 	int r;
 
-	r = check_memory_region_flags(mem);
+	r = check_memory_region_flags(kvm, mem);
 	if (r)
 		return r;
 
Signature · 1 line
2.25.1
#28 lore raw

RFC PATCH v3 17/59 KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add a flag to disallow MCE injection and reject KVM_X86_SETUP_MCE with -EINVAL when set. TDX doesn't support injecting exceptions, including (virtual) #MCs.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/x86.c              | 14 +++++++-------
 2 files changed, 8 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index ebc4de32bf0e..e912e1e853ef 100644
@@ -1142,6 +1142,7 @@ struct kvm_arch {
 
 	bool bus_lock_detection_enabled;
 	bool irq_injection_disallowed;
+	bool mce_injection_disallowed;
 	/*
 	 * If exit_on_emulation_error is set, and the in-kernel instruction
 	 * emulator fails to emulate an instruction, allow userspace
arch/x86/kvm/x86.c +7 -7
index b399e64e8863..fefa4602e879 100644
@@ -4561,15 +4561,16 @@ static int vcpu_ioctl_tpr_access_reporting(struct kvm_vcpu *vcpu,
 static int kvm_vcpu_ioctl_x86_setup_mce(struct kvm_vcpu *vcpu,
 					u64 mcg_cap)
 {
-	int r;
 	unsigned bank_num = mcg_cap & 0xff, bank;
 
-	r = -EINVAL;
+	if (vcpu->kvm->arch.mce_injection_disallowed)
+		return -EINVAL;
+
 	if (!bank_num || bank_num > KVM_MAX_MCE_BANKS)
-		goto out;
+		return -EINVAL;
 	if (mcg_cap & ~(kvm_mce_cap_supported | 0xff | 0xff0000))
-		goto out;
-	r = 0;
+		return -EINVAL;
+
 	vcpu->arch.mcg_cap = mcg_cap;
 	/* Init IA32_MCG_CTL to all 1s */
 	if (mcg_cap & MCG_CTL_P)
@@ -4579,8 +4580,7 @@ static int kvm_vcpu_ioctl_x86_setup_mce(struct kvm_vcpu *vcpu,
 		vcpu->arch.mce_banks[bank*4] = ~(u64)0;
 
 	static_call(kvm_x86_setup_mce)(vcpu);
-out:
-	return r;
+	return 0;
 }
 
 static int kvm_vcpu_ioctl_x86_set_mce(struct kvm_vcpu *vcpu,
Signature · 1 line
2.25.1
#23 lore raw

RFC PATCH v3 15/59 KVM: x86: Introduce "protected guest" concept and block disallowed ioctls

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add 'guest_state_protected' to mark a VM's state as being protected by hardware/firmware, e.g. SEV-ES or TDX-SEAM. Use the flag to disallow ioctls() and/or flows that attempt to access protected state.

Return an error if userspace attempts to get/set register state for a protected VM, e.g. a non-debug TDX guest. KVM can't provide sane data, it's userspace's responsibility to avoid attempting to read guest state when it's known to be inaccessible.

Retrieving vCPU events is the one exception, as the userspace VMM is allowed to inject NMIs.

---
 arch/x86/kvm/x86.c | 80 ++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 66 insertions(+), 14 deletions(-)
arch/x86/kvm/x86.c +66 -14
index 5f4b6f70489b..b21fcf3c0cc8 100644
@@ -4539,6 +4539,10 @@ static int kvm_vcpu_ioctl_nmi(struct kvm_vcpu *vcpu)
 
 static int kvm_vcpu_ioctl_smi(struct kvm_vcpu *vcpu)
 {
+	/* TODO: use more precise flag */
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;
+
 	kvm_make_request(KVM_REQ_SMI, vcpu);
 
 	return 0;
@@ -4585,6 +4589,10 @@ static int kvm_vcpu_ioctl_x86_set_mce(struct kvm_vcpu *vcpu,
 	unsigned bank_num = mcg_cap & 0xff;
 	u64 *banks = vcpu->arch.mce_banks;
 
+	/* TODO: use more precise flag */
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;
+
 	if (mce->bank >= bank_num || !(mce->status & MCI_STATUS_VAL))
 		return -EINVAL;
 	/*
@@ -4680,7 +4688,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu *vcpu,
 		vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft;
 	events->interrupt.nr = vcpu->arch.interrupt.nr;
 	events->interrupt.soft = 0;
-	events->interrupt.shadow = static_call(kvm_x86_get_interrupt_shadow)(vcpu);
+	if (!vcpu->arch.guest_state_protected)
+		events->interrupt.shadow = static_call(kvm_x86_get_interrupt_shadow)(vcpu);
 
 	events->nmi.injected = vcpu->arch.nmi_injected;
 	events->nmi.pending = vcpu->arch.nmi_pending != 0;
@@ -4709,11 +4718,17 @@ static void kvm_smm_changed(struct kvm_vcpu *vcpu, bool entering_smm);
 static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu,
 					      struct kvm_vcpu_events *events)
 {
-	if (events->flags & ~(KVM_VCPUEVENT_VALID_NMI_PENDING
-			      | KVM_VCPUEVENT_VALID_SIPI_VECTOR
-			      | KVM_VCPUEVENT_VALID_SHADOW
-			      | KVM_VCPUEVENT_VALID_SMM
-			      | KVM_VCPUEVENT_VALID_PAYLOAD))
+	u32 allowed_flags = KVM_VCPUEVENT_VALID_NMI_PENDING |
+			    KVM_VCPUEVENT_VALID_SIPI_VECTOR |
+			    KVM_VCPUEVENT_VALID_SHADOW |
+			    KVM_VCPUEVENT_VALID_SMM |
+			    KVM_VCPUEVENT_VALID_PAYLOAD;
+
+	/* TODO: introduce more precise flag */
+	if (vcpu->arch.guest_state_protected)
+		allowed_flags = KVM_VCPUEVENT_VALID_NMI_PENDING;
+
+	if (events->flags & ~allowed_flags)
 		return -EINVAL;
 
 	if (events->flags & KVM_VCPUEVENT_VALID_PAYLOAD) {
@@ -4789,17 +4804,22 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
-static void kvm_vcpu_ioctl_x86_get_debugregs(struct kvm_vcpu *vcpu,
-					     struct kvm_debugregs *dbgregs)
+static int kvm_vcpu_ioctl_x86_get_debugregs(struct kvm_vcpu *vcpu,
+					    struct kvm_debugregs *dbgregs)
 {
 	unsigned long val;
 
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;
+
 	memcpy(dbgregs->db, vcpu->arch.db, sizeof(vcpu->arch.db));
 	kvm_get_dr(vcpu, 6, &val);
 	dbgregs->dr6 = val;
 	dbgregs->dr7 = vcpu->arch.dr7;
 	dbgregs->flags = 0;
 	memset(&dbgregs->reserved, 0, sizeof(dbgregs->reserved));
+
+	return 0;
 }
 
 static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu,
@@ -4813,6 +4833,9 @@ static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu,
 	if (!kvm_dr7_valid(dbgregs->dr7))
 		return -EINVAL;
 
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;
+
 	memcpy(vcpu->arch.db, dbgregs->db, sizeof(vcpu->arch.db));
 	kvm_update_dr0123(vcpu);
 	vcpu->arch.dr6 = dbgregs->dr6;
@@ -4845,18 +4868,22 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu,
 					      supported_xcr0, &vcpu->arch.pkru);
 }
 
-static void kvm_vcpu_ioctl_x86_get_xcrs(struct kvm_vcpu *vcpu,
-					struct kvm_xcrs *guest_xcrs)
+static int kvm_vcpu_ioctl_x86_get_xcrs(struct kvm_vcpu *vcpu,
+				       struct kvm_xcrs *guest_xcrs)
 {
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;
+
 	if (!boot_cpu_has(X86_FEATURE_XSAVE)) {
 		guest_xcrs->nr_xcrs = 0;
-		return;
+		return 0;
 	}
 
 	guest_xcrs->nr_xcrs = 1;
 	guest_xcrs->flags = 0;
 	guest_xcrs->xcrs[0].xcr = XCR_XFEATURE_ENABLED_MASK;
 	guest_xcrs->xcrs[0].value = vcpu->arch.xcr0;
+	return 0;
 }
 
 static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu *vcpu,
@@ -4864,6 +4891,9 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu *vcpu,
 {
 	int i, r = 0;
 
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;
+
 	if (!boot_cpu_has(X86_FEATURE_XSAVE))
 		return -EINVAL;
 
@@ -5247,7 +5277,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 	case KVM_GET_DEBUGREGS: {
 		struct kvm_debugregs dbgregs;
 
-		kvm_vcpu_ioctl_x86_get_debugregs(vcpu, &dbgregs);
+		r = kvm_vcpu_ioctl_x86_get_debugregs(vcpu, &dbgregs);
+		if (r)
+			break;
 
 		r = -EFAULT;
 		if (copy_to_user(argp, &dbgregs,
@@ -5297,7 +5329,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 		if (!u.xcrs)
 			break;
 
-		kvm_vcpu_ioctl_x86_get_xcrs(vcpu, u.xcrs);
+		r = kvm_vcpu_ioctl_x86_get_xcrs(vcpu, u.xcrs);
+		if (r)
+			break;
 
 		r = -EFAULT;
 		if (copy_to_user(argp, u.xcrs,
@@ -10188,6 +10222,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 		goto out;
 	}
 
+	if (vcpu->arch.guest_state_protected &&
+	    (kvm_run->kvm_valid_regs || kvm_run->kvm_dirty_regs)) {
+		r = -EINVAL;
+		goto out;
+	}
+
 	if (kvm_run->kvm_dirty_regs) {
 		r = sync_regs(vcpu);
 		if (r != 0)
@@ -10218,7 +10258,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 
 out:
 	kvm_put_guest_fpu(vcpu);
-	if (kvm_run->kvm_valid_regs)
+	if (kvm_run->kvm_valid_regs && !vcpu->arch.guest_state_protected)
 		store_regs(vcpu);
 	post_kvm_run_save(vcpu);
 	kvm_sigset_deactivate(vcpu);
@@ -10265,6 +10305,9 @@ static void __get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
 
 int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
 {
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;
+
 	vcpu_load(vcpu);
 	__get_regs(vcpu, regs);
 	vcpu_put(vcpu);
@@ -10305,6 +10348,9 @@ static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
 
 int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
 {
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;
+
 	vcpu_load(vcpu);
 	__set_regs(vcpu, regs);
 	vcpu_put(vcpu);
@@ -10387,6 +10433,9 @@ static void __get_sregs2(struct kvm_vcpu *vcpu, struct kvm_sregs2 *sregs2)
 int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu,
 				  struct kvm_sregs *sregs)
 {
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;
+
 	vcpu_load(vcpu);
 	__get_sregs(vcpu, sregs);
 	vcpu_put(vcpu);
@@ -10635,6 +10684,9 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,
 {
 	int ret;
 
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;
+
 	vcpu_load(vcpu);
 	ret = __set_sregs(vcpu, sregs);
 	vcpu_put(vcpu);
Signature · 1 line
2.25.1
#25 lore raw

RFC PATCH v3 16/59 KVM: x86: Add per-VM flag to disable direct IRQ injection

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add a flag to disable IRQ injection, which is not supported by TDX.

---
 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/x86.c              | 4 +++-
 2 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
index cdb908ed7d5b..ebc4de32bf0e 100644
@@ -1141,6 +1141,7 @@ struct kvm_arch {
 	bool exception_payload_enabled;
 
 	bool bus_lock_detection_enabled;
+	bool irq_injection_disallowed;
 	/*
 	 * If exit_on_emulation_error is set, and the in-kernel instruction
 	 * emulator fails to emulate an instruction, allow userspace
arch/x86/kvm/x86.c +3 -1
index b21fcf3c0cc8..b399e64e8863 100644
@@ -4506,7 +4506,8 @@ static int kvm_vcpu_ready_for_interrupt_injection(struct kvm_vcpu *vcpu)
 static int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
 				    struct kvm_interrupt *irq)
 {
-	if (irq->irq >= KVM_NR_INTERRUPTS)
+	if (irq->irq >= KVM_NR_INTERRUPTS ||
+	    vcpu->kvm->arch.irq_injection_disallowed)
 		return -EINVAL;
 
 	if (!irqchip_in_kernel(vcpu->kvm)) {
@@ -8997,6 +8998,7 @@ static int emulator_fix_hypercall(struct x86_emulate_ctxt *ctxt)
 static int dm_request_for_irq_injection(struct kvm_vcpu *vcpu)
 {
 	return vcpu->run->request_interrupt_window &&
+	       !vcpu->kvm->arch.irq_injection_disallowed &&
 		likely(!pic_in_kernel(vcpu->kvm));
 }
 
Signature · 1 line
2.25.1
#16 lore raw

RFC PATCH v3 13/59 KVM: Add max_vcpus field in common 'struct kvm'

In reply to [email protected]

From: Sean Christopherson <[email protected]>

---
 arch/arm64/include/asm/kvm_host.h | 3 ---
 arch/arm64/kvm/arm.c              | 7 ++-----
 arch/arm64/kvm/vgic/vgic-init.c   | 6 +++---
 include/linux/kvm_host.h          | 1 +
 virt/kvm/kvm_main.c               | 3 ++-
 5 files changed, 8 insertions(+), 12 deletions(-)
arch/arm64/include/asm/kvm_host.h +0 -3
index 2a5f7f38006f..ff3f4ab57eba 100644
@@ -108,9 +108,6 @@ struct kvm_arch {
 	/* VTCR_EL2 value for this VM */
 	u64    vtcr;
 
-	/* The maximum number of vCPUs depends on the used GIC model */
-	int max_vcpus;
-
 	/* Interrupt controller */
 	struct vgic_dist	vgic;
 
arch/arm64/kvm/arm.c +2 -5
index 2f03cbfefe67..14df7ee047e4 100644
@@ -153,7 +153,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	kvm_vgic_early_init(kvm);
 
 	/* The maximum number of VCPUs is limited by the host's GIC model */
-	kvm->arch.max_vcpus = kvm_arm_default_max_vcpus();
+	kvm->max_vcpus = kvm_arm_default_max_vcpus();
 
 	set_default_spectre(kvm);
 
@@ -228,7 +228,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_MAX_VCPUS:
 	case KVM_CAP_MAX_VCPU_ID:
 		if (kvm)
-			r = kvm->arch.max_vcpus;
+			r = kvm->max_vcpus;
 		else
 			r = kvm_arm_default_max_vcpus();
 		break;
@@ -304,9 +304,6 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
 	if (irqchip_in_kernel(kvm) && vgic_initialized(kvm))
 		return -EBUSY;
 
-	if (id >= kvm->arch.max_vcpus)
-		return -EINVAL;
-
 	return 0;
 }
 
arch/arm64/kvm/vgic/vgic-init.c +3 -3
index 0a06d0648970..906aee52f2bc 100644
@@ -97,11 +97,11 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)
 	ret = 0;
 
 	if (type == KVM_DEV_TYPE_ARM_VGIC_V2)
-		kvm->arch.max_vcpus = VGIC_V2_MAX_CPUS;
+		kvm->max_vcpus = VGIC_V2_MAX_CPUS;
 	else
-		kvm->arch.max_vcpus = VGIC_V3_MAX_CPUS;
+		kvm->max_vcpus = VGIC_V3_MAX_CPUS;
 
-	if (atomic_read(&kvm->online_vcpus) > kvm->arch.max_vcpus) {
+	if (atomic_read(&kvm->online_vcpus) > kvm->max_vcpus) {
 		ret = -E2BIG;
 		goto out_unlock;
 	}
include/linux/kvm_host.h +1 -0
index 9e0667e3723e..625de21ceb43 100644
@@ -566,6 +566,7 @@ struct kvm {
 	 * and is accessed atomically.
 	 */
 	atomic_t online_vcpus;
+	int max_vcpus;
 	int created_vcpus;
 	int last_boosted_vcpu;
 	struct list_head vm_list;
virt/kvm/kvm_main.c +2 -1
index 8544e92db2da..4a1dbf0b1d3d 100644
@@ -1052,6 +1052,7 @@ static struct kvm *kvm_create_vm(unsigned long type)
 	rcuwait_init(&kvm->mn_memslots_update_rcuwait);
 
 	INIT_LIST_HEAD(&kvm->devices);
+	kvm->max_vcpus = KVM_MAX_VCPUS;
 
 	BUILD_BUG_ON(KVM_MEM_SLOTS_NUM > SHRT_MAX);
 
@@ -3599,7 +3600,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, u32 id)
 		return -EINVAL;
 
 	mutex_lock(&kvm->lock);
-	if (kvm->created_vcpus == KVM_MAX_VCPUS) {
+	if (kvm->created_vcpus >= kvm->max_vcpus) {
 		mutex_unlock(&kvm->lock);
 		return -EINVAL;
 	}
Signature · 1 line
2.25.1
#115 lore raw
In reply to [email protected]
Quoted 9 lines · On 11/25/2021 8:19 AM, [email protected] wrote:

From: Isaku Yamahata <[email protected]>

Changes from v2:
- update based on patch review
- support TDP MMU
- drop non-essential fetures (ftrace etc.) to reduce patch size

TODO:
- integrate vm type patch

Hi, everyone plans to review this series,

Please skip patch 14-22, which aer old and a separate series can be found at https://lore.kernel.org/all/[email protected]/

I will post v2 with all the comments from Sean addressed.

thanks,
-Xiaoyao

Quoted 1 line

- integrate unmapping user space mapping

#9 lore raw

Re: [RFC PATCH v3 08/59] KVM: Export kvm_io_bus_read for use by TDX for PV MMIO

In reply to [email protected]

On Wed, Nov 24 2021 at 16:19, isaku yamahata wrote:

What is PV MMIO? This has nothing to do with PV=, aka paravirt because it's used in the tdx exit handler for emulation. Please stop confusing concepts.

Quoted 3 lines

From: Sean Christopherson <[email protected]>

Later kvm_intel.ko will use it.

That sentence is useless, as are the other 'later patches' phrases all over the place. At submission time it's obvious and once this is merged it's not helpful at all.

What's even worse is that this happens right at the beginning of the series and the actual use case is introduced 50 patches later. That's not how exports are done. Exports are next to the use case and in the best case they can be just part of the use case patch.

This is not how fine granular patching works. Just splitting out stupid things into separate patches does not make a proper patch series. It creates an illusion, that's it.

And if I look at the patch which makes actual use of that export then it's just the proof. I'll come to that later.

Thanks,

tglx

#11 lore raw

Re: [RFC PATCH v3 09/59] KVM: Enable hardware before doing arch VM initialization

In reply to [email protected]
Quoted 9 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Swap the order of hardware_enable_all() and kvm_arch_init_vm() to accommodate Intel's TDX, which needs VMX to be enabled during VM init in order to make SEAMCALLs.

This also provides consistent ordering between kvm_create_vm() and kvm_destroy_vm() with respect to calling kvm_arch_destroy_vm() and hardware_disable_all().

So far so good. This should also explain that there is no downside with that order change especially as this order has been that way for a long time.

Thanks,

tglx

#15 lore raw

Re: [RFC PATCH v3 12/59] KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default

In reply to [email protected]
Quoted 5 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Zap only leaf SPTEs when deleting/moving a memslot by default, and add a module param to allow reverting to the old behavior of zapping all SPTEs at all levels and memslots when any memslot is updated.

This changelog describes WHAT the change is doing, which can be seen in the patch itself, but it gives zero justification neither for the change nor for the existance of the module parameter.

Thanks,

tglx

#19 lore raw

Re: [RFC PATCH v3 14/59] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs

In reply to [email protected]
Quoted 4 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Add a capability to effectively allow userspace to query what VM types are supported by KVM.

I really don't see why this has to be named legacy. There are enough reasonable use cases which are perfectly fine using the non-encrypted muck. Just because there is a new hyped feature does not make anything else legacy.

Thanks,

tglx

#24 lore raw

Re: [RFC PATCH v3 15/59] KVM: x86: Introduce "protected guest" concept and block disallowed ioctls

In reply to [email protected]

Review

Quoted 4 lines · isaku yamahata

static int kvm_vcpu_ioctl_smi(struct kvm_vcpu *vcpu) {

(quoted diff) +1 -0
+	/* TODO: use more precise flag */

Why is this still a todo and not implemented properly from the very beginning?

And then you have this:

Quoted 2 lines
(quoted diff) +2 -0
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;

...

arch/x86/kvm/x86.c static int kvm_vcpu_ioctl_nmi(struct kvm_vcpu *vcpu) · line 4544

Quoted 3 lines
arch/x86/kvm/x86.c +3 -0
+	/* TODO: use more precise flag */
+	if (vcpu->arch.guest_state_protected)
+		return -EINVAL;

and a gazillion of other places. That's beyond lame.

The obvious place to do such a decision is kvm_arch_vcpu_ioctl(), no?

kvm_arch_vcpu_ioctl(.., unsigneg int ioctl, ...)

     if (vcpu->arch.guest_state_protected) {
     	  if (!(test_bit(_IOC_NR(ioctl), vcpu->ioctl_allowed))
          	return -EINVAL;
     }

is way too simple and obvious, right?

Even if you want more fine grained control, then having an array of flags per ioctl number is way better than sprinkling this protected muck conditions all over the place.

Thanks,

tglx

#26 lore raw

Re: [RFC PATCH v3 16/59] KVM: x86: Add per-VM flag to disable direct IRQ injection

In reply to [email protected]
Quoted 3 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Add a flag to disable IRQ injection, which is not supported by TDX.

...

arch/x86/kvm/x86.c static int kvm_vcpu_ready_for_interrupt_injection(struct kvm_vcpu *vcpu) · line 4511

Quoted 8 lines
arch/x86/kvm/x86.c +2 -1
@@ -4506,7 +4506,8 @@ static int kvm_vcpu_ready_for_interrupt_injection(struct kvm_vcpu *vcpu)
 static int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,
 				    struct kvm_interrupt *irq)
 {
-	if (irq->irq >= KVM_NR_INTERRUPTS)
+	if (irq->irq >= KVM_NR_INTERRUPTS ||
+	    vcpu->kvm->arch.irq_injection_disallowed)
 		return -EINVAL;

That's required here because you forgot to copy & pasta the protect guest condition muck into that ioctl, right?

#29 lore raw

Re: [RFC PATCH v3 17/59] KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE

In reply to [email protected]

arch/x86/kvm/x86.c static int vcpu_ioctl_tpr_access_reporting(struct kvm_vcpu *vcpu, · line 4567

Quoted 9 lines · isaku yamahata

static int kvm_vcpu_ioctl_x86_setup_mce(struct kvm_vcpu *vcpu,

arch/x86/kvm/x86.c +2 -2
 					u64 mcg_cap)
 {
-	int r;
 	unsigned bank_num = mcg_cap & 0xff, bank;
 
-	r = -EINVAL;
+	if (vcpu->kvm->arch.mce_injection_disallowed)
+		return -EINVAL;

Yet another flag because the copy & pasta orgy did not cover that ioctl. What for?

#31 lore raw

Re: [RFC PATCH v3 18/59] KVM: x86: Add flag to mark TSC as immutable (for TDX)

In reply to [email protected]
Quoted 3 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

The TSC for TDX1 guests is fixed at TD creation time. Add tsc_immutable

What's a TDX1 guest?

Quoted 3 lines

to reflect that the TSC of the guest cannot be changed in any way, and use it to short circuit all paths that lead to one of the myriad TSC adjustment flows.

I can kinda see the reason for this being valuable on it's own, but in general why does TDX need a gazillion flags to disable tons of different things if _ALL_ these flags are going to be set by for TDX guests anyway?

Seperate flags make only sense when they have a value on their own, i.e. are useful for things outside of TDX. If not they are just useless ballast.

Thanks,

tglx

#38 lore raw

Re: [RFC PATCH v3 23/59] KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID

In reply to [email protected]
Quoted 6 lines · isaku yamahata

Let userspace, or in the case of TDX, KVM itself, enable X2APIC even if X2APIC is not reported as supported in the guest's CPU model. KVM generally does not force specific ordering between ioctls(), e.g. this forces userspace to configure CPUID before MSRs. And for TDX, vCPUs will always run with X2APIC enabled, e.g. KVM will want/need to enable X2APIC from time zero.

This is complete crap. Fix the broken user space and do not add horrible hacks to the kernel.

Thanks,

tglx

#45 lore raw

Re: [RFC PATCH v3 26/59] KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()

In reply to [email protected]
Quoted 8 lines · isaku yamahata

Add a second kvm_x86_ops hook in kvm_arch_vm_destroy() to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.

Note, this knowingly creates a discrepancy in nomenclature for SVM as svm_vm_teardown() invokes avic_vm_destroy() and sev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.

That's just the wrong way around. Fixup SVM first and then add the TDX muck on top. Stop this 'left to a future patch' nonsense. I know for sure that those future patches never materialize.

The way it works is:

1) Refactor the code to make room for your new functionality in a way that the existing code still works.

2) Add your new muck on top.

Anything else is not acceptable at all.

Thanks,

tglx

#54 lore raw

Re: [RFC PATCH v3 29/59] KVM: x86: Add option to force LAPIC expiration wait

In reply to [email protected]
Quoted 1 line · isaku yamahata

Add an option to skip the IRR check in kvm_wait_lapic_expire().

Yay for consistency. $Subject says 'Add option to force wait'. Changelog says 'Add option to skip ... check'. Can you make your mind up?

Also the change at hand is not adding an option. It's adding a function argument. You surely can spot the difference between option and function argument, right?

Changelogs are meant to be readable by mere mortals.

Thanks,

tglx

#56 lore raw

Re: [RFC PATCH v3 30/59] KVM: x86: Add guest_supported_xss placholder

In reply to [email protected]
Quoted 7 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Add a per-vcpu placeholder for the support XSS of the guest so that the TDX configuration code doesn't need to hack in manual computation of the supported XSS. KVM XSS enabling is currently being upstreamed, i.e. guest_supported_xss will no longer be a placeholder by the time TDX is ready for upstreaming (hopefully).

Yes, hope dies last... Definitely technical useful information for a changelog. There is a reason why notes should go below the --- separator line.

Thanks,

tglx

#58 lore raw

Re: [RFC PATCH v3 31/59] KVM: x86: Add infrastructure for stolen GPA bits

In reply to [email protected]
Quoted 41 lines · isaku yamahata

Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA alising will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.

Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.

With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.

To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.

To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.

GFNs (without stolen bits) continue to be used to:
-Specify physical memory by userspace via memslots -Map GPAs to TDP PTEs via RMAP
-Specify dirty tracking and write protection -Look up MTRR types
-Inject async page faults

Since there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN, kvm_handle_hva() based operations will be applied to both aliases.

In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.

Quoted 6 lines

For TLB flushes that are address based, make sure to flush both aliases in the stolen bits case.

Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.

This is a real reasonable and informative changelog. Thanks to Rick for writing this up!

Thanks,

tglx

#67 lore raw

Re: [RFC PATCH v3 39/59] KVM: VMX: Modify NMI and INTR handlers to take intr_info as param

In reply to [email protected]
Quoted 1 line · isaku yamahata

From: Sean Christopherson <[email protected]>

$subject: s/as param/as function arguments/

    param != function argument

It's not rocket science to use the proper terms and to write out words in changelogs and comments instead of using half baken abbreviations.

Precise language matters and especially so for the benefit on non-native speakers and people who are not familiar with a particular corporate slang.

Thanks,

tglx

#78 lore raw

Re: [RFC PATCH v3 46/59] KVM: VMX: Move register caching logic to common code

In reply to [email protected]
Quoted 10 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Move the guts of vmx_cache_reg() to vt_cache_reg() in preparation for reusing the bulk of the code for TDX, which can access guest state for debug TDs.

Use kvm_x86_ops.cache_reg() in ept_update_paging_mode_cr0() rather than trying to expose vt_cache_reg() to vmx.c, even though it means taking a retpoline. The code runs if and only if EPT is enabled but unrestricted guest.

This sentence does not parse because it's not a proper sentence.

Quoted 3 lines

Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.

This one is only significantly better and lacks an explanation what this means for the dumb case.

Thanks,

tglx

maintainer
#79 lore raw

Re: [RFC PATCH v3 46/59] KVM: VMX: Move register caching logic to common code

Quoted 5 lines · Thomas Gleixner
Quoted 5 lines

Use kvm_x86_ops.cache_reg() in ept_update_paging_mode_cr0() rather than trying to expose vt_cache_reg() to vmx.c, even though it means taking a retpoline. The code runs if and only if EPT is enabled but unrestricted guest.

Quoted 7 lines

This sentence does not parse because it's not a proper sentence.

Quoted 3 lines

Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.

This one is only significantly better and lacks an explanation what this means for the dumb case.

Well, it means a retpoline (see paragraph before). However, I'm not sure why it one wouldn't create a vt.h header with all vt_* functions.

Paolo

#83 lore raw

Re: [RFC PATCH v3 47/59] KVM: TDX: Define TDCALL exit reason

In reply to [email protected]
Quoted 5 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Define the TDCALL exit reason, which is carved out from the VMX exit reason namespace as the TDCALL exit from TDX guest to TDX-SEAM is really just a VM-Exit.

How is this carved out? What's the value of this word salad?

It's simply a new exit reason. Not more, not less. So what?

Quoted 3 lines

I'm pretty sure that it does not take two engineers to add a new exit reason define, but it takes at least two engineers to come up with a convoluted explanation for it.

Thanks,

tglx

#87 lore raw

Re: [RFC PATCH v3 49/59] KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs

In reply to [email protected]
Quoted 4 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Add a macro framework to hide VMX vs. TDX details of VMREAD and VMWRITE so the VMX and TDX can shared common flows, e.g. accessing DTs.

s/shared/share/

Quoted 3 lines

Note, the TDX paths are dead code at this time. There is no great way to deal with the chicken-and-egg scenario of having things in place for TDX without first having TDX.

That's more than obvious and the whole point of building infrastructure in the first place, isn't it?

arch/x86/kvm/vmx/common.h line 27

Quoted 12 lines
arch/x86/kvm/vmx/common.h +12 -0
+#ifdef CONFIG_INTEL_TDX_HOST
+#define VT_BUILD_VMCS_HELPERS(type, bits, tdbits)			   \
+static __always_inline type vmread##bits(struct kvm_vcpu *vcpu,		   \
+					 unsigned long field)		   \
+{									   \
+	if (unlikely(is_td_vcpu(vcpu))) {				   \
+		if (KVM_BUG_ON(!is_debug_td(vcpu), vcpu->kvm))		   \
+			return 0;					   \
+		return td_vmcs_read##tdbits(to_tdx(vcpu), field);	   \
+	}								   \
+	return vmcs_read##bits(field);					   \
+}									   \

New lines exist for a reason to visually separate things and are even possible in macro blocks. This includes the defines.

Aside of that is there any reason why the end of the macro block has to be 3 spaces instead of a tab?

Thanks,

tglx

#94 lore raw

Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

In reply to [email protected]
Quoted 3 lines · isaku yamahata

From: Chao Gao <[email protected]>

$Subject: KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

Which user space are you talking about? This subject line is misleading at best. The unconditional reset is happening when a TDX VM exits because the SEAM firmware enforces this to prevent unformation leaks.

It also does not matter whether this are four or ten MSR. Fact is that the SEAM firmware is buggy because it does not save/restore those MSRs.

So the proper subject line is:

KVM: x86: Add infrastructure to handle MSR corruption by broken TDX firmware

Quoted 10 lines

The TDX module unconditionally reset 4 host MSRs (MSR_SYSCALL_MASK, MSR_START, MSR_LSTAR, MSR_TSC_AUX) to architectural INIT state on exit from TDX VM to KVM. KVM needs to save their values before TD enter and restore them on exit to userspace.

Reuse current kvm_user_return mechanism and introduce a function to update cached values and register the user return notifier in this new function.

The later patch will use the helper function to save/restore 4 host MSRs.

'The later patch ...' is useless information. Of course there will be a later patch to make use of this which is implied by 'Add infrastructure ...'. Can we please get rid of these useless phrases which have no value at patch submission time and are even more confusing once the pile is merged?

Thanks,

tglx

maintainer
#103 lore raw

Re: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

In reply to [email protected]
Quoted 10 lines · On 11/25/21 01:20, [email protected] wrote:

From: Xiaoyao Li<[email protected]>

Introduce a per-vm variable initial_tsc_khz to hold the default tsc_khz for kvm_arch_vcpu_create().

This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.

Probably some incorrect squashing happened here, since...

Quoted 5 lines
 arch/x86/include/asm/kvm_host.h |    1 +
 arch/x86/kvm/vmx/tdx.c          | 2233 +++++++++++++++++++++++++++++++
 arch/x86/kvm/x86.c              |    3 +-
 3 files changed, 2236 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kvm/vmx/tdx.c

This patch does a bit more than that. :)

Paolo

maintainer
#51 lore raw

Re: [RFC PATCH v3 28/59] KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()

In reply to [email protected]
Quoted 8 lines · On 11/25/21 01:20, [email protected] wrote:

From: Sean Christopherson<[email protected]>

Return true for kvm_vcpu_has_events() if the vCPU has a pending APICv interrupt to support TDX's usage of APICv. Unlike VMX, TDX doesn't have access to vmcs.GUEST_INTR_STATUS and so can't emulate posted interrupts, i.e. needs to generate a posted interrupt and more importantly can't manually move requested interrupts into the vIRR (which it also doesn't have access to).

Does this mean it is impossible to disable APICv on TDX? If so, please add a WARN.

Paolo

maintainer
#46 lore raw

Re: [RFC PATCH v3 26/59] KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()

In reply to Thomas Gleixner
Quoted 5 lines · Thomas Gleixner
Quoted 4 lines · isaku yamahata

Add a second kvm_x86_ops hook in kvm_arch_vm_destroy() to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.

Quoted 7 lines
Quoted 4 lines

Note, this knowingly creates a discrepancy in nomenclature for SVM as svm_vm_teardown() invokes avic_vm_destroy() and sev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.

That's just the wrong way around. Fixup SVM first and then add the TDX muck on top. Stop this 'left to a future patch' nonsense. I know for sure that those future patches never materialize.

Or just keep vm_destroy for the "early" destruction, and give a new name to the new hook. It is used to give back the TDCS memory, so perhaps you can call it vm_free?

Paolo

#104 lore raw

Re: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

In reply to [email protected]
Quoted 7 lines · isaku yamahata

From: Xiaoyao Li <[email protected]>

Introduce a per-vm variable initial_tsc_khz to hold the default tsc_khz for kvm_arch_vcpu_create().

This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.

So now almost 50 patches after exporting kvm_io_bus_read() I have finally reached the place which makes use of it. But that's just a minor detail compared to this:

Quoted 2 lines
 arch/x86/include/asm/kvm_host.h |    1 +
 arch/x86/kvm/vmx/tdx.c          | 2233 +++++++++++++++++++++++++++++++

Can you pretty please explain how this massive pile of code is related to the subject line and the change log of this patch?

It takes more than 2000 lines of code to add a new member to struct kvm_arch?

Seriously? This definitely earns an award for the most disconnected changelog ever.

arch/x86/kvm/vmx/tdx.c

Quoted 19 lines
 arch/x86/kvm/x86.c              |    3 +-
 3 files changed, 2236 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kvm/vmx/tdx.c
arch/x86/include/asm/kvm_host.h +1 -0
index 17d6e4bcf84b..f10c7c2830e5 100644
@@ -1111,6 +1111,7 @@ struct kvm_arch {
 	u64 last_tsc_write;
 	u32 last_tsc_khz;
 	u64 last_tsc_offset;
+	u32 initial_tsc_khz;
 	u64 cur_tsc_nsec;
 	u64 cur_tsc_write;
 	u64 cur_tsc_offset;
arch/x86/kvm/vmx/tdx.c new file +0 -0
new file mode 100644
index 000000000000..64b2841064c4

< SNIP>

Yes, the above would be the patch which would be expected according to the changelog and the subject line.

And no. Throwing a 2000+ lines patch over the fence which builds up the complete infrastructure for TDX in KVM is not how that works. This patch (aside of the silly changelog) is an unreviewable maze.

So far in this series, there was a continuous build up of infrastructure in reviewable chunks and then you expect a reviewer to digest 2000+ lines at once for no reason and with the most disconneted changelog ever?

This can be done structured and gradually, really.

1) Add the basic infrastructure

2) Add functionality piece by piece

There is no fundamental dependency up to the point where you enable TDX, but there is a fundamental difference of reviewing 2000+ lines of code at once or reviewing a gradual build up of 2000+ lines of code in small pieces with proper changelogs for each of them.

You can argue that my request is unreasonable until you are blue in your face, it's not going to lift my NAK on this.

I've mopped up enough half baken crap in x86/kvm over the years and I have absolutely no interest at all to mop up after you again.

x86/kvm is not a special part of the kernel and neither exempt from general kernel process nor from x86 specific scrunity rules.

That said, I'm stopping the review right here simply because looking at any further changes does not make any sense at all.

Thanks,

tglx

#47 lore raw

Re: [RFC PATCH v3 26/59] KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()

In reply to Paolo Bonzini
Quoted 10 lines · Paolo Bonzini
Quoted 9 lines · Thomas Gleixner
Quoted 8 lines · isaku yamahata

Add a second kvm_x86_ops hook in kvm_arch_vm_destroy() to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.

Note, this knowingly creates a discrepancy in nomenclature for SVM as svm_vm_teardown() invokes avic_vm_destroy() and sev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.

Quoted 7 lines
Quoted 3 lines

That's just the wrong way around. Fixup SVM first and then add the TDX muck on top. Stop this 'left to a future patch' nonsense. I know for sure that those future patches never materialize.

Or just keep vm_destroy for the "early" destruction, and give a new name to the new hook. It is used to give back the TDCS memory, so perhaps you can call it vm_free?

Up to you, but the current approach is bogus. I rather go for a fully symmetric interface and let the various incarnations opt in at the right place. Similar to what cpu hotplug states are implementing.

Thanks,

tglx

maintainer
#105 lore raw

Re: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

In reply to Thomas Gleixner
Quoted 2 lines · Thomas Gleixner

You can argue that my request is unreasonable until you are blue in your face, it's not going to lift my NAK on this.

There's no need for that. I'd be saying the same, and I don't think it's particularly helpful that you made it almost a personal issue.

While in this series there is a separation of changes to existing code vs. new code, what's not clear is _why_ you have all those changes. These are not code cleanups or refactorings that can stand on their own feet; lots of the early patches are actually part of the new functionality. And being in the form of "add an argument here" or "export a function there", it's not really easy (or feasible) to review them without seeing how the new functionality is used, which requires a constant back and forth between early patches and the final 2000 line file.

In some sense, the poor commit messages at the beginning of the series are just a symptom of not having any meat until too late, and then dropping it all at once. There's only so much that you can say about an EXPORT_SYMBOL_GPL, the real thing to talk about is probably the thing that refers to that symbol.

If there are some patches that are actually independent, go ahead and submit them early. But more practically, for the bulk of the changes what you need to do is:

1) incorporate into patch 55 a version of tdx.c that essentially does KVM_BUG_ON or WARN_ON for each function. Temporarily keep the same huge patch that adds the remaining 2000 lines of tdx.c

2) squash the tdx.c stub with patch 44.

3) gather a strace of QEMU starting up a TDX domain.

4) figure out which parts of the code are needed to run until the first ioctl. Make that a first patch.

5) repeat step 4 until you have covered all the code

5) Move the new "KVM: VMX: Add 'main.c' to wrap VMX and TDX" (which also adds the tdx.c stub) as possible in the series.

6) Move each of the new patches as early as possible in the series.

7) Look for candidates for squashing (e.g. commit messages that say it's "used later"; now the use should be very close and the two can be merged). Add to the commit message a note about changes outside VMX.

The resulting series may not be perfect, but it would be a much better starting point for review.

Paolo

#106 lore raw

Re: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

Paolo,

Quoted 6 lines · Paolo Bonzini
Quoted 2 lines · Thomas Gleixner

You can argue that my request is unreasonable until you are blue in your face, it's not going to lift my NAK on this.

There's no need for that. I'd be saying the same, and I don't think it's particularly helpful that you made it almost a personal issue.

To be entirely clear. It's a personal issue.

Being on the receiving end of a firehose which spills liquid manure for more than 15 years is not a pleasant experience at all.

Being accused to have unreasonable, unactionable and even bizarre review requests (mostly behind the scenes) is not helping either.

I'm dead tired of sitting down for hours explaining basics in lengthy emails over and over just to get yet another experience of being ignored after a couple of months filled with silence.

While I did not do that in this particular context, I read back on the previous incarnations of this crud and found out that you are in the exact same situation.

So what's the difference? Nothing at all.

It's about time that someone speaks up in unambiguous words to make it entirely clear that this is not acceptable at all in the kernel community as a whole and not just because a single person being unreasonable or whatever.

Both of us wasted enough time already to explain how to do it right. There is no point anymore to proliferate any of these political correctness games further.

Thanks,

Thomas

#107 lore raw

Re: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

In reply to Paolo Bonzini

Paolo,

Quoted 8 lines · Paolo Bonzini

On 11/25/21 22:05, Thomas Gleixner wrote:
If there are some patches that are actually independent, go ahead and submit them early. But more practically, for the bulk of the changes what you need to do is:

1) incorporate into patch 55 a version of tdx.c that essentially does KVM_BUG_ON or WARN_ON for each function. Temporarily keep the same huge patch that adds the remaining 2000 lines of tdx.c

There is absolutely no reason to populate anything upfront at all.

Why?

Simply because that whole muck cannot work until all pieces are in place.

That's the classic PoC vs. usable code situation. I.e. you know what needs to be done from a PoC point of view and you have to get there by building up the infrastructure piece by piece.

So why would you provide:

handle_A(...) { BUG(); }
..
handle_Z(...) { BUG(); }

with all the invocations in the common emulation dispatcher:

  handle_stuff(reason)
  {
        switch(reason) {
        case A: return handle_A();
        ...
        case Z: return handle_Z();
        default: BUG();
        }
  };

in the first place instead of providing:

1)

        handle_stuff(reason)
        {
                switch(reason) {
                default: BUG();
                };
        }

2)

        handle_A(...)
        {
                return do_something_understandable()
        }
        
        handle_stuff(reason)
        {
                switch(reason) {
                case A: return handle_A();
                default: BUG();
        }

$Z)

        handle_Z(...)
        {
                return do_something_understandable()
        }
        
        handle_stuff(reason)
        {
                switch(reason) {
                case A: return handle_A();
                ...
                case Z: return handle_Z();
                default: BUG();
        }

where each of (1..$Z) is a single patch doing exactly _ONE_ thing at a time and if there is some extra magic required like an additional export for handle_Y() then this export patch is either part of the patch dealing with $Y or just before that.

In both scenarious you cannot boot a TDX guest until you reached $Z, but in the gradual one you and the reviewers have the pleasure of looking at one thing at a time.

No?

Thanks,

tglx

maintainer
#108 lore raw

Re: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

Quoted 8 lines · Thomas Gleixner

Paolo,

Quoted 5 lines · Paolo Bonzini

On 11/25/21 22:05, Thomas Gleixner wrote:
If there are some patches that are actually independent, go ahead and submit them early. But more practically, for the bulk of the changes what you need to do is:

Quoted 8 lines
Quoted 3 lines

1) incorporate into patch 55 a version of tdx.c that essentially does KVM_BUG_ON or WARN_ON for each function. Temporarily keep the same huge patch that adds the remaining 2000 lines of tdx.c

There is absolutely no reason to populate anything upfront at all. Why?

Simply because that whole muck cannot work until all pieces are in place.

It can, sort of. It cannot run a complete guest, but it could in principle run a toy guest with a custom userspace, like the ones that make up tools/testing/selftests/kvm. (Note that KVM_BUG_ON marks the VM as bugged but doesn't hang the whole machine).

AMD was working on infrastructure to do this for SEV and SEV-ES.

Quoted 8 lines

So why would you provide:

handle_A(...) { BUG(); }
..
handle_Z(...) { BUG(); }

with all the invocations in the common emulation dispatcher:

Quoted 9 lines
   handle_stuff(reason)
   {
         switch(reason) {
         case A: return handle_A();
         ...
         case Z: return handle_Z();
         default: BUG();
         }
   };

If it's a switch statement that's good, but the common case is more similar to this:

vmx_handle_A(...) { ... }
+tdx_handle_A(...) { ... }
+
+vt_handle_A(...) {
+ if (is_tdx(vcpu->kvm))
+ tdx_handle_A(...);
+ else
+ vmx_handle_A(...);
+}

...

- .handle_A = vmx_handle_A,
+ .handle_A = vt_handle_A,

And you could indeed do it in a single patch, without adding the stub tdx_handle_A upfront. But you would have code that is broken and who knows what the effects would be of calling vmx_handle_A on a TDX virtual machine. It could be an error, or it could be memory corruption.

Quoted 3 lines

In both scenarious you cannot boot a TDX guest until you reached $Z, but in the gradual one you and the reviewers have the pleasure of looking at one thing at a time.

I think both of them are gradual. Not having the stubs might be a little more gradual, but it is a very minor issue for the reviewability of the whole thing.

Paolo

maintainer
#39 lore raw

Re: [RFC PATCH v3 23/59] KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID

In reply to Thomas Gleixner
Quoted 10 lines · Thomas Gleixner
Quoted 6 lines · isaku yamahata

Let userspace, or in the case of TDX, KVM itself, enable X2APIC even if X2APIC is not reported as supported in the guest's CPU model. KVM generally does not force specific ordering between ioctls(), e.g. this forces userspace to configure CPUID before MSRs. And for TDX, vCPUs will always run with X2APIC enabled, e.g. KVM will want/need to enable X2APIC from time zero.

This is complete crap. Fix the broken user space and do not add horrible hacks to the kernel.

tl;dr: I agree that it's a userspace issue but "configure CPUID before MSR" is not the issue (in fact QEMU calls KVM_SET_CPUID2 before any call to KVM_SET_MSRS).

We have quite a few other cases in which KVM_GET/SET_MSR is allowed to get/set MSRs in ways that the guests are not allowed to do.

In general, there are several reasons for this:

- simplifying userspace so that it can use the same list of MSRs for all guests (likely, the list that KVM provides with KVM_GET_MSR_INDEX_LIST). For example MSR_TSC_AUX is only exposed to the guest if RDTSCP or
RDPID are available, but the host can always access it. This is usually the reason why host accesses to MSRs override CPUID.

- simplifying userspace so that it does not have to go through the various steps of a state machine; for example, it's okay if userspace goes DISABLED->X2APIC instead of having to do DISABLED->XAPIC->X2APIC.

- allowing userspace to set a reset value, for example overriding the lock bit in MSR_IA32_FEAT_CTL.

- read-only MSRs that are really "CPUID-like", i.e. they give the guest information about processor features (for example the VMX feature MSRs)

- MSRs had some weird limitations that were lifted later by introducing additional MSRs; for example KVM always allows the host to write to the full-width MSR_IA32_PMC0 counters, because they are a saner version of MSR_IA32_PERFCTR0 and there's no reason for userspace to inflict MSR_IA32_PERFCTR0 on userspace.

So the host_initiated check doesn't _necessarily_ count as a horrible hack in the kernel. However, in this case we have a trusted domain without X2APIC. I'm not sure this configuration is clearly bogus. One could imagine special-purpose VMs that don't need interrupts at all. For full guests such as the ones that QEMU runs, I agree with Thomas that userspace must be fixed to enforce x2apic for TDX guests.

Paolo

#95 lore raw

Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

In reply to Thomas Gleixner
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines · isaku yamahata

From: Chao Gao <[email protected]>

$Subject: KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

Which user space are you talking about? This subject line is misleading

Host Ring3.

Quoted 2 lines

at best. The unconditional reset is happening when a TDX VM exits because the SEAM firmware enforces this to prevent unformation leaks.

Yes.

Quoted 2 lines

It also does not matter whether this are four or ten MSR.

Indeed, the number of MSRs doesn't matter.

Quoted 2 lines

Fact is that
the SEAM firmware is buggy because it does not save/restore those MSRs.

It is done deliberately. It gives host a chance to do "lazy" restoration. "lazy" means don't save/restore them on each TD entry/exit but defer restoration to when it is neccesary e.g., when vCPU is scheduled out or when kernel is about to return to Ring3.

Quoted 4 lines

So the proper subject line is:

KVM: x86: Add infrastructure to handle MSR corruption by broken TDX firmware

I rewrote the commit message:

KVM: x86: Allow to update cached values in kvm_user_return_msrs w/o wrmsr

Several MSRs are constant and only used in userspace. But VMs may have different values. KVM uses kvm_set_user_return_msr() to switch to guest's values and leverages user return notifier to restore them when kernel is to return to userspace. In order to save unnecessary wrmsr, KVM also caches the value it wrote to a MSR last time.

TDX module unconditionally resets some of these MSRs to architectural INIT state on TD exit. It makes the cached values in kvm_user_return_msrs are inconsistent with values in hardware. This inconsistency needs to be fixed otherwise, it may mislead kvm_on_user_return() to skip restoring some MSRs to host's values. kvm_set_user_return_msr() can help to correct this case but it is not optimal as it always does a wrmsr. So, introduce a variation of kvm_set_user_return_msr() to update the cached value but skip the wrmsr.

Quoted 9 lines
Quoted 9 lines

The TDX module unconditionally reset 4 host MSRs (MSR_SYSCALL_MASK, MSR_START, MSR_LSTAR, MSR_TSC_AUX) to architectural INIT state on exit from TDX VM to KVM. KVM needs to save their values before TD enter and restore them on exit to userspace.

Reuse current kvm_user_return mechanism and introduce a function to update cached values and register the user return notifier in this new function.

Quoted 8 lines
Quoted 2 lines

The later patch will use the helper function to save/restore 4 host MSRs.

'The later patch ...' is useless information. Of course there will be a later patch to make use of this which is implied by 'Add infrastructure ...'. Can we please get rid of these useless phrases which have no value at patch submission time and are even more confusing once the pile is merged?

Of course. Will remove all "later patch" phrases.

Thanks
Chao

Quoted 4 lines

Thanks,

tglx

maintainer
#96 lore raw

Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

Quoted 1 line · Chao Gao

Of course. Will remove all "later patch" phrases.

Chao, see my comment on patch 54, for how to do this. In this case, this patch would be right before the one that enables KVM_RUN on a TDX module.

Paolo

#27 lore raw

Re: [RFC PATCH v3 16/59] KVM: x86: Add per-VM flag to disable direct IRQ injection

In reply to [email protected]
Quoted 6 lines · On Fri, Nov 26, 2021 at 3:44 PM <[email protected]> wrote:

static int dm_request_for_irq_injection(struct kvm_vcpu *vcpu)

 {
        return vcpu->run->request_interrupt_window &&
+              !vcpu->kvm->arch.irq_injection_disallowed &&
                likely(!pic_in_kernel(vcpu->kvm));

}

Just judged superficially by the function names, it seems that the logic is better to be put in kvm_cpu_accept_dm_intr() or some deeper function nested in kvm_cpu_accept_dm_intr().

The function name will tell us that the interrupt is not injected because the CPU doesn't accept it. And it will also have an effect that vcpu->run->ready_for_interrupt_injection will always be false which I think is better to have for TDX.

Quoted 4 lines

--
2.25.1

#97 lore raw

Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

In reply to Chao Gao
Quoted 24 lines · Chao Gao
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines · isaku yamahata

From: Chao Gao <[email protected]>

$Subject: KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

Which user space are you talking about? This subject line is misleading

Host Ring3.

Quoted 2 lines

at best. The unconditional reset is happening when a TDX VM exits because the SEAM firmware enforces this to prevent unformation leaks.

Yes.

Quoted 2 lines

It also does not matter whether this are four or ten MSR.

Indeed, the number of MSRs doesn't matter.

Quoted 2 lines

Fact is that
the SEAM firmware is buggy because it does not save/restore those MSRs.

Quoted 8 lines

It is done deliberately. It gives host a chance to do "lazy" restoration. "lazy" means don't save/restore them on each TD entry/exit but defer restoration to when it is neccesary e.g., when vCPU is scheduled out or when kernel is about to return to Ring3.

The TDX module unconditionally reset 4 host MSRs (MSR_SYSCALL_MASK, MSR_START, MSR_LSTAR, MSR_TSC_AUX) to architectural INIT state on exit from TDX VM to KVM.

I did not find the information in intel-tdx-module-1eas.pdf nor intel-tdx-cpu-architectural-specification.pdf.

Maybe the version I downloaded is outdated.

I guess that the "lazy" restoration mode is not a valid optimization. The SEAM module should restore it to the original value when it tries to reset it to architectural INIT state on exit from TDX VM to KVM since the SEAM module also does it via wrmsr (correct me if not).

If the SEAM module doesn't know "the original value" of the these MSRs, it would be mere an optimization to save an rdmsr in SEAM. But there are a lot of other ways for the host to share the values to SEAM in zero overhead.

Could you provide more information?

#98 lore raw

Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

Quoted 30 lines · Lai Jiangshan
Quoted 29 lines · Chao Gao
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines · isaku yamahata

From: Chao Gao <[email protected]>

$Subject: KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

Which user space are you talking about? This subject line is misleading

Host Ring3.

Quoted 2 lines

at best. The unconditional reset is happening when a TDX VM exits because the SEAM firmware enforces this to prevent unformation leaks.

Yes.

Quoted 2 lines

It also does not matter whether this are four or ten MSR.

Indeed, the number of MSRs doesn't matter.

Quoted 2 lines

Fact is that
the SEAM firmware is buggy because it does not save/restore those MSRs.

It is done deliberately. It gives host a chance to do "lazy" restoration. "lazy" means don't save/restore them on each TD entry/exit but defer restoration to when it is neccesary e.g., when vCPU is scheduled out or when kernel is about to return to Ring3.

Quoted 8 lines
Quoted 3 lines

The TDX module unconditionally reset 4 host MSRs (MSR_SYSCALL_MASK, MSR_START, MSR_LSTAR, MSR_TSC_AUX) to architectural INIT state on exit from TDX VM to KVM.

I did not find the information in intel-tdx-module-1eas.pdf nor intel-tdx-cpu-architectural-specification.pdf.

Maybe the version I downloaded is outdated.

Hi Jiangshan,

Please refer to Table 22.162 MSRs that may be Modified by TDH.VP.ENTER, in section 22.2.40 TDH.VP.ENTER leaf.

Quoted 5 lines

I guess that the "lazy" restoration mode is not a valid optimization. The SEAM module should restore it to the original value when it tries to reset it to architectural INIT state on exit from TDX VM to KVM since the SEAM module also does it via wrmsr (correct me if not).

Correct.

Quoted 3 lines

If the SEAM module doesn't know "the original value" of the these MSRs, it would be mere an optimization to save an rdmsr in SEAM.

Yes. Just a rdmsr is saved in TDX module at the cost of host's restoring a MSR. If restoration (wrmsr) can be done in a lazy fashion or even the MSR isn't used by host, some CPU cycles can be saved.

Quoted 2 lines

But there are a lot of other ways for the host to share the values to SEAM in zero overhead.

I am not sure. Looks it requests a new interface between host and TDX module. I guess one problem is how/when to verify host's inputs in case they are invalid.

Thanks
Chao

Quoted 2 lines

Could you provide more information?

#80 lore raw

Re: [RFC PATCH v3 46/59] KVM: VMX: Move register caching logic to common code

In reply to Paolo Bonzini
Quoted 7 lines · Paolo Bonzini
Quoted 6 lines · Thomas Gleixner
Quoted 5 lines

Use kvm_x86_ops.cache_reg() in ept_update_paging_mode_cr0() rather than trying to expose vt_cache_reg() to vmx.c, even though it means taking a retpoline. The code runs if and only if EPT is enabled but unrestricted guest.

This sentence does not parse because it's not a proper sentence.

Heh, supposed to be "... but unrestricted guest is disabled".

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.

This one is only significantly better and lacks an explanation what this means for the dumb case.

Well, it means a retpoline (see paragraph before).

No, the point being made is that, on a CPU that supports Unrestricted Guest (UG), disabling UG without disabling EPT is really, really stupid. UG requires EPT, so disabling EPT _and_ UG is reasonable as there are scenarios where using shadow paging is desirable. But inentionally disabling UG and enabling EPT makes no sense. It forces KVM to emulate non-trivial amounts of guest code and has zero benefits for anything other than testing KVM itself.

Quoted 4 lines

why it one wouldn't create a vt.h header with all vt_* functions.

Paolo

#40 lore raw

Re: [RFC PATCH v3 23/59] KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID

In reply to Paolo Bonzini
Quoted 3 lines · Paolo Bonzini
Quoted 2 lines · Thomas Gleixner
Quoted 1 line · isaku yamahata

Let userspace, or in the case of TDX, KVM itself, enable X2APIC even if

                                            ^^^^^^^^^^
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

X2APIC is not reported as supported in the guest's CPU model. KVM generally does not force specific ordering between ioctls(), e.g. this forces userspace to configure CPUID before MSRs. And for TDX, vCPUs will always run with X2APIC enabled, e.g. KVM will want/need to enable

                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line
Quoted 1 line
Quoted 1 line

X2APIC from time zero.

      ^^^^^^^^^^^^^^^^^^^^^
Quoted 7 lines
Quoted 3 lines

This is complete crap. Fix the broken user space and do not add horrible hacks to the kernel.

tl;dr: I agree that it's a userspace issue but "configure CPUID before MSR" is not the issue (in fact QEMU calls KVM_SET_CPUID2 before any call to KVM_SET_MSRS).

Specifically for TDX, it's not a userspace issue. To simplify other checks and to report sane values for KVM_GET_MSRS, KVM forces X2APIC for TDX guests when the vCPU is created, before its exposed to usersepace. The bit about not forcing specific ordering is justification for making the change independent of TDX, i.e. to call out that APIC_BASE is different from every other MSR, and is even inconsistent in its own behavior since illegal transitions are allowed when userspace is stuffing the MSR.

IMO, this patch is valid irrespective of TDX. It's included in the TDX series because TDX support forces the issue.

That said, an alternative for TDX would be do handle this in kvm_lapic_reset() now that the lAPIC RESET flows are consolidated. Back when this patch was first written, that wasn't really an option.

#48 lore raw

Re: [RFC PATCH v3 26/59] KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()

In reply to Thomas Gleixner
Quoted 15 lines · Thomas Gleixner
Quoted 14 lines · Paolo Bonzini
Quoted 12 lines · Thomas Gleixner
Quoted 8 lines · isaku yamahata

Add a second kvm_x86_ops hook in kvm_arch_vm_destroy() to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.

Note, this knowingly creates a discrepancy in nomenclature for SVM as svm_vm_teardown() invokes avic_vm_destroy() and sev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.

That's just the wrong way around. Fixup SVM first and then add the TDX muck on top. Stop this 'left to a future patch' nonsense. I know for sure that those future patches never materialize.

Quoted 7 lines
Quoted 3 lines

Or just keep vm_destroy for the "early" destruction, and give a new name to the new hook. It is used to give back the TDCS memory, so perhaps you can call it vm_free?

Up to you, but the current approach is bogus. I rather go for a fully symmetric interface and let the various incarnations opt in at the right place. Similar to what cpu hotplug states are implementing.

Naming aside, that's what is being done, TDX simply needs two hooks instead of one due to the way KVM handles VM and vCPU destruction. The alternative would be to shove and duplicate what is currently common x86 code into VMX/SVM, which IMO is far worse.

Regarding the naming, I 100% agree SVM should be refactored prior to adding TDX stuff if we choose to go with vm_teardown() and vm_destroy() instead of Paolo's suggestion of vm_destroy() and vm_free(). When this patch/code was originally written, letting SVM become stale was a deliberate choice to reduce conflicts with upstream as we knew the code would live out of tree for quite some time. But that was purely meant to be development "hack", not upstream behavior.

#52 lore raw

Re: [RFC PATCH v3 28/59] KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()

In reply to Paolo Bonzini
Quoted 10 lines · Paolo Bonzini
Quoted 8 lines · On 11/25/21 01:20, [email protected] wrote:

From: Sean Christopherson<[email protected]>

Return true for kvm_vcpu_has_events() if the vCPU has a pending APICv interrupt to support TDX's usage of APICv. Unlike VMX, TDX doesn't have access to vmcs.GUEST_INTR_STATUS and so can't emulate posted interrupts, i.e. needs to generate a posted interrupt and more importantly can't manually move requested interrupts into the vIRR (which it also doesn't have access to).

Quoted 2 lines

Does this mean it is impossible to disable APICv on TDX? If so, please add a WARN.

Yes, APICv is forced.

Rereading this patch, checking only for a pending posted interrupt isn't correct, a pending interrupt that's below the PPR shouldn't be considered a wake event.

A much better approach would be to have vt_sync_pir_to_irr() redirect to a TDX implementation to read the PIR but not update the vIRR, that way common x86 doesn't need to be touched. Hopefully that can be done in a race-free way.

maintainer
#81 lore raw

Re: [RFC PATCH v3 46/59] KVM: VMX: Move register caching logic to common code

In reply to Sean Christopherson
Quoted 9 lines · Sean Christopherson
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines

Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.

This one is only significantly better and lacks an explanation what this means for the dumb case.

Well, it means a retpoline (see paragraph before).

No, the point being made is that, on a CPU that supports Unrestricted Guest (UG), disabling UG without disabling EPT is really, really stupid.

Yes, I understand that.

Thomas was asking what it means to "Move register caching logic to common code", i.e. what the consequences are. The missing words at the end of the first paragraph didn't make the connection obvious between the extra retpoline and the "dumb case".

Paolo

#20 lore raw

Re: [RFC PATCH v3 14/59] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs

In reply to Thomas Gleixner
Quoted 10 lines · Thomas Gleixner
Quoted 4 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Add a capability to effectively allow userspace to query what VM types are supported by KVM.

I really don't see why this has to be named legacy. There are enough reasonable use cases which are perfectly fine using the non-encrypted muck. Just because there is a new hyped feature does not make anything else legacy.

Yeah, this was brought up in the past. The current proposal is to use KVM_X86_DEFAULT_VM[1], though at one point the plan was to use a generic KVM_VM_TYPE_DEFAULT for all architectures[2], not sure what happened to that idea.

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

#84 lore raw

Re: [RFC PATCH v3 47/59] KVM: TDX: Define TDCALL exit reason

In reply to Thomas Gleixner
Quoted 10 lines · Thomas Gleixner
Quoted 5 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Define the TDCALL exit reason, which is carved out from the VMX exit reason namespace as the TDCALL exit from TDX guest to TDX-SEAM is really just a VM-Exit.

How is this carved out? What's the value of this word salad?

It's simply a new exit reason. Not more, not less. So what?

The changelog is alluding to the fact that KVM should never directly see a TDCALL VM-Exit. For TDX, KVM deals only with "returns" from the TDX-Module. The "carved out" bit is calling out that the transition from SEAM Non-Root (the TDX guest) to SEAM Root (the TDX Module) is actually a VT-x/VMX VM-Exit, e.g. if TDX were somehow implemented without relying on VT-x/VMX, then the TDCALL exit reason wouldn't exist.

Quoted 7 lines
Quoted 3 lines

I'm pretty sure that it does not take two engineers to add a new exit reason define, but it takes at least two engineers to come up with a convoluted explanation for it.

Nah, just one ;-)

#32 lore raw

Re: [RFC PATCH v3 18/59] KVM: x86: Add flag to mark TSC as immutable (for TDX)

In reply to Thomas Gleixner
Quoted 6 lines · Thomas Gleixner
Quoted 3 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

The TSC for TDX1 guests is fixed at TD creation time. Add tsc_immutable

What's a TDX1 guest?

The "revision 1.0" version of TDX. Some of these patches use "TDX1" to identify behaviors that may not hold true in future iterations of TDX, and also to highlight things that are dictated by the spec, e.g. some of the guest TSC frequency values. For this patch in particular, there's probably no need to differentiate TDX1 vs. TDX, the qualification was more for cases where KVM needs to define magic values to adhere to the spec, e.g. to make it clear that the magic values aren't made up by KVM.

Quoted 4 lines
Quoted 3 lines

to reflect that the TSC of the guest cannot be changed in any way, and use it to short circuit all paths that lead to one of the myriad TSC adjustment flows.

Quoted 8 lines

I can kinda see the reason for this being valuable on it's own, but in general why does TDX need a gazillion flags to disable tons of different things if _ALL_ these flags are going to be set by for TDX guests anyway?

Seperate flags make only sense when they have a value on their own, i.e. are useful for things outside of TDX. If not they are just useless ballast.

SEV-SNP and TDX have different, but overlapping, restrictions. And SEV-ES also shares most SEV-SNP's restrictions. TDX guests that can be debugged and/or profiled also have different restrictions, though I forget if any of these flags would be affected.

The goal with individual flags is to avoid seemingly arbitrary is_snp_guest() and is_tdx_guest() checks throughout common x86 code, e.g. to avoid confusion over why KVM does X for TDX but Y for SNP. And I personally find it easer to audit KVM behavior with respect to the SNP/TDX specs if the non-obvious restrictions are explicitly set when the VM is created.

For some of the flags, there's also hope that future iterations of TDX will remove some of the restrictions, though that's more of a bonus than a direct justification for adding individual flags.

#109 lore raw

Re: [RFC PATCH v3 54/59] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

In reply to Paolo Bonzini
Quoted 30 lines · Paolo Bonzini
Quoted 2 lines · Thomas Gleixner

You can argue that my request is unreasonable until you are blue in your face, it's not going to lift my NAK on this.

There's no need for that. I'd be saying the same, and I don't think it's particularly helpful that you made it almost a personal issue.

While in this series there is a separation of changes to existing code vs. new code, what's not clear is _why_ you have all those changes. These are not code cleanups or refactorings that can stand on their own feet; lots of the early patches are actually part of the new functionality. And being in the form of "add an argument here" or "export a function there", it's not really easy (or feasible) to review them without seeing how the new functionality is used, which requires a constant back and forth between early patches and the final 2000 line file.

In some sense, the poor commit messages at the beginning of the series are just a symptom of not having any meat until too late, and then dropping it all at once. There's only so much that you can say about an EXPORT_SYMBOL_GPL, the real thing to talk about is probably the thing that refers to that symbol.

If there are some patches that are actually independent, go ahead and submit them early. But more practically, for the bulk of the changes what you need to do is:

1) incorporate into patch 55 a version of tdx.c that essentially does KVM_BUG_ON or WARN_ON for each function. Temporarily keep the same huge patch that adds the remaining 2000 lines of tdx.c

Quoted 6 lines

2) squash the tdx.c stub with patch 44.

3) gather a strace of QEMU starting up a TDX domain.

4) figure out which parts of the code are needed to run until the first ioctl. Make that a first patch.

Hmm, I don't think this approach will work as well as it did for SEV when applied
at a per-ioctl granuarity, I suspect several patches will end up quite large. I
completely agree with the overall idea, but I'd encourage the TDX folks to have a finer granularity where it makes sense, e.g. things like the x2APIC behavior, immutable TSC, memory management, etc... can probably be sliced up into separate patches.

Quoted 10 lines

5) repeat step 4 until you have covered all the code

5) Move the new "KVM: VMX: Add 'main.c' to wrap VMX and TDX" (which also adds the tdx.c stub) as possible in the series.

6) Move each of the new patches as early as possible in the series.

7) Look for candidates for squashing (e.g. commit messages that say it's "used later"; now the use should be very close and the two can be merged). Add to the commit message a note about changes outside VMX.

Generally speaking, I agree. For the flag exposion, I 100% agree that setting the flag in TDX, adding it in x86 is best done in a signal patch, and handling all side effects is best done in a single patch.

But for things like letting debug TDs access registers, I would prefer not to actually squash the two (or more) patches. I agree that two related patches need to be contiguous in the series, but I'd prefer that things with non-trivial changes, especially in common code, are kept separate.

Quoted 4 lines

The resulting series may not be perfect, but it would be a much better starting point for review.

Paolo

#99 lore raw

Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

In reply to Chao Gao
Quoted 9 lines · Chao Gao
Quoted 4 lines

I did not find the information in intel-tdx-module-1eas.pdf nor intel-tdx-cpu-architectural-specification.pdf.

Maybe the version I downloaded is outdated.

Hi Jiangshan,

Please refer to Table 22.162 MSRs that may be Modified by TDH.VP.ENTER, in section 22.2.40 TDH.VP.ENTER leaf.

No file in this link:
https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html
has chapter 22.

Quoted 10 lines
Quoted 6 lines

I guess that the "lazy" restoration mode is not a valid optimization. The SEAM module should restore it to the original value when it tries to reset it to architectural INIT state on exit from TDX VM to KVM since the SEAM module also does it via wrmsr (correct me if not).

Correct.

Quoted 1 line
Quoted 6 lines
Quoted 2 lines

If the SEAM module doesn't know "the original value" of the these MSRs, it would be mere an optimization to save an rdmsr in SEAM.

Yes. Just a rdmsr is saved in TDX module at the cost of host's restoring a MSR. If restoration (wrmsr) can be done in a lazy fashion or even the MSR isn't used by host, some CPU cycles can be saved.

But it adds overall overhead because the wrmsr in TDX module can't be skipped while the unneeded potential overhead of wrmsr is added in user return path.

If TDX module restores the original MSR value, the host hypervisor doesn't need to step in.

I think I'm reviewing the code without the code. It is definitely wrong design to (ab)use the host's user-return-msr mechanism.

Quoted 8 lines
Quoted 3 lines

But there are a lot of other ways for the host to share the values to SEAM in zero overhead.

I am not sure. Looks it requests a new interface between host and TDX module. I guess one problem is how/when to verify host's inputs in case they are invalid.

If the requirement of "lazy restoration" is being added (not seen in the published document yet), you are changing the ABI between the host and the TDX module.

#100 lore raw

Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

Quoted 7 lines · Lai Jiangshan
Quoted 5 lines · Chao Gao
Quoted 4 lines

I did not find the information in intel-tdx-module-1eas.pdf nor intel-tdx-cpu-architectural-specification.pdf.

Maybe the version I downloaded is outdated.

Quoted 8 lines
Quoted 4 lines

Hi Jiangshan,

Please refer to Table 22.162 MSRs that may be Modified by TDH.VP.ENTER, in section 22.2.40 TDH.VP.ENTER leaf.

No file in this link:
https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html
has chapter 22.

https://www.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1.0-public-spec-v0.931.pdf

Quoted 14 lines
Quoted 14 lines
Quoted 7 lines

I guess that the "lazy" restoration mode is not a valid optimization. The SEAM module should restore it to the original value when it tries to reset it to architectural INIT state on exit from TDX VM to KVM since the SEAM module also does it via wrmsr (correct me if not).

Correct.

Quoted 3 lines

If the SEAM module doesn't know "the original value" of the these MSRs, it would be mere an optimization to save an rdmsr in SEAM.

Quoted 7 lines
Quoted 3 lines

Yes. Just a rdmsr is saved in TDX module at the cost of host's restoring a MSR. If restoration (wrmsr) can be done in a lazy fashion or even the MSR isn't used by host, some CPU cycles can be saved.

But it adds overall overhead because the wrmsr in TDX module can't be skipped while the unneeded potential overhead of wrmsr is added in user return path.

It is a trade-off. In this way, TDX module needn't save some host MSRs. Of course, host has to restore those MSRs. the overall impact depends on the frequency of restoring: on every exit from TD VM, on vCPU being scheduled out or on exit to userspace. In the last two cases, it is supposed to have less overhead than saving host MSRs and restoring them on every exit from TD VM.

Quoted 6 lines

If TDX module restores the original MSR value, the host hypervisor doesn't need to step in.

I think I'm reviewing the code without the code. It is definitely wrong design to (ab)use the host's user-return-msr mechanism.

I cannot follow.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

But there are a lot of other ways for the host to share the values to SEAM in zero overhead.

Quoted 8 lines
Quoted 4 lines

I am not sure. Looks it requests a new interface between host and TDX module. I guess one problem is how/when to verify host's inputs in case they are invalid.

If the requirement of "lazy restoration" is being added (not seen in the published document yet), you are changing the ABI between the host and the TDX module.

No, it is already documented in public spec.

TDX module spec just says some MSRs are reset to INIT state by TDX module (un)conditionally during TD exit. When to restore these MSRs to host's values is decided by host.

#101 lore raw

Re: [RFC PATCH v3 53/59] KVM: x86: Add a helper function to restore 4 host MSRs on exit to user space

Quoted 2 lines · Chao Gao

No, it is already documented in public spec.

In intel-tdx-module-1.5-base-spec-348549001.pdf, page 79 "recoverability hist", I think it is a "hint".

Quoted 5 lines

TDX module spec just says some MSRs are reset to INIT state by TDX module (un)conditionally during TD exit. When to restore these MSRs to host's values is decided by host.

Sigh, it is quite a common solution to reset a register to a default value here, for example in VMX, the host GDT.limit and host TSS.limit is reset after VMEXIT, so load_fixmap_gdt() and invalidate_tss_limit() have to be called in host in vmx_vcpu_put().

Off-topic: Is it a good idea to also put load_fixmap_gdt() and invalidate_tss_limit() in user-return-notfier?

#117 lore raw
Quoted 4 lines · On Tue, 30 Nov 2021 18:51:52 +0000 Sean Christopherson wrote:
Quoted 1 line · On Wed, Nov 24, 2021, [email protected] wrote:

- drop load/initialization of TDX module

So what's the plan for loading and initializing TDX modules?

Hi Sean,

Although I don't quite understand what does Isaku mean here (I thought loading/initializing TDX module was never part of TDX KVM series), for this part we are working internally to improve the quality and finalize the code, but currently I don't have ETA of being able to send patches out, but we are trying to send out asap. Sorry this is what I can say for now : (

maintainer
#121 lore raw
In reply to Sean Christopherson
Quoted 4 lines · Sean Christopherson
Quoted 1 line · On Wed, Nov 24, 2021,[email protected] wrote:

- drop load/initialization of TDX module

So what's the plan for loading and initializing TDX modules?

The latest news I got are that Intel has an EFI application that loads it, so loading it from Linux and updating it at runtime can be punted to later.

Paolo

#118 lore raw
In reply to Kai Huang
Quoted 11 lines · Kai Huang
Quoted 4 lines · On Tue, 30 Nov 2021 18:51:52 +0000 Sean Christopherson wrote:
Quoted 1 line · On Wed, Nov 24, 2021, [email protected] wrote:

- drop load/initialization of TDX module

So what's the plan for loading and initializing TDX modules?

Although I don't quite understand what does Isaku mean here (I thought loading/initializing TDX module was never part of TDX KVM series), for this part we are working internally to improve the quality and finalize the code, but currently I don't have ETA of being able to send patches out, but we are trying to send out asap. Sorry this is what I can say for now : (

v1/v2 has it. Anyway The plan is what Kai said. The code will reside in the x86 common directory instead of kvm.

The only dependency between the part of loading/initializing TDX module and TDX KVM is only single function to get the info about the TDX module.

Signature · 1 line
Isaku Yamahata <[email protected]>
#119 lore raw
Quoted 9 lines · Isaku Yamahata
Quoted 6 lines · Kai Huang
Quoted 4 lines · On Tue, 30 Nov 2021 18:51:52 +0000 Sean Christopherson wrote:
Quoted 1 line · On Wed, Nov 24, 2021, [email protected] wrote:

- drop load/initialization of TDX module

So what's the plan for loading and initializing TDX modules?

Quoted 7 lines
Quoted 5 lines

Although I don't quite understand what does Isaku mean here (I thought loading/initializing TDX module was never part of TDX KVM series), for this part we are working internally to improve the quality and finalize the code, but currently I don't have ETA of being able to send patches out, but we are trying to send out asap. Sorry this is what I can say for now : (

v1/v2 has it.

No, v1 had support for the old architecture where SEAMLDR was a single ACM, it did not support the new split persistent/non-persistent architcture. v2 didn't have support for either architecture.

Quoted 2 lines

Anyway The plan is what Kai said. The code will reside in the x86 common directory instead of kvm.

But what's the plan at a higher level? Will the kernel load the ACM or is that done by firmware? If it's done by firmware, which entity is responsibile for loading the TDX module? If firmware loads the module, what's the plan for upgrading the module without a reboot? When will the kernel initialize the module, regardless of who loads it?

All of those unanswered questions make it nigh impossible to review the KVM support because the code organization and APIs provided will differ based on how the kernel handles loading and initializing the TDX module.

#21 lore raw

Re: [RFC PATCH v3 14/59] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs

In reply to Sean Christopherson
Quoted 12 lines · Sean Christopherson
Quoted 10 lines · Thomas Gleixner
Quoted 4 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Add a capability to effectively allow userspace to query what VM types are supported by KVM.

I really don't see why this has to be named legacy. There are enough reasonable use cases which are perfectly fine using the non-encrypted muck. Just because there is a new hyped feature does not make anything else legacy.

Quoted 6 lines

Yeah, this was brought up in the past. The current proposal is to use KVM_X86_DEFAULT_VM[1], though at one point the plan was to use a generic KVM_VM_TYPE_DEFAULT for all architectures[2], not sure what happened to that idea.

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Currently <feature>_{unsupported, disallowed} are added and the check is
sprinkled and warn in the corresponding low level tdx code. It helped to detect dubious behavior of guest or qemu.

The other approach is to silently ignore them (SMI, INIT, IRQ etc) without such check. The pros is, the code would be simpler and it's what SEV does today. the cons is, it would bes hard to track down such cases and the user would be confused. For example, when user requests reset/SMI, it's silently ignored. The some check would still be needed.
Any thoughts?

Signature · 1 line
Isaku Yamahata <[email protected]>
#122 lore raw
In reply to Paolo Bonzini
Quoted 10 lines · On Wed, 1 Dec 2021 16:05:32 +0100 Paolo Bonzini wrote:
Quoted 4 lines · Sean Christopherson
Quoted 1 line · On Wed, Nov 24, 2021,[email protected] wrote:

- drop load/initialization of TDX module

So what's the plan for loading and initializing TDX modules?

The latest news I got are that Intel has an EFI application that loads it, so loading it from Linux and updating it at runtime can be punted to later.

Yes we are heading this approach.

#120 lore raw
In reply to Sean Christopherson
Quoted 10 lines
Quoted 4 lines

Anyway The plan is what Kai said. The code will reside in the x86 common directory instead of kvm.

But what's the plan at a higher level? Will the kernel load the ACM or is that done by firmware? If it's done by firmware, which entity is responsibile for loading the TDX module? If firmware loads the module, what's the plan for upgrading the module without a reboot? When will the kernel initialize the module, regardless of who loads it?

The UEFI loads both ACM and TDX module before booting into kernel by using UEFI tool. The runtime update is pushed out for future support. One goal of this is to reduce the code size so that it can be reviewed more easily and quickly.

And yes kernel will initialize the TDX module. The direction we are heading is to allow to defer TDX module initialization when TDX is truly needed, i.e. When KVM is loaded with TDX support, or first TD is created. The code will basically still reside in host kernel, provided as functions, etc. And at first stage, KVM will call those functions to initialize TDX when needed.

The advantage of this approach is it provides more flexibility: the TDX module initialization code can be reused by future TDX runtime update, etc. And with only initializing TDX in KVM, the host kernel doesn't need to handle entering VMX operation, etc. It can be introduced later when needed.

Quoted 4 lines

All of those unanswered questions make it nigh impossible to review the KVM support because the code organization and APIs provided will differ based on how the kernel handles loading and initializing the TDX module.

I think theoretically loading/initializing module should be quite independent from KVM series, but yes in practice the APIs matter, but I also don't expect this will reduce the ability to review KVM series a lot as RFC.

Anyway sending out host kernel patches is our top priority now and we are trying to do asap.

#22 lore raw

Re: [RFC PATCH v3 14/59] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs

In reply to Isaku Yamahata
Quoted 19 lines · Isaku Yamahata
Quoted 16 lines · Sean Christopherson
Quoted 10 lines · Thomas Gleixner
Quoted 4 lines · isaku yamahata

From: Sean Christopherson <[email protected]>

Add a capability to effectively allow userspace to query what VM types are supported by KVM.

I really don't see why this has to be named legacy. There are enough reasonable use cases which are perfectly fine using the non-encrypted muck. Just because there is a new hyped feature does not make anything else legacy.

Yeah, this was brought up in the past. The current proposal is to use KVM_X86_DEFAULT_VM[1], though at one point the plan was to use a generic KVM_VM_TYPE_DEFAULT for all architectures[2], not sure what happened to that idea.

Quoted 6 lines
Quoted 2 lines

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Currently <feature>_{unsupported, disallowed} are added and the check is
sprinkled and warn in the corresponding low level tdx code. It helped to detect dubious behavior of guest or qemu.

KVM shouldn't log a message or WARN unless the issue is detected at a late sanity check, i.e. where failure indicates a KVM bug. Other than that, I agree that KVM should reject ioctls() that directly violate the rules of a confidential VM with an appropriate error code. I don't think KVM should reject everything though, e.g. if the guest attempts to send an SMI, dropping the request on the floor is the least awful option because we can't communicate an error to the guest without making up our own architecture, and exiting to userspace with -EINVAL from deep in KVM would be both painful to implement and an overreaction since doing so would likely kill the guest.

Quoted 9 lines

The other approach is to silently ignore them (SMI, INIT, IRQ etc) without such check. The pros is, the code would be simpler and it's what SEV does today. the cons is, it would bes hard to track down such cases and the user would be confused. For example, when user requests reset/SMI, it's silently ignored. The some check would still be needed.
Any thoughts?

Signature · 1 line
Isaku Yamahata <[email protected]>