Skip to content

RFC KVM: X86: TDX support

[email protected] kvm lkml 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 RFC Latest59 patches 63 repliesIn review
  2. v2 RFC69 patches 105 repliesApplied
  3. v1 RFC67 patches 12 repliesSuperseded

Changes in v3 from the cover letter

- update based on patch review
- support TDP MMU
- drop non-essential fetures (ftrace etc.) to reduce patch size

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/59x86/mktme: move out MKTME related constatnts/macro to msr-index.harch/x86/include/asm/msr-index.h arch/x86/kernel/cpu/intel.c+16 -140 replies
2/59x86/mtrr: mask out keyid bits from variable mtrr mask registerarch/x86/kernel/cpu/mtrr/mtrr.c+9 -00 replies
3/59KVM: TDX: Define TDX architectural definitionsarch/x86/kvm/vmx/tdx_arch.h+219 -00 replies
4/59KVM: TDX: Add TDX "architectural" error codesarch/x86/kvm/vmx/tdx_errno.h+111 -00 replies
5/59KVM: TDX: add a helper function for kvm to call seamcallarch/x86/kvm/vmx/seamcall.h+113 -00 replies
6/59KVM: TDX: Add C wrapper functions for TDX SEAMCALLsarch/x86/kvm/vmx/tdx_ops.h+210 -00 replies
7/59KVM: TDX: Add helper functions to print TDX SEAMCALL errorarch/x86/kvm/Makefile arch/x86/kvm/vmx/seamcall.h arch/x86/kvm/vmx/tdx_error.c+57 -00 replies
8/59KVM: Export kvm_io_bus_read for use by TDX for PV MMIOvirt/kvm/kvm_main.c+1 -01 reply
9/59KVM: Enable hardware before doing arch VM initializationvirt/kvm/kvm_main.c+8 -81 reply
10/59KVM: x86: Split core of hypercall emulation to helper functionarch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c+38 -210 replies
11/59KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIOarch/x86/kvm/x86.c+1 -00 replies
12/59KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by defaultarch/x86/kvm/mmu/mmu.c+20 -11 reply
13/59KVM: Add max_vcpus field in common 'struct kvm'arch/arm64/include/asm/kvm_host.h arch/arm64/kvm/arm.c arch/arm64/kvm/vgic/vgic-init.c +2 more+8 -121 reply
14/59KVM: x86: Add vm_type to differentiate legacy VMs from protected VMsarch/x86/include/asm/kvm-x86-ops.h arch/x86/include/asm/kvm_host.h arch/x86/include/uapi/asm/kvm.h +6 more+35 -14 replies
15/59KVM: x86: Introduce "protected guest" concept and block disallowed ioctlsarch/x86/kvm/x86.c+66 -141 reply
16/59KVM: x86: Add per-VM flag to disable direct IRQ injectionarch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c+4 -12 replies
17/59KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCEarch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c+8 -71 reply
18/59KVM: x86: Add flag to mark TSC as immutable (for TDX)arch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c+28 -82 replies
19/59KVM: Add per-VM flag to mark read-only memory as unsupportedarch/x86/kvm/x86.c include/linux/kvm_host.h virt/kvm/kvm_main.c+11 -40 replies
20/59KVM: Add per-VM flag to disable dirty logging of memslots for TDsinclude/linux/kvm_host.h virt/kvm/kvm_main.c+5 -10 replies
21/59KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level routesarch/x86/include/asm/kvm_host.h arch/x86/kvm/ioapic.c arch/x86/kvm/irq_comm.c +2 more+20 -30 replies
22/59KVM: x86: add per-VM flags to disable SMI/INIT/SIPIarch/x86/include/asm/kvm_host.h arch/x86/kvm/irq_comm.c arch/x86/kvm/x86.c+7 -20 replies
23/59KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUIDarch/x86/kvm/x86.c+5 -23 replies
24/59KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()arch/x86/include/asm/kvm-x86-ops.h arch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c+8 -00 replies
25/59KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OParch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c+14 -01 reply
26/59KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()arch/x86/include/asm/kvm-x86-ops.h arch/x86/include/asm/kvm_host.h arch/x86/kvm/svm/svm.c +2 more+14 -34 replies
27/59KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched behaviorarch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c+10 -30 replies
28/59KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()arch/x86/kvm/x86.c+3 -12 replies
29/59KVM: x86: Add option to force LAPIC expiration waitarch/x86/kvm/lapic.c arch/x86/kvm/lapic.h arch/x86/kvm/svm/svm.c +1 more+5 -51 reply
30/59KVM: x86: Add guest_supported_xss placholderarch/x86/include/asm/kvm_host.h+1 -01 reply
31/59KVM: x86: Add infrastructure for stolen GPA bitsarch/x86/kvm/mmu.h arch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/mmu_internal.h +1 more+100 -351 reply
32/59KVM: x86/mmu: Explicitly check for MMIO spte in fast page faultarch/x86/kvm/mmu/mmu.c+1 -10 replies
33/59KVM: x86/mmu: Ignore bits 63 and 62 when checking for "present" SPTEsarch/x86/kvm/mmu/paging_tmpl.h arch/x86/kvm/mmu/spte.h+14 -10 replies
34/59KVM: x86/mmu: Allow non-zero init value for shadow PTEarch/x86/kvm/mmu.h arch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/spte.c +1 more+56 -70 replies
35/59KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()arch/x86/kvm/mmu/mmu.c+1 -10 replies
36/59KVM: x86/mmu: Frame in support for private/inaccessible shadow pagesarch/x86/include/asm/kvm-x86-ops.h arch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu.h +6 more+463 -1210 replies
37/59KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDXarch/x86/kvm/mmu.h arch/x86/kvm/mmu/mmu.c+41 -00 replies
38/59KVM: x86/mmu: Allow per-VM override of the TDP max page levelarch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu.h arch/x86/kvm/mmu/mmu.c+4 -10 replies
39/59KVM: VMX: Modify NMI and INTR handlers to take intr_info as paramarch/x86/kvm/vmx/vmx.c+7 -81 reply
40/59KVM: VMX: Move NMI/exception handler to common helperarch/x86/kvm/vmx/common.h arch/x86/kvm/vmx/vmx.c+60 -441 reply
41/59KVM: VMX: Split out guts of EPT violation to common/exposed functionarch/x86/kvm/vmx/common.h arch/x86/kvm/vmx/vmx.c+34 -281 reply
42/59KVM: VMX: Define EPT Violation architectural bitsarch/x86/include/asm/vmx.h+4 -00 replies
43/59KVM: VMX: Define VMCS encodings for shared EPT pointerarch/x86/include/asm/vmx.h+2 -00 replies
44/59KVM: VMX: Add 'main.c' to wrap VMX and TDXarch/x86/kvm/Makefile arch/x86/kvm/vmx/main.c arch/x86/kvm/vmx/vmx.c +1 more+1000 -3180 replies
45/59KVM: VMX: Move setting of EPT MMU masks to common VT-x codearch/x86/kvm/vmx/main.c arch/x86/kvm/vmx/vmx.c+5 -41 reply
46/59KVM: VMX: Move register caching logic to common codearch/x86/kvm/vmx/main.c arch/x86/kvm/vmx/vmx.c arch/x86/kvm/vmx/x86_ops.h+43 -474 replies
47/59KVM: TDX: Define TDCALL exit reasonarch/x86/include/uapi/asm/vmx.h+3 -12 replies
48/59KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpersarch/x86/kvm/vmx/tdx.h+169 -00 replies
49/59KVM: VMX: Add macro framework to read/write VMCS for VMs and TDsarch/x86/kvm/vmx/common.h+41 -01 reply
50/59KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.harch/x86/kvm/vmx/common.h arch/x86/kvm/vmx/vmx.c+45 -430 replies
51/59KVM: VMX: MOVE GDT and IDT accessors to common codearch/x86/kvm/vmx/main.c arch/x86/kvm/vmx/vmx.c arch/x86/kvm/vmx/x86_ops.h+5 -161 reply
52/59KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX codearch/x86/kvm/vmx/common.h arch/x86/kvm/vmx/vmx.c+15 -91 reply
53/59KVM: x86: Add a helper function to restore 4 host MSRs on exit to user spacearch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c+21 -58 replies
54/59KVM: X86: Introduce initial_tsc_khz in struct kvm_archarch/x86/include/asm/kvm_host.h arch/x86/kvm/vmx/tdx.c arch/x86/kvm/x86.c+2236 -17 replies
55/59KVM: TDX: Add "basic" support for building and running Trust Domainsarch/x86/events/intel/ds.c arch/x86/include/uapi/asm/kvm.h arch/x86/include/uapi/asm/vmx.h +16 more+946 -630 replies
56/59KVM: TDX: Protect private mapping related SEAMCALLs with spinlockarch/x86/kvm/vmx/tdx.c arch/x86/kvm/vmx/tdx.h+108 -100 replies
57/59KVM, x86/mmu: Support TDX private mapping for TDP MMUarch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/spte.h arch/x86/kvm/mmu/tdp_iter.h +2 more+567 -770 replies
58/59KVM: TDX: exit to user space on GET_QUOTE, SETUP_EVENT_NOTIFY_INTERRUPTarch/x86/kvm/vmx/tdx.c include/uapi/linux/kvm.h+170 -00 replies
59/59Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)Documentation/virt/kvm/api.rst Documentation/virt/kvm/intel-tdx.rst+367 -10 replies

Reviews trailers per patch

1 of 59 patches has a Reviewed-by or Acked-by.

PatchPaolo Bonzini
1/59
2/59
3/59
4/59
5/59
6/59
7/59
8/59R-b
9/59
10/59
11/59
12/59
13/59
14/59
15/59
16/59
17/59
18/59
19/59
20/59
21/59
22/59
23/59
24/59
25/59
26/59
27/59
28/59
29/59
30/59
31/59
32/59
33/59
34/59
35/59
36/59
37/59
38/59
39/59
40/59
41/59
42/59
43/59
44/59
45/59
46/59
47/59
48/59
49/59
50/59
51/59
52/59
53/59
54/59
55/59
56/59
57/59
58/59
59/59

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [RFC PATCH v3 00/59] KVM: X86: TDX support

From: Isaku Yamahata <[email protected]>

Changes from v2:
- update based on patch review
- support TDP MMU
- drop non-essential fetures (ftrace etc.) to reduce patch size

TODO:
- integrate vm type patch
- integrate unmapping user space mapping

---
* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform. [1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.

* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.

- 01-13: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 14-30: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 31-38: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 39-54: refactoring KVM
- 55: main patch to add "basic" support for building/running TDX.
- 56-57: TDP MMU support
- 58: support TDX hypercall, GetQuote and SetupEventNotifyInterrupt, that
requires qemu help
- 59: Documentation

* Missing features
Those major features are intentionally missing from this patch series to keep this patch series small. They are addressed as independent patch series.

- qemu gdb stub support
- Large page support
- guest PMU support
- and more

Changes from v1:
- rebase to v5.13
- drop load/initialization of TDX module
- catch up the update of related specifications.
- rework on C-wrapper function to invoke seamcall
- various code clean up

[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAS
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf
[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVF

Chao Gao (1):
KVM: x86: Add a helper function to restore 4 host MSRs on exit to user
space

Isaku Yamahata (9):
x86/mktme: move out MKTME related constatnts/macro to msr-index.h x86/mtrr: mask out keyid bits from variable mtrr mask register
KVM: TDX: Define TDX architectural definitions
KVM: TDX: add a helper function for kvm to call seamcall
KVM: TDX: Add helper functions to print TDX SEAMCALL error
KVM: Add per-VM flag to mark read-only memory as unsupported
KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
KVM: TDX: exit to user space on GET_QUOTE,
SETUP_EVENT_NOTIFY_INTERRUPT
Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)

Kai Huang (3):
KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level
routes
KVM: TDX: Protect private mapping related SEAMCALLs with spinlock KVM, x86/mmu: Support TDX private mapping for TDP MMU

Rick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bits

Sean Christopherson (44):
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Split core of hypercall emulation to helper function
KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO
KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
KVM: Add max_vcpus field in common 'struct kvm'
KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
KVM: x86: Introduce "protected guest" concept and block disallowed
ioctls
KVM: x86: Add per-VM flag to disable direct IRQ injection
KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE
KVM: x86: Add flag to mark TSC as immutable (for TDX)
KVM: Add per-VM flag to disable dirty logging of memslots for TDs
KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()
KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched
behavior
KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: x86: Add guest_supported_xss placholder
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Ignore bits 63 and 62 when checking for "present" SPTEs
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()
KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
KVM: VMX: Move NMI/exception handler to common helper
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Define EPT Violation architectural bits
KVM: VMX: Define VMCS encodings for shared EPT pointer
KVM: VMX: Add 'main.c' to wrap VMX and TDX
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: VMX: Move register caching logic to common code
KVM: TDX: Define TDCALL exit reason
KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h
KVM: VMX: MOVE GDT and IDT accessors to common code
KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX
code
KVM: TDX: Add "basic" support for building and running Trust Domains

Xiaoyao Li (1):
KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

 Documentation/virt/kvm/api.rst        |    9 +-
 Documentation/virt/kvm/intel-tdx.rst  |  359 ++++
 arch/arm64/include/asm/kvm_host.h     |    3 -
 arch/arm64/kvm/arm.c                  |    7 +-
 arch/arm64/kvm/vgic/vgic-init.c       |    6 +-
 arch/x86/events/intel/ds.c            |    1 +
 arch/x86/include/asm/kvm-x86-ops.h    |   11 +
 arch/x86/include/asm/kvm_host.h       |   63 +-
 arch/x86/include/asm/msr-index.h      |   16 +
 arch/x86/include/asm/vmx.h            |    6 +
 arch/x86/include/uapi/asm/kvm.h       |   60 +
 arch/x86/include/uapi/asm/vmx.h       |    7 +-
 arch/x86/kernel/cpu/intel.c           |   14 -
 arch/x86/kernel/cpu/mtrr/mtrr.c       |    9 +
 arch/x86/kvm/Makefile                 |    6 +-
 arch/x86/kvm/ioapic.c                 |    4 +
 arch/x86/kvm/irq_comm.c               |   13 +-
 arch/x86/kvm/lapic.c                  |    7 +-
 arch/x86/kvm/lapic.h                  |    2 +-
 arch/x86/kvm/mmu.h                    |   29 +-
 arch/x86/kvm/mmu/mmu.c                |  667 ++++++-
 arch/x86/kvm/mmu/mmu_internal.h       |   12 +
 arch/x86/kvm/mmu/paging_tmpl.h        |   32 +-
 arch/x86/kvm/mmu/spte.c               |   15 +-
 arch/x86/kvm/mmu/spte.h               |   51 +-
 arch/x86/kvm/mmu/tdp_iter.h           |    2 +-
 arch/x86/kvm/mmu/tdp_mmu.c            |  544 +++++-
 arch/x86/kvm/mmu/tdp_mmu.h            |   15 +-
 arch/x86/kvm/svm/svm.c                |   13 +-
 arch/x86/kvm/vmx/common.h             |  178 ++
 arch/x86/kvm/vmx/main.c               | 1152 ++++++++++++
 arch/x86/kvm/vmx/posted_intr.c        |    6 +
 arch/x86/kvm/vmx/seamcall.h           |  116 ++
 arch/x86/kvm/vmx/tdx.c                | 2437 +++++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.h                |  290 +++
 arch/x86/kvm/vmx/tdx_arch.h           |  239 +++
 arch/x86/kvm/vmx/tdx_errno.h          |  111 ++
 arch/x86/kvm/vmx/tdx_error.c          |   53 +
 arch/x86/kvm/vmx/tdx_ops.h            |  224 +++
 arch/x86/kvm/vmx/tdx_stubs.c          |   50 +
 arch/x86/kvm/vmx/vmenter.S            |  146 ++
 arch/x86/kvm/vmx/vmx.c                |  689 ++-----
 arch/x86/kvm/vmx/x86_ops.h            |  203 ++
 arch/x86/kvm/x86.c                    |  276 ++-
 include/linux/kvm_host.h              |    5 +
 include/uapi/linux/kvm.h              |   59 +
 tools/arch/x86/include/uapi/asm/kvm.h |   55 +
 tools/include/uapi/linux/kvm.h        |    2 +
 virt/kvm/kvm_main.c                   |   34 +-
 49 files changed, 7469 insertions(+), 839 deletions(-)
 create mode 100644 Documentation/virt/kvm/intel-tdx.rst
 create mode 100644 arch/x86/kvm/vmx/common.h
 create mode 100644 arch/x86/kvm/vmx/main.c
 create mode 100644 arch/x86/kvm/vmx/seamcall.h
 create mode 100644 arch/x86/kvm/vmx/tdx.c
 create mode 100644 arch/x86/kvm/vmx/tdx.h
 create mode 100644 arch/x86/kvm/vmx/tdx_arch.h
 create mode 100644 arch/x86/kvm/vmx/tdx_errno.h
 create mode 100644 arch/x86/kvm/vmx/tdx_error.c
 create mode 100644 arch/x86/kvm/vmx/tdx_ops.h
 create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c
 create mode 100644 arch/x86/kvm/vmx/x86_ops.h
Signature · 1 line
2.25.1