Skip to content

RFC TDX: Dynamic PAMT + S-EPT Hugepage

Sean Christopherson kvm linux-coco lkml 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 RFC Latest45 patches 111 repliesIn review
  2. v11 patch 5 repliesSuperseded

Patches in v5 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/45x86/tdx: Use pg_level in TDX APIs, not the TDX-Module's 0-based levelarch/x86/include/asm/tdx.h arch/x86/kvm/vmx/tdx.c arch/x86/virt/vmx/tdx/tdx.c+26 -251 reply
2/45KVM: x86/mmu: Update iter->old_spte if cmpxchg64 on mirror SPTE "fails"arch/x86/kvm/mmu/tdp_mmu.c+5 -57 replies
3/45KVM: TDX: Account all non-transient page allocations for per-TD structuresarch/x86/kvm/vmx/tdx.c+6 -62 replies
4/45KVM: x86: Make "external SPTE" ops that can fail RET0 static callsarch/x86/include/asm/kvm-x86-ops.h+3 -35 replies
5/45KVM: TDX: Drop kvm_x86_ops.link_external_spt(), use .set_external_spte() for allarch/x86/include/asm/kvm-x86-ops.h arch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu/tdp_mmu.c +1 more+48 -548 replies
6/45KVM: x86/mmu: Fold set_external_spte_present() into its sole callerarch/x86/kvm/mmu/tdp_mmu.c+36 -463 replies
7/45KVM: x86/mmu: Plumb the SPTE _pointer_ into the TDP MMU's handle_changed_spte()arch/x86/kvm/mmu/tdp_mmu.c+11 -100 replies
8/45KVM: x86/mmu: Propagate mirror SPTE removal to S-EPT in handle_changed_spte()arch/x86/kvm/mmu/tdp_mmu.c+14 -2910 replies
9/45KVM: x86: Rework .free_external_spt() into .reclaim_external_sp()arch/x86/include/asm/kvm-x86-ops.h arch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu/tdp_mmu.c +1 more+17 -294 replies
10/45x86/tdx: Move all TDX error defines into <asm/shared/tdx_errno.h>arch/x86/include/asm/shared/tdx.h arch/x86/include/asm/shared/tdx_errno.h arch/x86/include/asm/tdx.h +1 more+23 -261 reply
11/45x86/tdx: Add helpers to check return status codesarch/x86/coco/tdx/tdx.c arch/x86/include/asm/shared/tdx_errno.h arch/x86/include/asm/tdx.h +2 more+73 -344 replies
12/45x86/virt/tdx: Simplify tdmr_get_pamt_sz()arch/x86/virt/vmx/tdx/tdx.c+29 -640 replies
13/45x86/virt/tdx: Allocate page bitmap for Dynamic PAMTarch/x86/include/asm/tdx.h arch/x86/include/asm/tdx_global_metadata.h arch/x86/virt/vmx/tdx/tdx.c +1 more+31 -10 replies
14/45x86/virt/tdx: Allocate reference counters for PAMT memoryarch/x86/virt/vmx/tdx/tdx.c+46 -10 replies
15/45x86/virt/tdx: Improve PAMT refcounts allocation for sparse memoryarch/x86/virt/vmx/tdx/tdx.c+118 -40 replies
16/45x86/virt/tdx: Add tdx_alloc/free_control_page() helpersarch/x86/include/asm/tdx.h arch/x86/virt/vmx/tdx/tdx.c arch/x86/virt/vmx/tdx/tdx.h+289 -111 replies
17/45x86/virt/tdx: Optimize tdx_alloc/free_control_page() helpersarch/x86/include/asm/shared/tdx_errno.h arch/x86/virt/vmx/tdx/tdx.c+54 -170 replies
18/45KVM: TDX: Allocate PAMT memory for TD and vCPU control structuresarch/x86/kvm/vmx/tdx.c+14 -210 replies
19/45KVM: Allow owner of kvm_mmu_memory_cache to provide a custom page allocatorinclude/linux/kvm_types.h virt/kvm/kvm_main.c+8 -17 replies
20/45KVM: x86/mmu: Allocate/free S-EPT pages using tdx_{alloc,free}_control_page()arch/x86/include/asm/kvm-x86-ops.h arch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu/mmu.c +3 more+30 -2515 replies
21/45x86/tdx: Add APIs to support get/put of DPAMT entries from KVM, under spinlockarch/x86/include/asm/tdx.h arch/x86/virt/vmx/tdx/tdx.c+76 -60 replies
22/45KVM: TDX: Get/put PAMT pages when (un)mapping private memoryarch/x86/include/asm/kvm-x86-ops.h arch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu/mmu.c +2 more+47 -58 replies
23/45x86/virt/tdx: Enable Dynamic PAMTarch/x86/include/asm/tdx.h arch/x86/virt/vmx/tdx/tdx.c arch/x86/virt/vmx/tdx/tdx.h+13 -40 replies
24/45Documentation/x86: Add documentation for TDX's Dynamic PAMTDocumentation/arch/x86/tdx.rst+21 -00 replies
25/45*** DO NOT MERGE *** x86/virt/tdx: Don't assume guest memory is backed by struct pagearch/x86/include/asm/tdx.h arch/x86/kvm/vmx/tdx.c arch/x86/virt/vmx/tdx/tdx.c+59 -620 replies
26/45x86/virt/tdx: Enhance tdh_mem_page_aug() to support huge pagesarch/x86/virt/vmx/tdx/tdx.c+5 -10 replies
27/45x86/virt/tdx: Enhance tdh_phymem_page_wbinvd_hkid() to invalidate huge pagesarch/x86/include/asm/tdx.h arch/x86/kvm/vmx/tdx.c arch/x86/virt/vmx/tdx/tdx.c+14 -60 replies
28/45x86/virt/tdx: Extend "reset page" quirk to support huge pagesarch/x86/include/asm/tdx.h arch/x86/kvm/vmx/tdx.c arch/x86/virt/vmx/tdx/tdx.c+10 -50 replies
29/45x86/virt/tdx: Get/Put DPAMT page pair if and only if mapping size is 4KBarch/x86/include/asm/tdx.h arch/x86/kvm/vmx/tdx.c arch/x86/virt/vmx/tdx/tdx.c+23 -110 replies
30/45x86/virt/tdx: Add API to demote a 2MB mapping to 512 4KB mappingsarch/x86/include/asm/tdx.h arch/x86/virt/vmx/tdx/tdx.c arch/x86/virt/vmx/tdx/tdx.h+64 -05 replies
31/45KVM: x86/mmu: Prevent hugepage promotion for mirror roots in fault patharch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/tdp_mmu.c+13 -20 replies
32/45KVM: x86/mmu: Plumb the old_spte into kvm_x86_ops.set_external_spte()arch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu/tdp_mmu.c arch/x86/kvm/vmx/tdx.c+11 -110 replies
33/45KVM: TDX: Hoist tdx_sept_remove_private_spte() above set_private_spte()arch/x86/kvm/vmx/tdx.c+97 -970 replies
34/45KVM: TDX: Handle removal of leaf SPTEs in .set_private_spte()arch/x86/include/asm/kvm-x86-ops.h arch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu/tdp_mmu.c +1 more+23 -210 replies
35/45KVM: TDX: Add helper to handle mapping leaf SPTE into S-EPTarch/x86/kvm/vmx/tdx.c+51 -460 replies
36/45KVM: TDX: Move S-EPT page demotion TODO to tdx_sept_set_private_spte()arch/x86/kvm/mmu/tdp_mmu.c arch/x86/kvm/vmx/tdx.c+5 -90 replies
37/45KVM: x86/tdp_mmu: Alloc external_spt page for mirror page table splittingarch/x86/kvm/mmu/tdp_mmu.c+11 -23 replies
38/45KVM: x86/mmu: Add Dynamic PAMT support in TDP MMU for vCPU-induced page splitarch/x86/kvm/mmu/tdp_mmu.c arch/x86/kvm/vmx/tdx.c+19 -90 replies
39/45KVM: TDX: Add core support for splitting/demoting 2MiB S-EPT to 4KiBarch/x86/kvm/vmx/tdx.c+48 -30 replies
40/45KVM: x86: Introduce hugepage_set_guest_inhibit()arch/x86/kvm/mmu.h arch/x86/kvm/mmu/mmu.c+22 -30 replies
41/45KVM: TDX: Honor the guest's accept level contained in an EPT violationarch/x86/kvm/mmu/tdp_mmu.c arch/x86/kvm/mmu/tdp_mmu.h arch/x86/kvm/vmx/tdx.c +1 more+92 -01 reply
42/45KVM: guest_memfd: Add helpers to get start/end gfns give gmem+slot+pgoffvirt/kvm/guest_memfd.c+17 -40 replies
43/45*** DO NOT MERGE *** KVM: guest_memfd: Add pre-zap arch hook for shared<=>private conversionarch/x86/kvm/Kconfig arch/x86/kvm/mmu/tdp_mmu.c include/linux/kvm_host.h +2 more+66 -21 reply
44/45KVM: x86/mmu: Add support for splitting S-EPT hugepages on conversionarch/x86/include/asm/kvm_host.h arch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/tdp_mmu.c +2 more+107 -115 replies
45/45KVM: TDX: Turn on PG_LEVEL_2March/x86/kvm/vmx/tdx.c+30 -70 replies

Reviews trailers per patch

9 of 45 patches have a Reviewed-by or Acked-by.

PatchKiryl ShutsemauKai HuangRick EdgecombeSagi ShaharBinbin WuDave Hansen
1/45A-b R-b T-b R-b T-b A-b
2/45R-b R-b
3/45R-b R-b
4/45R-b R-b
5/45R-b
6/45
7/45
8/45
9/45
10/45T-b
11/45T-b
12/45T-b R-b
13/45T-b R-b
14/45T-b
15/45T-b
16/45
17/45T-b
18/45
19/45R-b
20/45R-b
21/45
22/45
23/45T-b
24/45T-b
25/45
26/45
27/45
28/45
29/45
30/45
31/45
32/45
33/45
34/45
35/45
36/45
37/45
38/45
39/45
40/45
41/45
42/45
43/45
44/45
45/45

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 6 reviewers.

Read the v5 thread

Cover letter [RFC PATCH v5 00/45] TDX: Dynamic PAMT + S-EPT Hugepage

This is a combined series of Dynamic PAMT (from Rick), and S-EPT hugepage support (from Yan). Except for some last minute tweaks to the DPAMT array args stuff, a version of this based on a Google-internal kernel has been moderately well tested (thanks Vishal!). But overall it's still firmly RFC as I have deliberately NOT addressed others feedback from v4 of DPAMT and v3 of S-EPT hugepage (mostly lack of cycles), and there's at least one patch in here that shouldn't be merged as-is (the quick-and-dirty switch from struct page to raw pfns).

My immediate goal is to solidify the designs for DPAMT and S-EPT hugepage. Given the substantial design changes I am proposing, posting an end-to-end RFC seemed like a much better method than trying to communicate my thoughts piecemeal.

As for landing these series, I think the fastest overall approach would be to land patches 1-4 asap (tangentially related cleanups and fixes), agree on a design (hopefully), and then hand control back to Rick and Yan to polish their respective series for merge.

I also want to land the VMXON series[*] before DPAMT, because there's a nasty wart where KVM wires up a DPAMT-specific hook even if DPAMT is disabled, because KVM's ordering needs to set the vendor hooks before tdx_sysinfo is ready. Decoupling VMXON from KVM solves that problem, because it lets the TDX subsystem parse sysinfo before TDX is loaded.

Beyond that dependency, I am comfortable landing both DPAMT and S-EPT hugepage support without any other prereqs, i.e. without an in-tree way to light up the S-EPT hugepage code due to lack of hugepage support in guest_memfd. Outside of the guest_memfd arch hook for in-place conversion, S-EPT hugepage support doesn't have any direction dependencies/conflicts with guest_memfd hugepage or in-place conversion support (which is great, because it means we didn't totally botch the design!). E.g. Vishal's been able to test this code precisely because it applies relatively cleanly on an internal branch with a whole pile of guest_memfd changes.

Applies on kvm-x86 next (specifically kvm-x86-next-2026.01.23).

[*] https://lore.kernel.org/all/[email protected]

P.S. I apologize if I clobbered any of the Author attribution or SoBs. I
was moving patches around and synchronizing between an internal tree and this upstream version, so things may have gotten a bit wonky.

Isaku Yamahata (1):
KVM: x86/tdp_mmu: Alloc external_spt page for mirror page table
splitting

Kiryl Shutsemau (12):
x86/tdx: Move all TDX error defines into <asm/shared/tdx_errno.h> x86/tdx: Add helpers to check return status codes x86/virt/tdx: Allocate page bitmap for Dynamic PAMT x86/virt/tdx: Allocate reference counters for PAMT memory x86/virt/tdx: Improve PAMT refcounts allocation for sparse memory x86/virt/tdx: Add tdx_alloc/free_control_page() helpers x86/virt/tdx: Optimize tdx_alloc/free_control_page() helpers
KVM: TDX: Allocate PAMT memory for TD and vCPU control structures
KVM: TDX: Get/put PAMT pages when (un)mapping private memory x86/virt/tdx: Enable Dynamic PAMT
Documentation/x86: Add documentation for TDX's Dynamic PAMT x86/virt/tdx: Get/Put DPAMT page pair if and only if mapping size is

    4KB

Rick Edgecombe (3):
x86/virt/tdx: Simplify tdmr_get_pamt_sz()
x86/tdx: Add APIs to support get/put of DPAMT entries from KVM, under
spinlock
KVM: x86/mmu: Prevent hugepage promotion for mirror roots in fault
path

Sean Christopherson (22):
x86/tdx: Use pg_level in TDX APIs, not the TDX-Module's 0-based level
KVM: x86/mmu: Update iter->old_spte if cmpxchg64 on mirror SPTE

    "fails"
  KVM: TDX: Account all non-transient page allocations for per-TD
    structures
  KVM: x86: Make "external SPTE" ops that can fail RET0 static calls
  KVM: TDX: Drop kvm_x86_ops.link_external_spt(), use
    .set_external_spte() for all
  KVM: x86/mmu: Fold set_external_spte_present() into its sole caller
  KVM: x86/mmu: Plumb the SPTE _pointer_ into the TDP MMU's
    handle_changed_spte()
  KVM: x86/mmu: Propagate mirror SPTE removal to S-EPT in
    handle_changed_spte()
  KVM: x86: Rework .free_external_spt() into .reclaim_external_sp()
  KVM: Allow owner of kvm_mmu_memory_cache to provide a custom page
    allocator
  KVM: x86/mmu: Allocate/free S-EPT pages using
    tdx_{alloc,free}_control_page()

*** DO NOT MERGE *** x86/virt/tdx: Don't assume guest memory is backed
by struct page
x86/virt/tdx: Extend "reset page" quirk to support huge pages
KVM: x86/mmu: Plumb the old_spte into kvm_x86_ops.set_external_spte()
KVM: TDX: Hoist tdx_sept_remove_private_spte() above
set_private_spte()
KVM: TDX: Handle removal of leaf SPTEs in .set_private_spte()
KVM: TDX: Add helper to handle mapping leaf SPTE into S-EPT
KVM: TDX: Move S-EPT page demotion TODO to tdx_sept_set_private_spte()
KVM: x86/mmu: Add Dynamic PAMT support in TDP MMU for vCPU-induced
page split
KVM: guest_memfd: Add helpers to get start/end gfns give
gmem+slot+pgoff
*** DO NOT MERGE *** KVM: guest_memfd: Add pre-zap arch hook for
shared<=>private conversion
KVM: x86/mmu: Add support for splitting S-EPT hugepages on conversion

Xiaoyao Li (1):
x86/virt/tdx: Add API to demote a 2MB mapping to 512 4KB mappings

Yan Zhao (6):
x86/virt/tdx: Enhance tdh_mem_page_aug() to support huge pages x86/virt/tdx: Enhance tdh_phymem_page_wbinvd_hkid() to invalidate huge
pages
KVM: TDX: Add core support for splitting/demoting 2MiB S-EPT to 4KiB
KVM: x86: Introduce hugepage_set_guest_inhibit()
KVM: TDX: Honor the guest's accept level contained in an EPT violation
KVM: TDX: Turn on PG_LEVEL_2M

 Documentation/arch/x86/tdx.rst              |  21 +
 arch/x86/coco/tdx/tdx.c                     |  10 +-
 arch/x86/include/asm/kvm-x86-ops.h          |   9 +-
 arch/x86/include/asm/kvm_host.h             |  36 +-
 arch/x86/include/asm/shared/tdx.h           |   1 +
 arch/x86/include/asm/shared/tdx_errno.h     | 104 +++
 arch/x86/include/asm/tdx.h                  | 127 ++--
 arch/x86/include/asm/tdx_global_metadata.h  |   1 +
 arch/x86/kvm/Kconfig                        |   1 +
 arch/x86/kvm/mmu.h                          |   4 +
 arch/x86/kvm/mmu/mmu.c                      |  34 +-
 arch/x86/kvm/mmu/mmu_internal.h             |  11 -
 arch/x86/kvm/mmu/tdp_mmu.c                  | 315 ++++----
 arch/x86/kvm/mmu/tdp_mmu.h                  |   2 +
 arch/x86/kvm/vmx/tdx.c                      | 468 +++++++++---
 arch/x86/kvm/vmx/tdx.h                      |   5 +-
 arch/x86/kvm/vmx/tdx_arch.h                 |   3 +
 arch/x86/kvm/vmx/tdx_errno.h                |  40 -
 arch/x86/virt/vmx/tdx/tdx.c                 | 762 +++++++++++++++++---
 arch/x86/virt/vmx/tdx/tdx.h                 |   6 +-
 arch/x86/virt/vmx/tdx/tdx_global_metadata.c |   7 +
 include/linux/kvm_host.h                    |   5 +
 include/linux/kvm_types.h                   |   2 +
 virt/kvm/Kconfig                            |   4 +
 virt/kvm/guest_memfd.c                      |  71 +-
 virt/kvm/kvm_main.c                         |   7 +-
 26 files changed, 1576 insertions(+), 480 deletions(-)
 create mode 100644 arch/x86/include/asm/shared/tdx_errno.h
 delete mode 100644 arch/x86/kvm/vmx/tdx_errno.h

base-commit: e81f7c908e1664233974b9f20beead78cde6343a

Signature · 1 line
2.53.0.rc1.217.geba53bf80e-goog