Skip to contentSkip to conversation

KVM: X86: TDX support

RFC PATCH Applied [email protected] 12 participants 175 messages Jul 2, 2021 – Nov 11, 2021 1 message not in the archive Page 2 of 2

Subsystems (8)
Documentation x86 KVM arm64 +4 more subsystems

v2 of 3 Applied Latest is v3 →

lore
Share & tools
lore
maintainer
#152 lore raw

Re: [RFC PATCH v2 63/69] KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX code

In reply to [email protected]
Quoted 46 lines · On 03/07/21 00:05, [email protected] wrote:

From: Sean Christopherson <[email protected]>

---
  arch/x86/kvm/vmx/common.h | 14 ++++++++++++++
  arch/x86/kvm/vmx/vmx.c    | 10 +---------
  2 files changed, 15 insertions(+), 9 deletions(-)
arch/x86/kvm/vmx/common.h +14 -0
index 755aaec85199..817ff3e74933 100644
@@ -120,6 +120,20 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa,
  	return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0);
  }
 
+static inline u32 __vmx_get_interrupt_shadow(struct kvm_vcpu *vcpu)
+{
+	u32 interruptibility;
+	int ret = 0;
+
+	interruptibility = vmread32(vcpu, GUEST_INTERRUPTIBILITY_INFO);
+	if (interruptibility & GUEST_INTR_STATE_STI)
+		ret |= KVM_X86_SHADOW_INT_STI;
+	if (interruptibility & GUEST_INTR_STATE_MOV_SS)
+		ret |= KVM_X86_SHADOW_INT_MOV_SS;
+
+	return ret;
+}
+
  static inline u32 vmx_encode_ar_bytes(struct kvm_segment *var)
  {
  	u32 ar;
arch/x86/kvm/vmx/vmx.c +0 -3
index d69d4dc7c071..d31cace67907 100644
@@ -1467,15 +1467,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
 
  u32 vmx_get_interrupt_shadow(struct kvm_vcpu *vcpu)
  {
-	u32 interruptibility = vmcs_read32(GUEST_INTERRUPTIBILITY_INFO);
-	int ret = 0;
-

arch/x86/kvm/vmx/vmx.c void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) · line 1473

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +1 -6
-	if (interruptibility & GUEST_INTR_STATE_STI)
-		ret |= KVM_X86_SHADOW_INT_STI;
-	if (interruptibility & GUEST_INTR_STATE_MOV_SS)
-		ret |= KVM_X86_SHADOW_INT_MOV_SS;
-
-	return ret;
+	return __vmx_get_interrupt_shadow(vcpu);
  }
 
  void vmx_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask)

Is there any reason to add the __ version, since at this point kvm_x86_ops is already pointing to vt_get_interrupt_shadow?

Paolo

#153 lore raw

Re: [RFC PATCH v2 63/69] KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX code

Quoted 16 lines · Paolo Bonzini
Quoted 15 lines · On 03/07/21 00:05, [email protected] wrote:
arch/x86/kvm/vmx/vmx.c +0 -8
index d69d4dc7c071..d31cace67907 100644
@@ -1467,15 +1467,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
  u32 vmx_get_interrupt_shadow(struct kvm_vcpu *vcpu)
  {
-	u32 interruptibility = vmcs_read32(GUEST_INTERRUPTIBILITY_INFO);
-	int ret = 0;
-
-	if (interruptibility & GUEST_INTR_STATE_STI)
-		ret |= KVM_X86_SHADOW_INT_STI;
-	if (interruptibility & GUEST_INTR_STATE_MOV_SS)
-		ret |= KVM_X86_SHADOW_INT_MOV_SS;
-
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +1 -1
-	return ret;
+	return __vmx_get_interrupt_shadow(vcpu);
  }
  void vmx_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask)
 

Is there any reason to add the __ version, since at this point kvm_x86_ops is already pointing to vt_get_interrupt_shadow?

Yeah, no idea what I was thinking, the whole thing can be moved as is, just need to delete the prototype in vmx.h.

#154 lore raw

RFC PATCH v2 64/69 cpu/hotplug: Document that TDX also depends on booting CPUs once

In reply to [email protected]

From: Kai Huang <[email protected]>

Add a comment to explain that TDX also depends on booting logical CPUs at least once.

TDSYSINITLP must be run on all CPUs, even software disabled CPUs in the -nosmt case. Fortunately, current SMT handling for #MC already supports booting all CPUs once; the to-be-disabled sibling is booted once (and later put into deep C-state to honor SMT=off) to allow the init code to set CR4.MCE and avoid an unwanted shutdown on a broadcasted MCE.

---
 kernel/cpu.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/cpu.c +4 -0
index e538518556f4..58377a03e9d6 100644
@@ -446,6 +446,10 @@ static inline bool cpu_smt_allowed(unsigned int cpu)
 	 * that the init code can get a chance to set CR4.MCE on each
 	 * CPU. Otherwise, a broadcasted MCE observing CR4.MCE=0b on any
 	 * core will shutdown the machine.
+	 *
+	 * Intel TDX also requires running TDH_SYS_LP_INIT on all logical CPUs
+	 * during boot, booting all CPUs once allows TDX to play nice with
+	 * 'nosmt'.
 	 */
 	return !cpumask_test_cpu(cpu, &cpus_booted_once_mask);
 }
Signature · 1 line
2.25.1
#155 lore raw

RFC PATCH v2 65/69 KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

In reply to [email protected]

From: Xiaoyao Li <[email protected]>

Introduce a per-vm variable initial_tsc_khz to hold the default tsc_khz for kvm_arch_vcpu_create().

This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.

---
 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/x86.c              | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
index a47e17892258..ae8b96e15e71 100644
@@ -1030,6 +1030,7 @@ struct kvm_arch {
 	u64 last_tsc_nsec;
 	u64 last_tsc_write;
 	u32 last_tsc_khz;
+	u32 initial_tsc_khz;
 	u64 cur_tsc_nsec;
 	u64 cur_tsc_write;
 	u64 cur_tsc_offset;
arch/x86/kvm/x86.c +2 -1
index a8299add443f..d3ebed784eac 100644
@@ -10441,7 +10441,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 	else
 		vcpu->arch.mp_state = KVM_MP_STATE_UNINITIALIZED;
 
-	kvm_set_tsc_khz(vcpu, max_tsc_khz);
+	kvm_set_tsc_khz(vcpu, vcpu->kvm->arch.initial_tsc_khz);
 
 	r = kvm_mmu_create(vcpu);
 	if (r < 0)
@@ -10894,6 +10894,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	pvclock_update_vm_gtod_copy(kvm);
 
 	kvm->arch.guest_can_read_msr_platform_info = true;
+	kvm->arch.initial_tsc_khz = max_tsc_khz;
 
 	INIT_DELAYED_WORK(&kvm->arch.kvmclock_update_work, kvmclock_update_fn);
 	INIT_DELAYED_WORK(&kvm->arch.kvmclock_sync_work, kvmclock_sync_fn);
Signature · 1 line
2.25.1
maintainer
#156 lore raw

Re: [RFC PATCH v2 65/69] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

Quoted 8 lines · On 03/07/21 00:05, [email protected] wrote:

From: Xiaoyao Li <[email protected]>

Introduce a per-vm variable initial_tsc_khz to hold the default tsc_khz for kvm_arch_vcpu_create().

This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.

Quoted 6 lines
---
  arch/x86/include/asm/kvm_host.h | 1 +
  arch/x86/kvm/x86.c              | 3 ++-
  2 files changed, 3 insertions(+), 1 deletion(-)

So this means disabling TSC frequency scaling on TDX. Would it make sense to delay VM creation to a separate ioctl, similar to KVM_ARM_VCPU_FINALIZE (KVM_VM_FINALIZE)?

Paolo

Quoted 34 lines
arch/x86/include/asm/kvm_host.h +1 -0
index a47e17892258..ae8b96e15e71 100644
@@ -1030,6 +1030,7 @@ struct kvm_arch {
  	u64 last_tsc_nsec;
  	u64 last_tsc_write;
  	u32 last_tsc_khz;
+	u32 initial_tsc_khz;
  	u64 cur_tsc_nsec;
  	u64 cur_tsc_write;
  	u64 cur_tsc_offset;
arch/x86/kvm/x86.c +2 -1
index a8299add443f..d3ebed784eac 100644
@@ -10441,7 +10441,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
  	else
  		vcpu->arch.mp_state = KVM_MP_STATE_UNINITIALIZED;
 
-	kvm_set_tsc_khz(vcpu, max_tsc_khz);
+	kvm_set_tsc_khz(vcpu, vcpu->kvm->arch.initial_tsc_khz);
 
  	r = kvm_mmu_create(vcpu);
  	if (r < 0)
@@ -10894,6 +10894,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
  	pvclock_update_vm_gtod_copy(kvm);
 
  	kvm->arch.guest_can_read_msr_platform_info = true;
+	kvm->arch.initial_tsc_khz = max_tsc_khz;
 
  	INIT_DELAYED_WORK(&kvm->arch.kvmclock_update_work, kvmclock_update_fn);
  	INIT_DELAYED_WORK(&kvm->arch.kvmclock_sync_work, kvmclock_sync_fn);
#157 lore raw

Re: [RFC PATCH v2 65/69] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

Quoted 16 lines · Paolo Bonzini
Quoted 8 lines · On 03/07/21 00:05, [email protected] wrote:

From: Xiaoyao Li <[email protected]>

Introduce a per-vm variable initial_tsc_khz to hold the default tsc_khz for kvm_arch_vcpu_create().

This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.

Quoted 6 lines
---
  arch/x86/include/asm/kvm_host.h | 1 +
  arch/x86/kvm/x86.c              | 3 ++-
  2 files changed, 3 insertions(+), 1 deletion(-)
Quoted 3 lines

So this means disabling TSC frequency scaling on TDX. Would it make sense to delay VM creation to a separate ioctl, similar to KVM_ARM_VCPU_FINALIZE (KVM_VM_FINALIZE)?

There's an equivalent of that in the next mega-patch, the KVM_TDX_INIT_VM sub-ioctl of KVM_MEMORY_ENCRYPT_OP. The TSC frequency for the guest gets provided at that time.

#158 lore raw

Re: [RFC PATCH v2 65/69] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Paolo Bonzini
Quoted 8 lines · On 03/07/21 00:05, [email protected] wrote:

From: Xiaoyao Li <[email protected]>

Introduce a per-vm variable initial_tsc_khz to hold the default tsc_khz for kvm_arch_vcpu_create().

This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
---
   arch/x86/include/asm/kvm_host.h | 1 +
   arch/x86/kvm/x86.c              | 3 ++-
   2 files changed, 3 insertions(+), 1 deletion(-)

So this means disabling TSC frequency scaling on TDX.

No. It still supports TSC frequency scaling on TDX. Only that we need to configure TSC frequency for TD guest at VM level, not vcpu level.

Quoted 8 lines
Quoted 3 lines

Would it make sense
to delay VM creation to a separate ioctl, similar to KVM_ARM_VCPU_FINALIZE (KVM_VM_FINALIZE)?

There's an equivalent of that in the next mega-patch, the KVM_TDX_INIT_VM sub-ioctl of KVM_MEMORY_ENCRYPT_OP. The TSC frequency for the guest gets provided at that time.

#160 lore raw

RFC PATCH v2 66/69 KVM: TDX: Add "basic" support for building and running Trust Domains

In reply to [email protected]

From: Sean Christopherson <[email protected]>

Add what is effectively a TDX-specific ioctl for initializing the guest Trust Domain. Implement the functionality as a subcommand of KVM_MEMORY_ENCRYPT_OP, analogous to how the ioctl is used by SVM to manage SEV guests.

For easy compatibility with future versions of TDX-SEAM, add a KVM-defined struct, tdx_capabilities, to track requirements/capabilities for the overall system, and define a global instance to serve as the canonical reference.

---
 arch/x86/include/uapi/asm/kvm.h       |   56 +
 arch/x86/include/uapi/asm/vmx.h       |    3 +-
 arch/x86/kvm/mmu.h                    |    2 +-
 arch/x86/kvm/mmu/mmu.c                |    1 +
 arch/x86/kvm/mmu/spte.c               |    5 +-
 arch/x86/kvm/vmx/common.h             |    1 +
 arch/x86/kvm/vmx/main.c               |  375 ++++-
 arch/x86/kvm/vmx/posted_intr.c        |    6 +
 arch/x86/kvm/vmx/tdx.c                | 1942 +++++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.h                |   97 ++
 arch/x86/kvm/vmx/tdx_arch.h           |    7 +
 arch/x86/kvm/vmx/tdx_ops.h            |   13 +
 arch/x86/kvm/vmx/tdx_stubs.c          |   45 +
 arch/x86/kvm/vmx/vmenter.S            |  146 ++
 arch/x86/kvm/x86.c                    |    8 +-
 tools/arch/x86/include/uapi/asm/kvm.h |   51 +
 16 files changed, 2736 insertions(+), 22 deletions(-)
 create mode 100644 arch/x86/kvm/vmx/tdx.c
 create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c
arch/x86/include/uapi/asm/kvm.h +56 -0
index 8341ec720b3f..18950b226d00 100644
@@ -494,4 +494,60 @@ struct kvm_pmu_event_filter {
 #define KVM_X86_SEV_ES_VM	1
 #define KVM_X86_TDX_VM		2
 
+/* Trust Domain eXtension sub-ioctl() commands. */
+enum kvm_tdx_cmd_id {
+	KVM_TDX_CAPABILITIES = 0,
+	KVM_TDX_INIT_VM,
+	KVM_TDX_INIT_VCPU,
+	KVM_TDX_INIT_MEM_REGION,
+	KVM_TDX_FINALIZE_VM,
+
+	KVM_TDX_CMD_NR_MAX,
+};
+
+struct kvm_tdx_cmd {
+	__u32 id;
+	__u32 metadata;
+	__u64 data;
+};
+
+struct kvm_tdx_cpuid_config {
+	__u32 leaf;
+	__u32 sub_leaf;
+	__u32 eax;
+	__u32 ebx;
+	__u32 ecx;
+	__u32 edx;
+};
+
+struct kvm_tdx_capabilities {
+	__u64 attrs_fixed0;
+	__u64 attrs_fixed1;
+	__u64 xfam_fixed0;
+	__u64 xfam_fixed1;
+
+	__u32 nr_cpuid_configs;
+	__u32 padding;
+	struct kvm_tdx_cpuid_config cpuid_configs[0];
+};
+
+struct kvm_tdx_init_vm {
+	__u32 max_vcpus;
+	__u32 tsc_khz;
+	__u64 attributes;
+	__u64 cpuid;
+	__u64 mrconfigid[6];	/* sha384 digest */
+	__u64 mrowner[6];	/* sha384 digest */
+	__u64 mrownerconfig[6];	/* sha348 digest */
+	__u64 reserved[43];	/* must be zero for future extensibility */
+};
+
+#define KVM_TDX_MEASURE_MEMORY_REGION	(1UL << 0)
+
+struct kvm_tdx_init_mem_region {
+	__u64 source_addr;
+	__u64 gpa;
+	__u64 nr_pages;
+};
+
 #endif /* _ASM_X86_KVM_H */
arch/x86/include/uapi/asm/vmx.h +2 -1
index ba5908dfc7c0..79843a0143d2 100644
@@ -32,8 +32,9 @@
 #define EXIT_REASON_EXCEPTION_NMI       0
 #define EXIT_REASON_EXTERNAL_INTERRUPT  1
 #define EXIT_REASON_TRIPLE_FAULT        2
-#define EXIT_REASON_INIT_SIGNAL			3
+#define EXIT_REASON_INIT_SIGNAL         3
 #define EXIT_REASON_SIPI_SIGNAL         4
+#define EXIT_REASON_OTHER_SMI           6
 
 #define EXIT_REASON_INTERRUPT_WINDOW    7
 #define EXIT_REASON_NMI_WINDOW          8
arch/x86/kvm/mmu.h +1 -1
index 8bf1c6dbac78..764283e0979d 100644
@@ -60,7 +60,7 @@ static __always_inline u64 rsvd_bits(int s, int e)
 }
 
 void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask);
-void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only);
+void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only, u64 init_value);
 void kvm_mmu_set_spte_init_value(u64 init_value);
 
 void
arch/x86/kvm/mmu/mmu.c +1 -0
index 4ee6d7803f18..2f8486f79ddb 100644
@@ -5163,6 +5163,7 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu)
 out:
 	return r;
 }
+EXPORT_SYMBOL_GPL(kvm_mmu_load);
 
 static void __kvm_mmu_unload(struct kvm_vcpu *vcpu, u32 roots_to_free)
 {
arch/x86/kvm/mmu/spte.c +3 -2
index 0b931f1c2210..076b489d56d5 100644
@@ -302,14 +302,15 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask)
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_set_mmio_spte_mask);
 
-void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only)
+void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only, u64 init_value)
 {
 	shadow_user_mask	= VMX_EPT_READABLE_MASK;
 	shadow_accessed_mask	= has_ad_bits ? VMX_EPT_ACCESS_BIT : 0ull;
 	shadow_dirty_mask	= has_ad_bits ? VMX_EPT_DIRTY_BIT : 0ull;
 	shadow_nx_mask		= 0ull;
 	shadow_x_mask		= VMX_EPT_EXECUTABLE_MASK;
-	shadow_present_mask	= has_exec_only ? 0ull : VMX_EPT_READABLE_MASK;
+	shadow_present_mask	=
+		(has_exec_only ? 0ull : VMX_EPT_READABLE_MASK) | init_value;
 	shadow_acc_track_mask	= VMX_EPT_RWX_MASK;
 	shadow_me_mask		= 0ull;
 
arch/x86/kvm/vmx/common.h +1 -0
index 817ff3e74933..a99fe68ad687 100644
@@ -9,6 +9,7 @@
 #include <asm/vmx.h>
 
 #include "mmu.h"
+#include "tdx.h"
 #include "vmcs.h"
 #include "vmx.h"
 #include "x86.h"
arch/x86/kvm/vmx/main.c +361 -14

872 lines not on this page · raw message

arch/x86/kvm/vmx/posted_intr.c +6 -0
index 5f81ef092bd4..28d1c252c2e8 100644
@@ -6,6 +6,7 @@
 
 #include "lapic.h"
 #include "posted_intr.h"
+#include "tdx.h"
 #include "trace.h"
 #include "vmx.h"
 
@@ -18,6 +19,11 @@ static DEFINE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock);
 
 static inline struct pi_desc *vcpu_to_pi_desc(struct kvm_vcpu *vcpu)
 {
+#ifdef CONFIG_KVM_INTEL_TDX
+	if (is_td_vcpu(vcpu))
+		return &(to_tdx(vcpu)->pi_desc);
+#endif
+
 	return &(to_vmx(vcpu)->pi_desc);
 }
 
arch/x86/kvm/vmx/tdx.c new file +1942 -0

1,942 lines not on this page · raw message

arch/x86/kvm/vmx/tdx.h +97 -0

59 lines not on this page · raw message

arch/x86/kvm/vmx/tdx_arch.h +7 -0
index 559a63290c4d..7258825b1e02 100644

Lines of this file are not shown. View the raw message.

arch/x86/kvm/vmx/tdx_ops.h +13 -0
index 8afcffa267dc..219473235c97 100644

Lines of this file are not shown. View the raw message.

arch/x86/kvm/vmx/tdx_stubs.c new file +45 -0

45 lines not on this page · raw message

arch/x86/kvm/vmx/vmenter.S +146 -0

146 lines not on this page · raw message

arch/x86/kvm/x86.c +4 -4
index d3ebed784eac..ba69abcc663a 100644

Lines of this file are not shown. View the raw message.

tools/arch/x86/include/uapi/asm/kvm.h +51 -0

51 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.25.1

Part of the diff is not shown here. View the raw message.

A message that is not in the archive (sent off-list or not yet received): CAAYXXYyz3S_cc9ohfkUWN4ohrNq5f+h3608CW5twb-n8i=ogBA@mail.gmail.com

#162 lore raw

Re: [RFC PATCH v2 66/69] KVM: TDX: Add "basic" support for building and running Trust Domains

Quoted 8 lines · Isaku Yamahata

Subject: [RFC PATCH v2 66/69] KVM: TDX: Add "basic" support for building and running Trust Domains

arch/x86/kvm/vmx/tdx.c new file +4 -0
+static int tdx_map_gpa(struct kvm_vcpu *vcpu)
+{
+       gpa_t gpa = tdvmcall_p1_read(vcpu);
+       gpa_t size = tdvmcall_p2_read(vcpu);

arch/x86/kvm/vmx/tdx.c line 914

Quoted 10 lines
arch/x86/kvm/vmx/tdx.c new file +10 -0
+
+       if (!IS_ALIGNED(gpa, 4096) || !IS_ALIGNED(size, 4096) ||
+           (gpa + size) < gpa ||
+           (gpa + size) > vcpu->kvm->arch.gfn_shared_mask << (PAGE_SHIFT + 1))
+               tdvmcall_set_return_code(vcpu, TDG_VP_VMCALL_INVALID_OPERAND);
+       else
+               tdvmcall_set_return_code(vcpu, TDG_VP_VMCALL_SUCCESS);
+
+       return 1;
+}

This function looks like a no op in case of success. Is this intentional? Is this mapping handled somewhere else later on?

#163 lore raw

Re: [RFC PATCH v2 66/69] KVM: TDX: Add "basic" support for building and running Trust Domains

Quoted 15 lines · Sagi Shahar
Quoted 13 lines · Isaku Yamahata

Subject: [RFC PATCH v2 66/69] KVM: TDX: Add "basic" support for building and running Trust Domains

arch/x86/kvm/vmx/tdx.c new file +9 -0
+static int tdx_map_gpa(struct kvm_vcpu *vcpu)
+{
+       gpa_t gpa = tdvmcall_p1_read(vcpu);
+       gpa_t size = tdvmcall_p2_read(vcpu);
+
+       if (!IS_ALIGNED(gpa, 4096) || !IS_ALIGNED(size, 4096) ||
+           (gpa + size) < gpa ||
+           (gpa + size) > vcpu->kvm->arch.gfn_shared_mask << (PAGE_SHIFT + 1))
+               tdvmcall_set_return_code(vcpu, TDG_VP_VMCALL_INVALID_OPERAND);
Quoted 9 lines

arch/x86/kvm/vmx/tdx.c line 761

Quoted 5 lines
arch/x86/kvm/vmx/tdx.c new file +5 -0
+       else
+               tdvmcall_set_return_code(vcpu, TDG_VP_VMCALL_SUCCESS);
+
+       return 1;
+}

This function looks like a no op in case of success. Is this intentional? Is this mapping handled somewhere else later on?

Yes, it's intentional.

The mapping will be exactly set up in EPT violation handler when the GPA is really accessed.

#164 lore raw

RFC PATCH v2 67/69 KVM: TDX: add trace point for TDVMCALL and SEPT operation

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

---
 arch/x86/kvm/trace.h        | 58 +++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.c      | 16 ++++++++++
 arch/x86/kvm/vmx/tdx_arch.h |  9 ++++++
 arch/x86/kvm/x86.c          |  2 ++
 4 files changed, 85 insertions(+)
arch/x86/kvm/trace.h +58 -0
index c3398d0de9a7..58631124f08d 100644
@@ -739,6 +739,64 @@ TRACE_EVENT(kvm_tdx_seamcall_exit,
 		  __entry->r9, __entry->r10, __entry->r11)
 );
 
+/*
+ * Tracepoint for TDVMCALL from a TDX guest
+ */
+TRACE_EVENT(kvm_tdvmcall,
+	TP_PROTO(struct kvm_vcpu *vcpu, __u32 exit_reason,
+		 __u64 p1, __u64 p2, __u64 p3, __u64 p4),
+	TP_ARGS(vcpu, exit_reason, p1, p2, p3, p4),
+
+	TP_STRUCT__entry(
+		__field(	__u64,		rip		)
+		__field(	__u32,		exit_reason	)
+		__field(	__u64,		p1		)
+		__field(	__u64,		p2		)
+		__field(	__u64,		p3		)
+		__field(	__u64,		p4		)
+	),
+
+	TP_fast_assign(
+		__entry->rip			= kvm_rip_read(vcpu);
+		__entry->exit_reason		= exit_reason;
+		__entry->p1			= p1;
+		__entry->p2			= p2;
+		__entry->p3			= p3;
+		__entry->p4			= p4;
+	),
+
+	TP_printk("rip: %llx reason: %s p1: %llx p2: %llx p3: %llx p4: %llx",
+		  __entry->rip,
+		  __print_symbolic(__entry->exit_reason,
+				   TDG_VP_VMCALL_EXIT_REASONS),
+		  __entry->p1, __entry->p2, __entry->p3, __entry->p4)
+);
+
+/*
+ * Tracepoint for SEPT related SEAMCALLs.
+ */
+TRACE_EVENT(kvm_sept_seamcall,
+	TP_PROTO(__u64 op, __u64 gpa, __u64 hpa, int level),
+	TP_ARGS(op, gpa, hpa, level),
+
+	TP_STRUCT__entry(
+		__field(	__u64,		op	)
+		__field(	__u64,		gpa	)
+		__field(	__u64,		hpa	)
+		__field(	int,		level	)
+	),
+
+	TP_fast_assign(
+		__entry->op			= op;
+		__entry->gpa			= gpa;
+		__entry->hpa			= hpa;
+		__entry->level			= level;
+	),
+
+	TP_printk("op: %llu gpa: 0x%llx hpa: 0x%llx level: %u",
+		  __entry->op, __entry->gpa, __entry->hpa, __entry->level)
+);
+
 /*
  * Tracepoint for nested #vmexit because of interrupt pending
  */
arch/x86/kvm/vmx/tdx.c +16 -0
index 1aed4286ce0c..63130fb5a003 100644
@@ -934,6 +934,10 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu)
 
 	exit_reason = tdvmcall_exit_reason(vcpu);
 
+	trace_kvm_tdvmcall(vcpu, exit_reason,
+			   tdvmcall_p1_read(vcpu), tdvmcall_p2_read(vcpu),
+			   tdvmcall_p3_read(vcpu), tdvmcall_p4_read(vcpu));
+
 	switch (exit_reason) {
 	case EXIT_REASON_CPUID:
 		return tdx_emulate_cpuid(vcpu);
@@ -1011,11 +1015,15 @@ static void tdx_sept_set_private_spte(struct kvm_vcpu *vcpu, gfn_t gfn,
 
 	/* Build-time faults are induced and handled via TDH_MEM_PAGE_ADD. */
 	if (is_td_finalized(kvm_tdx)) {
+		trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_PAGE_AUG, gpa, hpa, level);
+
 		err = tdh_mem_page_aug(kvm_tdx->tdr.pa, gpa, hpa, &ex_ret);
 		SEPT_ERR(err, &ex_ret, TDH_MEM_PAGE_AUG, vcpu->kvm);
 		return;
 	}
 
+	trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_PAGE_ADD, gpa, hpa, level);
+
 	source_pa = kvm_tdx->source_pa & ~KVM_TDX_MEASURE_MEMORY_REGION;
 
 	err = tdh_mem_page_add(kvm_tdx->tdr.pa,  gpa, hpa, source_pa, &ex_ret);
@@ -1039,6 +1047,8 @@ static void tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, int level,
 		return;
 
 	if (is_hkid_assigned(kvm_tdx)) {
+		trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_PAGE_REMOVE, gpa, hpa, level);
+
 		err = tdh_mem_page_remove(kvm_tdx->tdr.pa, gpa, level, &ex_ret);
 		if (SEPT_ERR(err, &ex_ret, TDH_MEM_PAGE_REMOVE, kvm))
 			return;
@@ -1063,6 +1073,8 @@ static int tdx_sept_link_private_sp(struct kvm_vcpu *vcpu, gfn_t gfn,
 	struct tdx_ex_ret ex_ret;
 	u64 err;
 
+	trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_SEPT_ADD, gpa, hpa, level);
+
 	err = tdh_mem_spet_add(kvm_tdx->tdr.pa, gpa, level, hpa, &ex_ret);
 	if (SEPT_ERR(err, &ex_ret, TDH_MEM_SEPT_ADD, vcpu->kvm))
 		return -EIO;
@@ -1077,6 +1089,8 @@ static void tdx_sept_zap_private_spte(struct kvm *kvm, gfn_t gfn, int level)
 	struct tdx_ex_ret ex_ret;
 	u64 err;
 
+	trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_RANGE_BLOCK, gpa, -1ull, level);
+
 	err = tdh_mem_range_block(kvm_tdx->tdr.pa, gpa, level, &ex_ret);
 	SEPT_ERR(err, &ex_ret, TDH_MEM_RANGE_BLOCK, kvm);
 }
@@ -1088,6 +1102,8 @@ static void tdx_sept_unzap_private_spte(struct kvm *kvm, gfn_t gfn, int level)
 	struct tdx_ex_ret ex_ret;
 	u64 err;
 
+	trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_RANGE_UNBLOCK, gpa, -1ull, level);
+
 	err = tdh_mem_range_unblock(kvm_tdx->tdr.pa, gpa, level, &ex_ret);
 	SEPT_ERR(err, &ex_ret, TDH_MEM_RANGE_UNBLOCK, kvm);
 }
arch/x86/kvm/vmx/tdx_arch.h +9 -0
index 7258825b1e02..414b933a3b03 100644
@@ -104,6 +104,15 @@
 #define TDG_VP_VMCALL_REPORT_FATAL_ERROR		0x10003
 #define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT	0x10004
 
+#define TDG_VP_VMCALL_EXIT_REASONS				\
+	{ TDG_VP_VMCALL_GET_TD_VM_CALL_INFO,			\
+			"GET_TD_VM_CALL_INFO" },		\
+	{ TDG_VP_VMCALL_MAP_GPA,	"MAP_GPA" },		\
+	{ TDG_VP_VMCALL_GET_QUOTE,	"GET_QUOTE" },		\
+	{ TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT,		\
+			"SETUP_EVENT_NOTIFY_INTERRUPT" },	\
+	VMX_EXIT_REASONS
+
 /* TDX control structure (TDR/TDCS/TDVPS) field access codes */
 #define TDX_CLASS_SHIFT		56
 #define TDX_FIELD_MASK		GENMASK_ULL(31, 0)
arch/x86/kvm/x86.c +2 -0
index ba69abcc663a..ad619c1b2a88 100644
@@ -12104,6 +12104,8 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_cr);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmrun);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmexit);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmexit_inject);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_tdvmcall);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_sept_seamcall);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_intr_vmexit);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmenter_failed);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_invlpga);
Signature · 1 line
2.25.1
maintainer
#165 lore raw

Re: [RFC PATCH v2 67/69] KVM: TDX: add trace point for TDVMCALL and SEPT operation

Quoted 173 lines · On 03/07/21 00:05, [email protected] wrote:

From: Isaku Yamahata <[email protected]>

---
  arch/x86/kvm/trace.h        | 58 +++++++++++++++++++++++++++++++++++++
  arch/x86/kvm/vmx/tdx.c      | 16 ++++++++++
  arch/x86/kvm/vmx/tdx_arch.h |  9 ++++++
  arch/x86/kvm/x86.c          |  2 ++
  4 files changed, 85 insertions(+)
arch/x86/kvm/trace.h +58 -0
index c3398d0de9a7..58631124f08d 100644
@@ -739,6 +739,64 @@ TRACE_EVENT(kvm_tdx_seamcall_exit,
  		  __entry->r9, __entry->r10, __entry->r11)
  );
 
+/*
+ * Tracepoint for TDVMCALL from a TDX guest
+ */
+TRACE_EVENT(kvm_tdvmcall,
+	TP_PROTO(struct kvm_vcpu *vcpu, __u32 exit_reason,
+		 __u64 p1, __u64 p2, __u64 p3, __u64 p4),
+	TP_ARGS(vcpu, exit_reason, p1, p2, p3, p4),
+
+	TP_STRUCT__entry(
+		__field(	__u64,		rip		)
+		__field(	__u32,		exit_reason	)
+		__field(	__u64,		p1		)
+		__field(	__u64,		p2		)
+		__field(	__u64,		p3		)
+		__field(	__u64,		p4		)
+	),
+
+	TP_fast_assign(
+		__entry->rip			= kvm_rip_read(vcpu);
+		__entry->exit_reason		= exit_reason;
+		__entry->p1			= p1;
+		__entry->p2			= p2;
+		__entry->p3			= p3;
+		__entry->p4			= p4;
+	),
+
+	TP_printk("rip: %llx reason: %s p1: %llx p2: %llx p3: %llx p4: %llx",
+		  __entry->rip,
+		  __print_symbolic(__entry->exit_reason,
+				   TDG_VP_VMCALL_EXIT_REASONS),
+		  __entry->p1, __entry->p2, __entry->p3, __entry->p4)
+);
+
+/*
+ * Tracepoint for SEPT related SEAMCALLs.
+ */
+TRACE_EVENT(kvm_sept_seamcall,
+	TP_PROTO(__u64 op, __u64 gpa, __u64 hpa, int level),
+	TP_ARGS(op, gpa, hpa, level),
+
+	TP_STRUCT__entry(
+		__field(	__u64,		op	)
+		__field(	__u64,		gpa	)
+		__field(	__u64,		hpa	)
+		__field(	int,		level	)
+	),
+
+	TP_fast_assign(
+		__entry->op			= op;
+		__entry->gpa			= gpa;
+		__entry->hpa			= hpa;
+		__entry->level			= level;
+	),
+
+	TP_printk("op: %llu gpa: 0x%llx hpa: 0x%llx level: %u",
+		  __entry->op, __entry->gpa, __entry->hpa, __entry->level)
+);
+
  /*
   * Tracepoint for nested #vmexit because of interrupt pending
   */
arch/x86/kvm/vmx/tdx.c +16 -0
index 1aed4286ce0c..63130fb5a003 100644
@@ -934,6 +934,10 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu)
 
  	exit_reason = tdvmcall_exit_reason(vcpu);
 
+	trace_kvm_tdvmcall(vcpu, exit_reason,
+			   tdvmcall_p1_read(vcpu), tdvmcall_p2_read(vcpu),
+			   tdvmcall_p3_read(vcpu), tdvmcall_p4_read(vcpu));
+
  	switch (exit_reason) {
  	case EXIT_REASON_CPUID:
  		return tdx_emulate_cpuid(vcpu);
@@ -1011,11 +1015,15 @@ static void tdx_sept_set_private_spte(struct kvm_vcpu *vcpu, gfn_t gfn,
 
  	/* Build-time faults are induced and handled via TDH_MEM_PAGE_ADD. */
  	if (is_td_finalized(kvm_tdx)) {
+		trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_PAGE_AUG, gpa, hpa, level);
+
  		err = tdh_mem_page_aug(kvm_tdx->tdr.pa, gpa, hpa, &ex_ret);
  		SEPT_ERR(err, &ex_ret, TDH_MEM_PAGE_AUG, vcpu->kvm);
  		return;
  	}
 
+	trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_PAGE_ADD, gpa, hpa, level);
+
  	source_pa = kvm_tdx->source_pa & ~KVM_TDX_MEASURE_MEMORY_REGION;
 
  	err = tdh_mem_page_add(kvm_tdx->tdr.pa,  gpa, hpa, source_pa, &ex_ret);
@@ -1039,6 +1047,8 @@ static void tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, int level,
  		return;
 
  	if (is_hkid_assigned(kvm_tdx)) {
+		trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_PAGE_REMOVE, gpa, hpa, level);
+
  		err = tdh_mem_page_remove(kvm_tdx->tdr.pa, gpa, level, &ex_ret);
  		if (SEPT_ERR(err, &ex_ret, TDH_MEM_PAGE_REMOVE, kvm))
  			return;
@@ -1063,6 +1073,8 @@ static int tdx_sept_link_private_sp(struct kvm_vcpu *vcpu, gfn_t gfn,
  	struct tdx_ex_ret ex_ret;
  	u64 err;
 
+	trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_SEPT_ADD, gpa, hpa, level);
+
  	err = tdh_mem_spet_add(kvm_tdx->tdr.pa, gpa, level, hpa, &ex_ret);
  	if (SEPT_ERR(err, &ex_ret, TDH_MEM_SEPT_ADD, vcpu->kvm))
  		return -EIO;
@@ -1077,6 +1089,8 @@ static void tdx_sept_zap_private_spte(struct kvm *kvm, gfn_t gfn, int level)
  	struct tdx_ex_ret ex_ret;
  	u64 err;
 
+	trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_RANGE_BLOCK, gpa, -1ull, level);
+
  	err = tdh_mem_range_block(kvm_tdx->tdr.pa, gpa, level, &ex_ret);
  	SEPT_ERR(err, &ex_ret, TDH_MEM_RANGE_BLOCK, kvm);
  }
@@ -1088,6 +1102,8 @@ static void tdx_sept_unzap_private_spte(struct kvm *kvm, gfn_t gfn, int level)
  	struct tdx_ex_ret ex_ret;
  	u64 err;
 
+	trace_kvm_sept_seamcall(SEAMCALL_TDH_MEM_RANGE_UNBLOCK, gpa, -1ull, level);
+
  	err = tdh_mem_range_unblock(kvm_tdx->tdr.pa, gpa, level, &ex_ret);
  	SEPT_ERR(err, &ex_ret, TDH_MEM_RANGE_UNBLOCK, kvm);
  }
arch/x86/kvm/vmx/tdx_arch.h +9 -0
index 7258825b1e02..414b933a3b03 100644
@@ -104,6 +104,15 @@
  #define TDG_VP_VMCALL_REPORT_FATAL_ERROR		0x10003
  #define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT	0x10004
 
+#define TDG_VP_VMCALL_EXIT_REASONS				\
+	{ TDG_VP_VMCALL_GET_TD_VM_CALL_INFO,			\
+			"GET_TD_VM_CALL_INFO" },		\
+	{ TDG_VP_VMCALL_MAP_GPA,	"MAP_GPA" },		\
+	{ TDG_VP_VMCALL_GET_QUOTE,	"GET_QUOTE" },		\
+	{ TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT,		\
+			"SETUP_EVENT_NOTIFY_INTERRUPT" },	\
+	VMX_EXIT_REASONS
+
  /* TDX control structure (TDR/TDCS/TDVPS) field access codes */
  #define TDX_CLASS_SHIFT		56
  #define TDX_FIELD_MASK		GENMASK_ULL(31, 0)
arch/x86/kvm/x86.c +0 -0
index ba69abcc663a..ad619c1b2a88 100644

arch/x86/kvm/x86.c EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_cr); · line 12111

Quoted 8 lines
arch/x86/kvm/x86.c +2 -0
@@ -12104,6 +12104,8 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_cr);
  EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmrun);
  EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmexit);
  EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmexit_inject);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_tdvmcall);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_sept_seamcall);
  EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_intr_vmexit);
  EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmenter_failed);
  EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_invlpga);

Please split this in two parts for each tracepoint, and squash it in the earlier patches that introduced handle_tdvmcall and tdx_sept_*.

Paolo

#166 lore raw

RFC PATCH v2 68/69 KVM: TDX: add document on TDX MODULE

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

Add a document on how to integrate TDX MODULE into initrd so that TDX MODULE can be updated on kernel startup.

---
 Documentation/virt/kvm/tdx-module.rst | 48 +++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
 create mode 100644 Documentation/virt/kvm/tdx-module.rst
Documentation/virt/kvm/tdx-module.rst new file +48 -0
new file mode 100644
index 000000000000..8beea8302f94
@@ -0,0 +1,48 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==========
+TDX MODULE
+==========
+
+Integrating TDX MODULE into initrd
+==================================
+If TDX is enabled in KVM(CONFIG_KVM_INTEL_TDX=y), kernel is able to load
+tdx seam module from initrd.
+The related modules (seamldr.ac, libtdx.so and libtdx.so.sigstruct) need to be
+stored in initrd.
+
+tdx-seam is a sample hook script for initramfs-tools.
+TDXSEAM_SRCDIR are the directory in the host file system to store files related
+to TDX MODULE.
+
+Since it heavily depends on distro how to prepare initrd, here's a example how
+to prepare an initrd.
+(Actually this is taken from Documentation/x86/microcode.rst)
+::
+  #!/bin/bash
+
+  if [ -z "$1" ]; then
+      echo "You need to supply an initrd file"
+      exit 1
+  fi
+
+  INITRD="$1"
+
+  DSTDIR=lib/firmware/intel-seam
+  TMPDIR=/tmp/initrd
+  LIBTDX="/lib/firmware/intel-seam/seamldr.acm /lib/firmware/intel-seam/libtdx.so /lib/firmware/intel-seam/libtdx.so.sigstruct"
+
+  rm -rf $TMPDIR
+
+  mkdir $TMPDIR
+  cd $TMPDIR
+  mkdir -p $DSTDIR
+
+  cp ${LIBTDX} ${DSTDIR}
+
+  find . | cpio -o -H newc > ../tdx-seam.cpio
+  cd ..
+  mv $INITRD $INITRD.orig
+  cat tdx-seam.cpio $INITRD.orig > $INITRD
+
+  rm -rf $TMPDIR
Signature · 1 line
2.25.1
maintainer
#167 lore raw

Re: [RFC PATCH v2 68/69] KVM: TDX: add document on TDX MODULE

Quoted 56 lines · On 03/07/21 00:05, [email protected] wrote:

From: Isaku Yamahata <[email protected]>

Add a document on how to integrate TDX MODULE into initrd so that TDX MODULE can be updated on kernel startup.

---
  Documentation/virt/kvm/tdx-module.rst | 48 +++++++++++++++++++++++++++
  1 file changed, 48 insertions(+)

create mode 100644 Documentation/virt/kvm/tdx-module.rst

Documentation/virt/kvm/tdx-module.rst new file +38 -0
new file mode 100644
index 000000000000..8beea8302f94
@@ -0,0 +1,48 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==========
+TDX MODULE
+==========
+
+Integrating TDX MODULE into initrd
+==================================
+If TDX is enabled in KVM(CONFIG_KVM_INTEL_TDX=y), kernel is able to load
+tdx seam module from initrd.
+The related modules (seamldr.ac, libtdx.so and libtdx.so.sigstruct) need to be
+stored in initrd.
+
+tdx-seam is a sample hook script for initramfs-tools.
+TDXSEAM_SRCDIR are the directory in the host file system to store files related
+to TDX MODULE.
+
+Since it heavily depends on distro how to prepare initrd, here's a example how
+to prepare an initrd.
+(Actually this is taken from Documentation/x86/microcode.rst)
+::
+  #!/bin/bash
+
+  if [ -z "$1" ]; then
+      echo "You need to supply an initrd file"
+      exit 1
+  fi
+
+  INITRD="$1"
+
+  DSTDIR=lib/firmware/intel-seam
+  TMPDIR=/tmp/initrd
+  LIBTDX="/lib/firmware/intel-seam/seamldr.acm /lib/firmware/intel-seam/libtdx.so /lib/firmware/intel-seam/libtdx.so.sigstruct"
+
+  rm -rf $TMPDIR
+
+  mkdir $TMPDIR
+  cd $TMPDIR

Documentation/virt/kvm/tdx-module.rst line 48

Quoted 10 lines
Documentation/virt/kvm/tdx-module.rst new file +10 -0
+  mkdir -p $DSTDIR
+
+  cp ${LIBTDX} ${DSTDIR}
+
+  find . | cpio -o -H newc > ../tdx-seam.cpio
+  cd ..
+  mv $INITRD $INITRD.orig
+  cat tdx-seam.cpio $INITRD.orig > $INITRD
+
+  rm -rf $TMPDIR

I think this belongs in a different series that adds SEAM loading?

Paolo

#168 lore raw

RFC PATCH v2 69/69 Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)

In reply to [email protected]

From: Isaku Yamahata <[email protected]>

Add a documentation to Intel Trusted Docmain Extensions(TDX) support.

---
 Documentation/virt/kvm/api.rst       |   6 +-
 Documentation/virt/kvm/intel-tdx.rst | 441 +++++++++++++++++++++++++++
 2 files changed, 446 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/virt/kvm/intel-tdx.rst
Documentation/virt/kvm/api.rst +5 -1
index 7fcb2fd38f42..b6a33a9bac87 100644
@@ -4297,7 +4297,7 @@ H_GET_CPU_CHARACTERISTICS hypercall.
 
 :Capability: basic
 :Architectures: x86
-:Type: vm
+:Type: vm ioctl, vcpu ioctl
 :Parameters: an opaque platform specific structure (in/out)
 :Returns: 0 on success; -1 on error
 
@@ -4309,6 +4309,10 @@ Currently, this ioctl is used for issuing Secure Encrypted Virtualization
 (SEV) commands on AMD Processors. The SEV commands are defined in
 Documentation/virt/kvm/amd-memory-encryption.rst.
 
+Currently, this ioctl is used for issuing Trusted Domain Extensions
+(TDX) commands on Intel Processors. The TDX commands are defined in
+Documentation/virt/kvm/intel-tdx.rst.
+
 4.111 KVM_MEMORY_ENCRYPT_REG_REGION
 -----------------------------------
 
Documentation/virt/kvm/intel-tdx.rst new file +441 -0

441 lines not on this page · raw message

Signature · 1 line
2.25.1
maintainer
#169 lore raw
In reply to [email protected]

Based on the initial review, I think patches 2-3-17-18-19-20-23-49 can already be merged for 5.15.

The next part should be the introduction of vm_types, blocking ioctls depending on the vm_type (patches 24-31) and possibly applying this block already to SEV-ES.

Quoted 234 lines · On 03/07/21 00:04, [email protected] wrote:

From: Isaku Yamahata <[email protected]>

* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform. [1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.

* The goal of this RFC patch
The purpose of this post is to get feedback early on high level design issue of KVM enhancement for TDX. The detailed coding (variable naming etc) is not cared of. This patch series is incomplete (not working). So it's RFC. Although multiple software components, not only KVM but also QEMU, guest Linux and virtual bios, need to be updated, this includes only KVM VMM part. For those who are curious to changes to other component, there are public repositories at github. [8], [9]

* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.

- 01-12: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 13-40: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 41-65: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 66: main patch to add "basic" support for building/running TDX.
- 67: trace points for
- 68-69: Documentation

* TODOs
Those major features are missing from this patch series to keep this patch series small.

- load/initialize TDX module
split out from this patch series.
- unmapping private page
Will integrate Kirill's patch to show how kvm will utilize it.
- qemu gdb stub support
- Large page support
- guest PMU support
- TDP MMU support
- and more

Changes from v1:
- rebase to v5.13
- drop load/initialization of TDX module
- catch up the update of related specifications.
- rework on C-wrapper function to invoke seamcall
- various code clean up

[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAS

    https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf

[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVF

Isaku Yamahata (11):
KVM: TDX: introduce config for KVM TDX support
KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h
KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
KVM: TDX: add a helper function for kvm to call seamcall
KVM: TDX: add trace point before/after TDX SEAMCALLs
KVM: TDX: Print the name of SEAMCALL status code
KVM: Add per-VM flag to mark read-only memory as unsupported
KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
KVM: TDX: add trace point for TDVMCALL and SEPT operation
KVM: TDX: add document on TDX MODULE Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)

Kai Huang (2):
KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level
routes
cpu/hotplug: Document that TDX also depends on booting CPUs once

Rick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bits

Sean Christopherson (53):
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add architectural definitions for structures and values
KVM: TDX: define and export helper functions for KVM TDX support
KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Split core of hypercall emulation to helper function
KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO
KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
KVM: Add infrastructure and macro to mark VM as bugged
KVM: Export kvm_make_all_cpus_request() for use in marking VMs as
bugged
KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the
VM
KVM: x86/mmu: Mark VM as bugged if page fault returns RET_PF_INVALID
KVM: Add max_vcpus field in common 'struct kvm'
KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
KVM: x86: Hoist kvm_dirty_regs check out of sync_regs()
KVM: x86: Introduce "protected guest" concept and block disallowed
ioctls
KVM: x86: Add per-VM flag to disable direct IRQ injection
KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE
KVM: x86: Add flag to mark TSC as immutable (for TDX)
KVM: Add per-VM flag to disable dirty logging of memslots for TDs
KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()
KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched
behavior
KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: x86: Add guest_supported_xss placholder
KVM: Export kvm_is_reserved_pfn() for use by TDX
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Refactor shadow walk in __direct_map() to reduce
indentation
KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()
KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
KVM: x86/mmu: Move 'pfn' variable to caller of direct_page_fault()
KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
KVM: VMX: Move NMI/exception handler to common helper
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Define EPT Violation architectural bits
KVM: VMX: Define VMCS encodings for shared EPT pointer
KVM: VMX: Add 'main.c' to wrap VMX and TDX
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: VMX: Move register caching logic to common code
KVM: TDX: Define TDCALL exit reason
KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h
KVM: VMX: MOVE GDT and IDT accessors to common code
KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX
code
KVM: TDX: Add "basic" support for building and running Trust Domains

Xiaoyao Li (2):
KVM: TDX: Introduce pr_seamcall_ex_ret_info() to print more info when
SEAMCALL fails
KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

  Documentation/virt/kvm/api.rst        |    6 +-
  Documentation/virt/kvm/intel-tdx.rst  |  441 ++++++
  Documentation/virt/kvm/tdx-module.rst |   48 +
  arch/arm64/include/asm/kvm_host.h     |    3 -
  arch/arm64/kvm/arm.c                  |    7 +-
  arch/arm64/kvm/vgic/vgic-init.c       |    6 +-
  arch/x86/Kbuild                       |    1 +
  arch/x86/include/asm/cpufeatures.h    |    2 +
  arch/x86/include/asm/kvm-x86-ops.h    |    8 +
  arch/x86/include/asm/kvm_boot.h       |   30 +
  arch/x86/include/asm/kvm_host.h       |   55 +-
  arch/x86/include/asm/virtext.h        |   25 +
  arch/x86/include/asm/vmx.h            |   17 +
  arch/x86/include/uapi/asm/kvm.h       |   60 +
  arch/x86/include/uapi/asm/vmx.h       |    7 +-
  arch/x86/kernel/asm-offsets_64.c      |   15 +
  arch/x86/kvm/Kconfig                  |   11 +
  arch/x86/kvm/Makefile                 |    3 +-
  arch/x86/kvm/boot/Makefile            |    6 +
  arch/x86/kvm/boot/seam/tdx_common.c   |  242 +++
  arch/x86/kvm/boot/seam/tdx_common.h   |   13 +
  arch/x86/kvm/ioapic.c                 |    4 +
  arch/x86/kvm/irq_comm.c               |   13 +-
  arch/x86/kvm/lapic.c                  |    7 +-
  arch/x86/kvm/lapic.h                  |    2 +-
  arch/x86/kvm/mmu.h                    |   31 +-
  arch/x86/kvm/mmu/mmu.c                |  526 +++++--
  arch/x86/kvm/mmu/mmu_internal.h       |    3 +
  arch/x86/kvm/mmu/paging_tmpl.h        |   25 +-
  arch/x86/kvm/mmu/spte.c               |   15 +-
  arch/x86/kvm/mmu/spte.h               |   18 +-
  arch/x86/kvm/svm/svm.c                |   18 +-
  arch/x86/kvm/trace.h                  |  138 ++
  arch/x86/kvm/vmx/common.h             |  178 +++
  arch/x86/kvm/vmx/main.c               | 1098 ++++++++++++++
  arch/x86/kvm/vmx/posted_intr.c        |    6 +
  arch/x86/kvm/vmx/seamcall.S           |   64 +
  arch/x86/kvm/vmx/seamcall.h           |   68 +
  arch/x86/kvm/vmx/tdx.c                | 1958 +++++++++++++++++++++++++
  arch/x86/kvm/vmx/tdx.h                |  267 ++++
  arch/x86/kvm/vmx/tdx_arch.h           |  370 +++++
  arch/x86/kvm/vmx/tdx_errno.h          |  202 +++
  arch/x86/kvm/vmx/tdx_ops.h            |  218 +++
  arch/x86/kvm/vmx/tdx_stubs.c          |   45 +
  arch/x86/kvm/vmx/vmcs.h               |   11 -
  arch/x86/kvm/vmx/vmenter.S            |  146 ++
  arch/x86/kvm/vmx/vmx.c                |  509 ++-----
  arch/x86/kvm/x86.c                    |  285 +++-
  include/linux/kvm_host.h              |   51 +-
  include/uapi/linux/kvm.h              |    2 +
  kernel/cpu.c                          |    4 +
  tools/arch/x86/include/uapi/asm/kvm.h |   55 +
  tools/include/uapi/linux/kvm.h        |    2 +
  virt/kvm/kvm_main.c                   |   44 +-
  54 files changed, 6717 insertions(+), 672 deletions(-)

create mode 100644 Documentation/virt/kvm/intel-tdx.rst create mode 100644 Documentation/virt/kvm/tdx-module.rst create mode 100644 arch/x86/include/asm/kvm_boot.h create mode 100644 arch/x86/kvm/boot/Makefile create mode 100644 arch/x86/kvm/boot/seam/tdx_common.c create mode 100644 arch/x86/kvm/boot/seam/tdx_common.h create mode 100644 arch/x86/kvm/vmx/common.h create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/seamcall.S create mode 100644 arch/x86/kvm/vmx/seamcall.h create mode 100644 arch/x86/kvm/vmx/tdx.c create mode 100644 arch/x86/kvm/vmx/tdx.h create mode 100644 arch/x86/kvm/vmx/tdx_arch.h create mode 100644 arch/x86/kvm/vmx/tdx_errno.h create mode 100644 arch/x86/kvm/vmx/tdx_ops.h create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c

maintainer
#170 lore raw
In reply to [email protected]

Based on the initial review, I think patches 2-3-17-18-19-20-23-49 can already be merged for 5.15.

The next part should be the introduction of vm_types, blocking ioctls depending on the vm_type (patches 24-31). Perhaps this blocking should be applied already to SEV-ES, so that the corresponding code in QEMU can be added early.

Paolo

Quoted 234 lines · On 03/07/21 00:04, [email protected] wrote:

From: Isaku Yamahata <[email protected]>

* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform. [1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.

* The goal of this RFC patch
The purpose of this post is to get feedback early on high level design issue of KVM enhancement for TDX. The detailed coding (variable naming etc) is not cared of. This patch series is incomplete (not working). So it's RFC. Although multiple software components, not only KVM but also QEMU, guest Linux and virtual bios, need to be updated, this includes only KVM VMM part. For those who are curious to changes to other component, there are public repositories at github. [8], [9]

* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.

- 01-12: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 13-40: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 41-65: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 66: main patch to add "basic" support for building/running TDX.
- 67: trace points for
- 68-69: Documentation

* TODOs
Those major features are missing from this patch series to keep this patch series small.

- load/initialize TDX module
split out from this patch series.
- unmapping private page
Will integrate Kirill's patch to show how kvm will utilize it.
- qemu gdb stub support
- Large page support
- guest PMU support
- TDP MMU support
- and more

Changes from v1:
- rebase to v5.13
- drop load/initialization of TDX module
- catch up the update of related specifications.
- rework on C-wrapper function to invoke seamcall
- various code clean up

[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAS

    https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf

[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVF

Isaku Yamahata (11):
KVM: TDX: introduce config for KVM TDX support
KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h
KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
KVM: TDX: add a helper function for kvm to call seamcall
KVM: TDX: add trace point before/after TDX SEAMCALLs
KVM: TDX: Print the name of SEAMCALL status code
KVM: Add per-VM flag to mark read-only memory as unsupported
KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
KVM: TDX: add trace point for TDVMCALL and SEPT operation
KVM: TDX: add document on TDX MODULE Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)

Kai Huang (2):
KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level
routes
cpu/hotplug: Document that TDX also depends on booting CPUs once

Rick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bits

Sean Christopherson (53):
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add architectural definitions for structures and values
KVM: TDX: define and export helper functions for KVM TDX support
KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Split core of hypercall emulation to helper function
KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO
KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
KVM: Add infrastructure and macro to mark VM as bugged
KVM: Export kvm_make_all_cpus_request() for use in marking VMs as
bugged
KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the
VM
KVM: x86/mmu: Mark VM as bugged if page fault returns RET_PF_INVALID
KVM: Add max_vcpus field in common 'struct kvm'
KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
KVM: x86: Hoist kvm_dirty_regs check out of sync_regs()
KVM: x86: Introduce "protected guest" concept and block disallowed
ioctls
KVM: x86: Add per-VM flag to disable direct IRQ injection
KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE
KVM: x86: Add flag to mark TSC as immutable (for TDX)
KVM: Add per-VM flag to disable dirty logging of memslots for TDs
KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()
KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched
behavior
KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: x86: Add guest_supported_xss placholder
KVM: Export kvm_is_reserved_pfn() for use by TDX
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Refactor shadow walk in __direct_map() to reduce
indentation
KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()
KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
KVM: x86/mmu: Move 'pfn' variable to caller of direct_page_fault()
KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
KVM: VMX: Move NMI/exception handler to common helper
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Define EPT Violation architectural bits
KVM: VMX: Define VMCS encodings for shared EPT pointer
KVM: VMX: Add 'main.c' to wrap VMX and TDX
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: VMX: Move register caching logic to common code
KVM: TDX: Define TDCALL exit reason
KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h
KVM: VMX: MOVE GDT and IDT accessors to common code
KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX
code
KVM: TDX: Add "basic" support for building and running Trust Domains

Xiaoyao Li (2):
KVM: TDX: Introduce pr_seamcall_ex_ret_info() to print more info when
SEAMCALL fails
KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

  Documentation/virt/kvm/api.rst        |    6 +-
  Documentation/virt/kvm/intel-tdx.rst  |  441 ++++++
  Documentation/virt/kvm/tdx-module.rst |   48 +
  arch/arm64/include/asm/kvm_host.h     |    3 -
  arch/arm64/kvm/arm.c                  |    7 +-
  arch/arm64/kvm/vgic/vgic-init.c       |    6 +-
  arch/x86/Kbuild                       |    1 +
  arch/x86/include/asm/cpufeatures.h    |    2 +
  arch/x86/include/asm/kvm-x86-ops.h    |    8 +
  arch/x86/include/asm/kvm_boot.h       |   30 +
  arch/x86/include/asm/kvm_host.h       |   55 +-
  arch/x86/include/asm/virtext.h        |   25 +
  arch/x86/include/asm/vmx.h            |   17 +
  arch/x86/include/uapi/asm/kvm.h       |   60 +
  arch/x86/include/uapi/asm/vmx.h       |    7 +-
  arch/x86/kernel/asm-offsets_64.c      |   15 +
  arch/x86/kvm/Kconfig                  |   11 +
  arch/x86/kvm/Makefile                 |    3 +-
  arch/x86/kvm/boot/Makefile            |    6 +
  arch/x86/kvm/boot/seam/tdx_common.c   |  242 +++
  arch/x86/kvm/boot/seam/tdx_common.h   |   13 +
  arch/x86/kvm/ioapic.c                 |    4 +
  arch/x86/kvm/irq_comm.c               |   13 +-
  arch/x86/kvm/lapic.c                  |    7 +-
  arch/x86/kvm/lapic.h                  |    2 +-
  arch/x86/kvm/mmu.h                    |   31 +-
  arch/x86/kvm/mmu/mmu.c                |  526 +++++--
  arch/x86/kvm/mmu/mmu_internal.h       |    3 +
  arch/x86/kvm/mmu/paging_tmpl.h        |   25 +-
  arch/x86/kvm/mmu/spte.c               |   15 +-
  arch/x86/kvm/mmu/spte.h               |   18 +-
  arch/x86/kvm/svm/svm.c                |   18 +-
  arch/x86/kvm/trace.h                  |  138 ++
  arch/x86/kvm/vmx/common.h             |  178 +++
  arch/x86/kvm/vmx/main.c               | 1098 ++++++++++++++
  arch/x86/kvm/vmx/posted_intr.c        |    6 +
  arch/x86/kvm/vmx/seamcall.S           |   64 +
  arch/x86/kvm/vmx/seamcall.h           |   68 +
  arch/x86/kvm/vmx/tdx.c                | 1958 +++++++++++++++++++++++++
  arch/x86/kvm/vmx/tdx.h                |  267 ++++
  arch/x86/kvm/vmx/tdx_arch.h           |  370 +++++
  arch/x86/kvm/vmx/tdx_errno.h          |  202 +++
  arch/x86/kvm/vmx/tdx_ops.h            |  218 +++
  arch/x86/kvm/vmx/tdx_stubs.c          |   45 +
  arch/x86/kvm/vmx/vmcs.h               |   11 -
  arch/x86/kvm/vmx/vmenter.S            |  146 ++
  arch/x86/kvm/vmx/vmx.c                |  509 ++-----
  arch/x86/kvm/x86.c                    |  285 +++-
  include/linux/kvm_host.h              |   51 +-
  include/uapi/linux/kvm.h              |    2 +
  kernel/cpu.c                          |    4 +
  tools/arch/x86/include/uapi/asm/kvm.h |   55 +
  tools/include/uapi/linux/kvm.h        |    2 +
  virt/kvm/kvm_main.c                   |   44 +-
  54 files changed, 6717 insertions(+), 672 deletions(-)

create mode 100644 Documentation/virt/kvm/intel-tdx.rst create mode 100644 Documentation/virt/kvm/tdx-module.rst create mode 100644 arch/x86/include/asm/kvm_boot.h create mode 100644 arch/x86/kvm/boot/Makefile create mode 100644 arch/x86/kvm/boot/seam/tdx_common.c create mode 100644 arch/x86/kvm/boot/seam/tdx_common.h create mode 100644 arch/x86/kvm/vmx/common.h create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/seamcall.S create mode 100644 arch/x86/kvm/vmx/seamcall.h create mode 100644 arch/x86/kvm/vmx/tdx.c create mode 100644 arch/x86/kvm/vmx/tdx.h create mode 100644 arch/x86/kvm/vmx/tdx_arch.h create mode 100644 arch/x86/kvm/vmx/tdx_errno.h create mode 100644 arch/x86/kvm/vmx/tdx_ops.h create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c

maintainer Applied
#171 lore raw
In reply to [email protected]
Quoted 23 lines · On 03/07/21 00:04, [email protected] wrote:

From: Isaku Yamahata <[email protected]>

* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform. [1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.

* The goal of this RFC patch
The purpose of this post is to get feedback early on high level design issue of KVM enhancement for TDX. The detailed coding (variable naming etc) is not cared of. This patch series is incomplete (not working). So it's RFC. Although multiple software components, not only KVM but also QEMU, guest Linux and virtual bios, need to be updated, this includes only KVM VMM part. For those who are curious to changes to other component, there are public repositories at github. [8], [9]

* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.

Quoted 9 lines

- 01-12: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 13-40: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 41-65: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 66: main patch to add "basic" support for building/running TDX.
- 67: trace points for
- 68-69: Documentation

Queued 2,3,17-20,23,44-45, thanks.

Paolo

Quoted 201 lines

* TODOs
Those major features are missing from this patch series to keep this patch series small.

- load/initialize TDX module
split out from this patch series.
- unmapping private page
Will integrate Kirill's patch to show how kvm will utilize it.
- qemu gdb stub support
- Large page support
- guest PMU support
- TDP MMU support
- and more

Changes from v1:
- rebase to v5.13
- drop load/initialization of TDX module
- catch up the update of related specifications.
- rework on C-wrapper function to invoke seamcall
- various code clean up

[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAS

    https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf

[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVF

Isaku Yamahata (11):
KVM: TDX: introduce config for KVM TDX support
KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h
KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
KVM: TDX: add a helper function for kvm to call seamcall
KVM: TDX: add trace point before/after TDX SEAMCALLs
KVM: TDX: Print the name of SEAMCALL status code
KVM: Add per-VM flag to mark read-only memory as unsupported
KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
KVM: TDX: add trace point for TDVMCALL and SEPT operation
KVM: TDX: add document on TDX MODULE Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)

Kai Huang (2):
KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level
routes
cpu/hotplug: Document that TDX also depends on booting CPUs once

Rick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bits

Sean Christopherson (53):
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add architectural definitions for structures and values
KVM: TDX: define and export helper functions for KVM TDX support
KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Split core of hypercall emulation to helper function
KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO
KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
KVM: Add infrastructure and macro to mark VM as bugged
KVM: Export kvm_make_all_cpus_request() for use in marking VMs as
bugged
KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the
VM
KVM: x86/mmu: Mark VM as bugged if page fault returns RET_PF_INVALID
KVM: Add max_vcpus field in common 'struct kvm'
KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
KVM: x86: Hoist kvm_dirty_regs check out of sync_regs()
KVM: x86: Introduce "protected guest" concept and block disallowed
ioctls
KVM: x86: Add per-VM flag to disable direct IRQ injection
KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE
KVM: x86: Add flag to mark TSC as immutable (for TDX)
KVM: Add per-VM flag to disable dirty logging of memslots for TDs
KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()
KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched
behavior
KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: x86: Add guest_supported_xss placholder
KVM: Export kvm_is_reserved_pfn() for use by TDX
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Refactor shadow walk in __direct_map() to reduce
indentation
KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()
KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
KVM: x86/mmu: Move 'pfn' variable to caller of direct_page_fault()
KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
KVM: VMX: Move NMI/exception handler to common helper
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Define EPT Violation architectural bits
KVM: VMX: Define VMCS encodings for shared EPT pointer
KVM: VMX: Add 'main.c' to wrap VMX and TDX
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: VMX: Move register caching logic to common code
KVM: TDX: Define TDCALL exit reason
KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h
KVM: VMX: MOVE GDT and IDT accessors to common code
KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX
code
KVM: TDX: Add "basic" support for building and running Trust Domains

Xiaoyao Li (2):
KVM: TDX: Introduce pr_seamcall_ex_ret_info() to print more info when
SEAMCALL fails
KVM: X86: Introduce initial_tsc_khz in struct kvm_arch

  Documentation/virt/kvm/api.rst        |    6 +-
  Documentation/virt/kvm/intel-tdx.rst  |  441 ++++++
  Documentation/virt/kvm/tdx-module.rst |   48 +
  arch/arm64/include/asm/kvm_host.h     |    3 -
  arch/arm64/kvm/arm.c                  |    7 +-
  arch/arm64/kvm/vgic/vgic-init.c       |    6 +-
  arch/x86/Kbuild                       |    1 +
  arch/x86/include/asm/cpufeatures.h    |    2 +
  arch/x86/include/asm/kvm-x86-ops.h    |    8 +
  arch/x86/include/asm/kvm_boot.h       |   30 +
  arch/x86/include/asm/kvm_host.h       |   55 +-
  arch/x86/include/asm/virtext.h        |   25 +
  arch/x86/include/asm/vmx.h            |   17 +
  arch/x86/include/uapi/asm/kvm.h       |   60 +
  arch/x86/include/uapi/asm/vmx.h       |    7 +-
  arch/x86/kernel/asm-offsets_64.c      |   15 +
  arch/x86/kvm/Kconfig                  |   11 +
  arch/x86/kvm/Makefile                 |    3 +-
  arch/x86/kvm/boot/Makefile            |    6 +
  arch/x86/kvm/boot/seam/tdx_common.c   |  242 +++
  arch/x86/kvm/boot/seam/tdx_common.h   |   13 +
  arch/x86/kvm/ioapic.c                 |    4 +
  arch/x86/kvm/irq_comm.c               |   13 +-
  arch/x86/kvm/lapic.c                  |    7 +-
  arch/x86/kvm/lapic.h                  |    2 +-
  arch/x86/kvm/mmu.h                    |   31 +-
  arch/x86/kvm/mmu/mmu.c                |  526 +++++--
  arch/x86/kvm/mmu/mmu_internal.h       |    3 +
  arch/x86/kvm/mmu/paging_tmpl.h        |   25 +-
  arch/x86/kvm/mmu/spte.c               |   15 +-
  arch/x86/kvm/mmu/spte.h               |   18 +-
  arch/x86/kvm/svm/svm.c                |   18 +-
  arch/x86/kvm/trace.h                  |  138 ++
  arch/x86/kvm/vmx/common.h             |  178 +++
  arch/x86/kvm/vmx/main.c               | 1098 ++++++++++++++
  arch/x86/kvm/vmx/posted_intr.c        |    6 +
  arch/x86/kvm/vmx/seamcall.S           |   64 +
  arch/x86/kvm/vmx/seamcall.h           |   68 +
  arch/x86/kvm/vmx/tdx.c                | 1958 +++++++++++++++++++++++++
  arch/x86/kvm/vmx/tdx.h                |  267 ++++
  arch/x86/kvm/vmx/tdx_arch.h           |  370 +++++
  arch/x86/kvm/vmx/tdx_errno.h          |  202 +++
  arch/x86/kvm/vmx/tdx_ops.h            |  218 +++
  arch/x86/kvm/vmx/tdx_stubs.c          |   45 +
  arch/x86/kvm/vmx/vmcs.h               |   11 -
  arch/x86/kvm/vmx/vmenter.S            |  146 ++
  arch/x86/kvm/vmx/vmx.c                |  509 ++-----
  arch/x86/kvm/x86.c                    |  285 +++-
  include/linux/kvm_host.h              |   51 +-
  include/uapi/linux/kvm.h              |    2 +
  kernel/cpu.c                          |    4 +
  tools/arch/x86/include/uapi/asm/kvm.h |   55 +
  tools/include/uapi/linux/kvm.h        |    2 +
  virt/kvm/kvm_main.c                   |   44 +-
  54 files changed, 6717 insertions(+), 672 deletions(-)

create mode 100644 Documentation/virt/kvm/intel-tdx.rst create mode 100644 Documentation/virt/kvm/tdx-module.rst create mode 100644 arch/x86/include/asm/kvm_boot.h create mode 100644 arch/x86/kvm/boot/Makefile create mode 100644 arch/x86/kvm/boot/seam/tdx_common.c create mode 100644 arch/x86/kvm/boot/seam/tdx_common.h create mode 100644 arch/x86/kvm/vmx/common.h create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/seamcall.S create mode 100644 arch/x86/kvm/vmx/seamcall.h create mode 100644 arch/x86/kvm/vmx/tdx.c create mode 100644 arch/x86/kvm/vmx/tdx.h create mode 100644 arch/x86/kvm/vmx/tdx_arch.h create mode 100644 arch/x86/kvm/vmx/tdx_errno.h create mode 100644 arch/x86/kvm/vmx/tdx_ops.h create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c

#172 lore raw
Quoted 15 lines · Paolo Bonzini
Quoted 4 lines · On 03/07/21 00:04, [email protected] wrote:

* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.

Quoted 9 lines

- 01-12: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 13-40: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 41-65: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 66: main patch to add "basic" support for building/running TDX.
- 67: trace points for
- 68-69: Documentation

Quoted 1 line

Queued 2,3,17-20,23,44-45, thanks.

I strongly object to merging these two until we see the new SEAMLDR code:

[RFC PATCH v2 02/69] KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h [RFC PATCH v2 03/69] KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86

If the SEAMLDR code ends up being fully contained in KVM, then this is unnecessary churn and exposes code outside of KVM that we may not want exposed (yet). E.g. setting and clearing CR4.VMXE (in the fault path) in cpu_vmxon() may not be necessary/desirable for SEAMLDR, we simply can't tell without seeing the code.

maintainer
#173 lore raw
Quoted 9 lines · Sean Christopherson

I strongly object to merging these two until we see the new SEAMLDR code:

[RFC PATCH v2 02/69] KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h [RFC PATCH v2 03/69] KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86

If the SEAMLDR code ends up being fully contained in KVM, then this is unnecessary churn and exposes code outside of KVM that we may not want exposed (yet). E.g. setting and clearing CR4.VMXE (in the fault path) in cpu_vmxon() may not be necessary/desirable for SEAMLDR, we simply can't tell without seeing the code.

Fair enough (though, for patch 2, it's a bit weird to have vmxoff in virtext.h and not vmxon).

Paolo

#174 lore raw
Quoted 6 lines · Paolo Bonzini
Quoted 5 lines · Sean Christopherson

I strongly object to merging these two until we see the new SEAMLDR code:

[RFC PATCH v2 02/69] KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h [RFC PATCH v2 03/69] KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86

Quoted 7 lines
Quoted 4 lines

If the SEAMLDR code ends up being fully contained in KVM, then this is unnecessary churn and exposes code outside of KVM that we may not want exposed (yet). E.g. setting and clearing CR4.VMXE (in the fault path) in cpu_vmxon() may not be necessary/desirable for SEAMLDR, we simply can't tell without seeing the code.

Fair enough (though, for patch 2, it's a bit weird to have vmxoff in virtext.h and not vmxon).

I don't really disagree, but vmxoff is already in virtext.h for the emergency reboot stuff. This series only touches the vmxon code.

maintainer
#175 lore raw
Quoted 5 lines · Sean Christopherson
Quoted 2 lines

Fair enough (though, for patch 2, it's a bit weird to have vmxoff in virtext.h and not vmxon).

I don't really disagree, but vmxoff is already in virtext.h for the emergency reboot stuff. This series only touches the vmxon code.

Yes, that's what I meant. But anyway I've dequeued them.

Paolo