KVM: X86: TDX support
Subsystems (8)
Paolo Bonzini
maintainerRe: [RFC PATCH v2 63/69] KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX code
In reply to [email protected]Quoted 46 lines · On 03/07/21 00:05, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/common.h | 14 ++++++++++++++ arch/x86/kvm/vmx/vmx.c | 10 +--------- 2 files changed, 15 insertions(+), 9 deletions(-)arch/x86/kvm/vmx/common.h +14
-0@@ -120,6 +120,20 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa,return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); } +static inline u32 __vmx_get_interrupt_shadow (struct kvm_vcpu *vcpu) +{ +u32 interruptibility; +int ret =0 ; + + interruptibility =vmread32 (vcpu,GUEST_INTERRUPTIBILITY_INFO ); +if (interruptibility &GUEST_INTR_STATE_STI ) + ret |=KVM_X86_SHADOW_INT_STI ; +if (interruptibility &GUEST_INTR_STATE_MOV_SS ) + ret |=KVM_X86_SHADOW_INT_MOV_SS ; + +return ret; +} +static inline u32 vmx_encode_ar_bytes (struct kvm_segment *var) {u32 ar;arch/x86/kvm/vmx/vmx.c +0
-3@@ -1467,15 +1467,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)u32 vmx_get_interrupt_shadow (struct kvm_vcpu *vcpu) { -u32 interruptibility =vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ); -int ret =0 ; -
arch/x86/kvm/vmx/vmx.c void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) · line 1473
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +1
-6-if (interruptibility &GUEST_INTR_STATE_STI ) - ret |=KVM_X86_SHADOW_INT_STI ; -if (interruptibility &GUEST_INTR_STATE_MOV_SS ) - ret |=KVM_X86_SHADOW_INT_MOV_SS ; - -return ret; +return __vmx_get_interrupt_shadow (vcpu); }void vmx_set_interrupt_shadow (struct kvm_vcpu *vcpu,int mask)
Is there any reason to add the __ version, since at this point kvm_x86_ops is already pointing to vt_get_interrupt_shadow?
Paolo
Sean Christopherson
maintainerRe: [RFC PATCH v2 63/69] KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX code
Quoted 16 lines · Paolo Bonzini
Quoted 15 lines · On 03/07/21 00:05, [email protected] wrote:
arch/x86/kvm/vmx/vmx.c +0
-8@@ -1467,15 +1467,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)u32 vmx_get_interrupt_shadow (struct kvm_vcpu *vcpu) { -u32 interruptibility =vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ); -int ret =0 ; - -if (interruptibility &GUEST_INTR_STATE_STI ) - ret |=KVM_X86_SHADOW_INT_STI ; -if (interruptibility &GUEST_INTR_STATE_MOV_SS ) - ret |=KVM_X86_SHADOW_INT_MOV_SS ; -
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +1
-1-return ret; +return __vmx_get_interrupt_shadow (vcpu); }void vmx_set_interrupt_shadow (struct kvm_vcpu *vcpu,int mask)Is there any reason to add the __ version, since at this point
kvm_x86_opsis already pointing tovt_get_interrupt_shadow?
Yeah, no idea what I was thinking, the whole thing can be moved as is, just need to delete the prototype in vmx.h.
RFC PATCH v2 64/69 cpu/hotplug: Document that TDX also depends on booting CPUs once
In reply to [email protected]From: Kai Huang <[email protected]>
Add a comment to explain that TDX also depends on booting logical CPUs at least once.
TDSYSINITLP must be run on all CPUs, even software disabled CPUs in the -nosmt case. Fortunately, current SMT handling for #MC already supports booting all CPUs once; the to-be-disabled sibling is booted once (and later put into deep C-state to honor SMT=off) to allow the init code to set CR4.MCE and avoid an unwanted shutdown on a broadcasted MCE.
--- kernel/cpu.c | 4 ++++ 1 file changed, 4 insertions(+)
kernel/cpu.c +4 -0
@@ -446,6 +446,10 @@ static inline bool cpu_smt_allowed(unsigned int cpu)* that the init code can get a chance to set CR4.MCE on each * CPU. Otherwise, a broadcasted MCE observing CR4.MCE=0b on any * core will shutdown the machine. +* +* Intel TDX also requires running TDH_SYS_LP_INIT on all logical CPUs +* during boot, booting all CPUs once allows TDX to play nice with +* 'nosmt'. */ return !cpumask_test_cpu (cpu, &cpus_booted_once_mask); }
Signature · 1 line
2.25.1
RFC PATCH v2 65/69 KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
In reply to [email protected]From: Xiaoyao Li <[email protected]>
Introduce a per-vm variable initial_tsc_khz to hold the default tsc_khz for kvm_arch_vcpu_create().
This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1030,6 +1030,7 @@ struct kvm_arch {u64 last_tsc_nsec;u64 last_tsc_write;u32 last_tsc_khz; +u32 initial_tsc_khz;u64 cur_tsc_nsec;u64 cur_tsc_write;u64 cur_tsc_offset;
arch/x86/kvm/x86.c +2 -1
@@ -10441,7 +10441,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)else vcpu->arch.mp_state =KVM_MP_STATE_UNINITIALIZED ; -kvm_set_tsc_khz (vcpu, max_tsc_khz); +kvm_set_tsc_khz (vcpu, vcpu->kvm->arch.initial_tsc_khz); r =kvm_mmu_create (vcpu);if (r <0 )
@@ -10894,6 +10894,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)pvclock_update_vm_gtod_copy (kvm); kvm->arch.guest_can_read_msr_platform_info = true; + kvm->arch.initial_tsc_khz = max_tsc_khz;INIT_DELAYED_WORK (&kvm->arch.kvmclock_update_work, kvmclock_update_fn);INIT_DELAYED_WORK (&kvm->arch.kvmclock_sync_work, kvmclock_sync_fn);
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 65/69] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
Quoted 8 lines · On 03/07/21 00:05, [email protected] wrote:
From: Xiaoyao Li <[email protected]>
Introduce a per-vm variable
initial_tsc_khzto hold the defaulttsc_khzforkvm_arch_vcpu_create().This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.
Quoted 6 lines
Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)
So this means disabling TSC frequency scaling on TDX. Would it make sense to delay VM creation to a separate ioctl, similar to KVM_ARM_VCPU_FINALIZE (KVM_VM_FINALIZE)?
Paolo
Quoted 34 lines
arch/x86/include/asm/kvm_host.h +1
-0@@ -1030,6 +1030,7 @@ struct kvm_arch {u64 last_tsc_nsec;u64 last_tsc_write;u32 last_tsc_khz; +u32 initial_tsc_khz;u64 cur_tsc_nsec;u64 cur_tsc_write;u64 cur_tsc_offset;arch/x86/kvm/x86.c +2
-1@@ -10441,7 +10441,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)else vcpu->arch.mp_state =KVM_MP_STATE_UNINITIALIZED ; -kvm_set_tsc_khz (vcpu, max_tsc_khz); +kvm_set_tsc_khz (vcpu, vcpu->kvm->arch.initial_tsc_khz); r =kvm_mmu_create (vcpu);if (r <0 )@@ -10894,6 +10894,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)pvclock_update_vm_gtod_copy (kvm); kvm->arch.guest_can_read_msr_platform_info = true; + kvm->arch.initial_tsc_khz = max_tsc_khz;INIT_DELAYED_WORK (&kvm->arch.kvmclock_update_work, kvmclock_update_fn);INIT_DELAYED_WORK (&kvm->arch.kvmclock_sync_work, kvmclock_sync_fn);
Sean Christopherson
maintainerRe: [RFC PATCH v2 65/69] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
Quoted 16 lines · Paolo Bonzini
Quoted 8 lines · On 03/07/21 00:05, [email protected] wrote:
From: Xiaoyao Li <[email protected]>
Introduce a per-vm variable
initial_tsc_khzto hold the defaulttsc_khzforkvm_arch_vcpu_create().This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.
Quoted 6 lines
Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)
Quoted 3 lines
So this means disabling TSC frequency scaling on TDX. Would it make sense to delay VM creation to a separate ioctl, similar to
KVM_ARM_VCPU_FINALIZE(KVM_VM_FINALIZE)?
There's an equivalent of that in the next mega-patch, the KVM_TDX_INIT_VM sub-ioctl of KVM_MEMORY_ENCRYPT_OP. The TSC frequency for the guest gets provided at that time.
Re: [RFC PATCH v2 65/69] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Paolo Bonzini
Quoted 8 lines · On 03/07/21 00:05, [email protected] wrote:
From: Xiaoyao Li <[email protected]>
Introduce a per-vm variable
initial_tsc_khzto hold the defaulttsc_khzforkvm_arch_vcpu_create().This field is going to be used by TDX since TSC frequency for TD guest is configured at TD VM initialization phase.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)So this means disabling TSC frequency scaling on TDX.
No. It still supports TSC frequency scaling on TDX. Only that we need to configure TSC frequency for TD guest at VM level, not vcpu level.
Quoted 8 lines
Quoted 3 lines
Would it make sense
to delay VM creation to a separate ioctl, similar toKVM_ARM_VCPU_FINALIZE(KVM_VM_FINALIZE)?There's an equivalent of that in the next mega-patch, the
KVM_TDX_INIT_VMsub-ioctl ofKVM_MEMORY_ENCRYPT_OP. The TSC frequency for the guest gets provided at that time.
Sean Christopherson
maintainerRe: [RFC PATCH v2 65/69] KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
In reply to [email protected]Nit on the shortlog (applies to other patches too): please use "x86", not "X86".
Thanks!
RFC PATCH v2 66/69 KVM: TDX: Add "basic" support for building and running Trust Domains
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add what is effectively a TDX-specific ioctl for initializing the guest Trust Domain. Implement the functionality as a subcommand of KVM_MEMORY_ENCRYPT_OP, analogous to how the ioctl is used by SVM to manage SEV guests.
For easy compatibility with future versions of TDX-SEAM, add a KVM-defined struct, tdx_capabilities, to track requirements/capabilities for the overall system, and define a global instance to serve as the canonical reference.
--- arch/x86/include/uapi/asm/kvm.h | 56 + arch/x86/include/uapi/asm/vmx.h | 3 +- arch/x86/kvm/mmu.h | 2 +- arch/x86/kvm/mmu/mmu.c | 1 + arch/x86/kvm/mmu/spte.c | 5 +- arch/x86/kvm/vmx/common.h | 1 + arch/x86/kvm/vmx/main.c | 375 ++++- arch/x86/kvm/vmx/posted_intr.c | 6 + arch/x86/kvm/vmx/tdx.c | 1942 +++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 97 ++ arch/x86/kvm/vmx/tdx_arch.h | 7 + arch/x86/kvm/vmx/tdx_ops.h | 13 + arch/x86/kvm/vmx/tdx_stubs.c | 45 + arch/x86/kvm/vmx/vmenter.S | 146 ++ arch/x86/kvm/x86.c | 8 +- tools/arch/x86/include/uapi/asm/kvm.h | 51 + 16 files changed, 2736 insertions(+), 22 deletions(-) create mode 100644 arch/x86/kvm/vmx/tdx.c create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c
arch/x86/include/uapi/asm/kvm.h +56 -0
@@ -494,4 +494,60 @@ struct kvm_pmu_event_filter {#define KVM_X86_SEV_ES_VM 1#define KVM_X86_TDX_VM 2 +/* Trust Domain eXtension sub-ioctl() commands. */ +enum kvm_tdx_cmd_id { +KVM_TDX_CAPABILITIES =0 , +KVM_TDX_INIT_VM , +KVM_TDX_INIT_VCPU , +KVM_TDX_INIT_MEM_REGION , +KVM_TDX_FINALIZE_VM , + +KVM_TDX_CMD_NR_MAX , +}; + +struct kvm_tdx_cmd { +__u32 id; +__u32 metadata; +__u64 data; +}; + +struct kvm_tdx_cpuid_config { +__u32 leaf; +__u32 sub_leaf; +__u32 eax; +__u32 ebx; +__u32 ecx; +__u32 edx; +}; + +struct kvm_tdx_capabilities { +__u64 attrs_fixed0; +__u64 attrs_fixed1; +__u64 xfam_fixed0; +__u64 xfam_fixed1; + +__u32 nr_cpuid_configs; +__u32 padding; +struct kvm_tdx_cpuid_config cpuid_configs[0 ]; +}; + +struct kvm_tdx_init_vm { +__u32 max_vcpus; +__u32 tsc_khz; +__u64 attributes; +__u64 cpuid; +__u64 mrconfigid[6 ];/* sha384 digest */ +__u64 mrowner[6 ];/* sha384 digest */ +__u64 mrownerconfig[6 ];/* sha348 digest */ +__u64 reserved[43 ];/* must be zero for future extensibility */ +}; + +#define KVM_TDX_MEASURE_MEMORY_REGION (1UL << 0) + +struct kvm_tdx_init_mem_region { +__u64 source_addr; +__u64 gpa; +__u64 nr_pages; +}; +#endif /* _ASM_X86_KVM_H */
arch/x86/include/uapi/asm/vmx.h +2 -1
@@ -32,8 +32,9 @@#define EXIT_REASON_EXCEPTION_NMI 0#define EXIT_REASON_EXTERNAL_INTERRUPT 1#define EXIT_REASON_TRIPLE_FAULT 2 -#define EXIT_REASON_INIT_SIGNAL 3 +#define EXIT_REASON_INIT_SIGNAL 3#define EXIT_REASON_SIPI_SIGNAL 4 +#define EXIT_REASON_OTHER_SMI 6#define EXIT_REASON_INTERRUPT_WINDOW 7#define EXIT_REASON_NMI_WINDOW 8
arch/x86/kvm/mmu.h +1 -1
@@ -60,7 +60,7 @@ static __always_inline u64 rsvd_bits(int s, int e) }void kvm_mmu_set_mmio_spte_mask (u64 mmio_value,u64 mmio_mask,u64 access_mask); -void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only); +void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only,u64 init_value);void kvm_mmu_set_spte_init_value (u64 init_value);void
arch/x86/kvm/mmu/mmu.c +1 -0
@@ -5163,6 +5163,7 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu) out:return r; } +EXPORT_SYMBOL_GPL (kvm_mmu_load);static void __kvm_mmu_unload (struct kvm_vcpu *vcpu,u32 roots_to_free) {
arch/x86/kvm/mmu/spte.c +3 -2
@@ -302,14 +302,15 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask) }EXPORT_SYMBOL_GPL (kvm_mmu_set_mmio_spte_mask); -void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only) +void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only,u64 init_value) { shadow_user_mask =VMX_EPT_READABLE_MASK ; shadow_accessed_mask = has_ad_bits ?VMX_EPT_ACCESS_BIT :0ull ; shadow_dirty_mask = has_ad_bits ?VMX_EPT_DIRTY_BIT :0ull ; shadow_nx_mask =0ull ; shadow_x_mask =VMX_EPT_EXECUTABLE_MASK ; - shadow_present_mask = has_exec_only ?0ull :VMX_EPT_READABLE_MASK ; + shadow_present_mask = + (has_exec_only ?0ull :VMX_EPT_READABLE_MASK ) | init_value; shadow_acc_track_mask =VMX_EPT_RWX_MASK ; shadow_me_mask =0ull ;
arch/x86/kvm/vmx/common.h +1 -0
@@ -9,6 +9,7 @@#include <asm/vmx.h> #include "mmu.h" +#include "tdx.h" #include "vmcs.h" #include "vmx.h" #include "x86.h"
arch/x86/kvm/vmx/main.c +361 -14
872 lines not on this page · raw message
arch/x86/kvm/vmx/posted_intr.c +6 -0
@@ -6,6 +6,7 @@#include "lapic.h" #include "posted_intr.h" +#include "tdx.h" #include "trace.h" #include "vmx.h"
@@ -18,6 +19,11 @@ static DEFINE_PER_CPU(spinlock_t, blocked_vcpu_on_cpu_lock);static inline struct pi_desc *vcpu_to_pi_desc (struct kvm_vcpu *vcpu) { +#ifdef CONFIG_KVM_INTEL_TDX +if (is_td_vcpu (vcpu)) +return &(to_tdx (vcpu)->pi_desc); +#endif +return &(to_vmx (vcpu)->pi_desc); }
arch/x86/kvm/vmx/tdx.c new file +1942 -0
1,942 lines not on this page · raw message
arch/x86/kvm/vmx/tdx.h +97 -0
59 lines not on this page · raw message
arch/x86/kvm/vmx/tdx_stubs.c new file +45 -0
45 lines not on this page · raw message
arch/x86/kvm/vmx/vmenter.S +146 -0
146 lines not on this page · raw message
tools/arch/x86/include/uapi/asm/kvm.h +51 -0
51 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.25.1
Part of the diff is not shown here. View the raw message.
A message that is not in the archive (sent off-list or not yet received): CAAYXXYyz3S_cc9ohfkUWN4ohrNq5f+h3608CW5twb-n8i=ogBA@mail.gmail.com
Re: [RFC PATCH v2 66/69] KVM: TDX: Add "basic" support for building and running Trust Domains
Quoted 8 lines · Isaku Yamahata
Subject: [RFC PATCH v2 66/69] KVM: TDX: Add "basic" support for building and running Trust Domains
arch/x86/kvm/vmx/tdx.c new file +4
-0+static int tdx_map_gpa (struct kvm_vcpu *vcpu) +{ +gpa_t gpa =tdvmcall_p1_read (vcpu); +gpa_t size =tdvmcall_p2_read (vcpu);
arch/x86/kvm/vmx/tdx.c line 914
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c new file +10
-0+ +if (!IS_ALIGNED (gpa,4096 ) || !IS_ALIGNED (size,4096 ) || + (gpa + size) < gpa || + (gpa + size) > vcpu->kvm->arch.gfn_shared_mask << (PAGE_SHIFT +1 )) +tdvmcall_set_return_code (vcpu,TDG_VP_VMCALL_INVALID_OPERAND ); +else +tdvmcall_set_return_code (vcpu,TDG_VP_VMCALL_SUCCESS ); + +return 1 ; +}
This function looks like a no op in case of success. Is this intentional? Is this mapping handled somewhere else later on?
Re: [RFC PATCH v2 66/69] KVM: TDX: Add "basic" support for building and running Trust Domains
Quoted 15 lines · Sagi Shahar
Quoted 13 lines · Isaku Yamahata
Subject: [RFC PATCH v2 66/69] KVM: TDX: Add "basic" support for building and running Trust Domains
arch/x86/kvm/vmx/tdx.c new file +9
-0+static int tdx_map_gpa (struct kvm_vcpu *vcpu) +{ +gpa_t gpa =tdvmcall_p1_read (vcpu); +gpa_t size =tdvmcall_p2_read (vcpu); + +if (!IS_ALIGNED (gpa,4096 ) || !IS_ALIGNED (size,4096 ) || + (gpa + size) < gpa || + (gpa + size) > vcpu->kvm->arch.gfn_shared_mask << (PAGE_SHIFT +1 )) +tdvmcall_set_return_code (vcpu,TDG_VP_VMCALL_INVALID_OPERAND );
Quoted 9 lines
arch/x86/kvm/vmx/tdx.c line 761
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c new file +5
-0+else +tdvmcall_set_return_code (vcpu,TDG_VP_VMCALL_SUCCESS ); + +return 1 ; +}This function looks like a no op in case of success. Is this intentional? Is this mapping handled somewhere else later on?
Yes, it's intentional.
The mapping will be exactly set up in EPT violation handler when the GPA is really accessed.
RFC PATCH v2 67/69 KVM: TDX: add trace point for TDVMCALL and SEPT operation
In reply to [email protected]From: Isaku Yamahata <[email protected]>
--- arch/x86/kvm/trace.h | 58 +++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.c | 16 ++++++++++ arch/x86/kvm/vmx/tdx_arch.h | 9 ++++++ arch/x86/kvm/x86.c | 2 ++ 4 files changed, 85 insertions(+)
arch/x86/kvm/trace.h +58 -0
@@ -739,6 +739,64 @@ TRACE_EVENT(kvm_tdx_seamcall_exit, __entry->r9, __entry->r10, __entry->r11) ); +/* +* Tracepoint for TDVMCALL from a TDX guest +*/ +TRACE_EVENT (kvm_tdvmcall, + TP_PROTO(struct kvm_vcpu *vcpu, __u32 exit_reason, + __u64 p1, __u64 p2, __u64 p3, __u64 p4), +TP_ARGS (vcpu, exit_reason, p1, p2, p3, p4), + +TP_STRUCT__entry ( + __field( __u64, rip ) + __field( __u32, exit_reason ) + __field( __u64, p1 ) + __field( __u64, p2 ) + __field( __u64, p3 ) +__field ( __u64, p4 ) + ), + +TP_fast_assign ( + __entry->rip = kvm_rip_read(vcpu); + __entry->exit_reason = exit_reason; + __entry->p1 = p1; + __entry->p2 = p2; + __entry->p3 = p3; + __entry->p4 = p4; + ), + +TP_printk ("rip: %llx reason: %s p1: %llx p2: %llx p3: %llx p4: %llx" , + __entry->rip, +__print_symbolic (__entry->exit_reason, +TDG_VP_VMCALL_EXIT_REASONS ), + __entry->p1, __entry->p2, __entry->p3, __entry->p4) +); + +/* +* Tracepoint for SEPT related SEAMCALLs. +*/ +TRACE_EVENT (kvm_sept_seamcall, + TP_PROTO(__u64 op, __u64 gpa, __u64 hpa, int level), +TP_ARGS (op, gpa, hpa, level), + +TP_STRUCT__entry ( + __field( __u64, op ) + __field( __u64, gpa ) + __field( __u64, hpa ) +__field ( int, level ) + ), + +TP_fast_assign ( + __entry->op = op; + __entry->gpa = gpa; + __entry->hpa = hpa; + __entry->level = level; + ), + +TP_printk ("op: %llu gpa: 0x%llx hpa: 0x%llx level: %u" , + __entry->op, __entry->gpa, __entry->hpa, __entry->level) +); +/* * Tracepoint for nested #vmexit because of interrupt pending */
arch/x86/kvm/vmx/tdx.c +16 -0
@@ -934,6 +934,10 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu) exit_reason =tdvmcall_exit_reason (vcpu); +trace_kvm_tdvmcall (vcpu, exit_reason, +tdvmcall_p1_read (vcpu),tdvmcall_p2_read (vcpu), +tdvmcall_p3_read (vcpu),tdvmcall_p4_read (vcpu)); +switch (exit_reason) {case EXIT_REASON_CPUID :return tdx_emulate_cpuid (vcpu);
@@ -1011,11 +1015,15 @@ static void tdx_sept_set_private_spte(struct kvm_vcpu *vcpu, gfn_t gfn,/* Build-time faults are induced and handled via TDH_MEM_PAGE_ADD. */ if (is_td_finalized (kvm_tdx)) { +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_PAGE_AUG , gpa, hpa, level); + err =tdh_mem_page_aug (kvm_tdx->tdr.pa, gpa, hpa, &ex_ret);SEPT_ERR (err, &ex_ret,TDH_MEM_PAGE_AUG , vcpu->kvm);return ; } +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_PAGE_ADD , gpa, hpa, level); + source_pa = kvm_tdx->source_pa & ~KVM_TDX_MEASURE_MEMORY_REGION ; err =tdh_mem_page_add (kvm_tdx->tdr.pa, gpa, hpa, source_pa, &ex_ret);
@@ -1039,6 +1047,8 @@ static void tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, int level,return ;if (is_hkid_assigned (kvm_tdx)) { +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_PAGE_REMOVE , gpa, hpa, level); + err =tdh_mem_page_remove (kvm_tdx->tdr.pa, gpa, level, &ex_ret);if (SEPT_ERR (err, &ex_ret,TDH_MEM_PAGE_REMOVE , kvm))return ;
@@ -1063,6 +1073,8 @@ static int tdx_sept_link_private_sp(struct kvm_vcpu *vcpu, gfn_t gfn,struct tdx_ex_ret ex_ret;u64 err; +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_SEPT_ADD , gpa, hpa, level); + err =tdh_mem_spet_add (kvm_tdx->tdr.pa, gpa, level, hpa, &ex_ret);if (SEPT_ERR (err, &ex_ret,TDH_MEM_SEPT_ADD , vcpu->kvm))return -EIO ;
@@ -1077,6 +1089,8 @@ static void tdx_sept_zap_private_spte(struct kvm *kvm, gfn_t gfn, int level)struct tdx_ex_ret ex_ret;u64 err; +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_RANGE_BLOCK , gpa,-1ull , level); + err =tdh_mem_range_block (kvm_tdx->tdr.pa, gpa, level, &ex_ret);SEPT_ERR (err, &ex_ret,TDH_MEM_RANGE_BLOCK , kvm); }
@@ -1088,6 +1102,8 @@ static void tdx_sept_unzap_private_spte(struct kvm *kvm, gfn_t gfn, int level)struct tdx_ex_ret ex_ret;u64 err; +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_RANGE_UNBLOCK , gpa,-1ull , level); + err =tdh_mem_range_unblock (kvm_tdx->tdr.pa, gpa, level, &ex_ret);SEPT_ERR (err, &ex_ret,TDH_MEM_RANGE_UNBLOCK , kvm); }
arch/x86/kvm/vmx/tdx_arch.h +9 -0
@@ -104,6 +104,15 @@#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004 +#define TDG_VP_VMCALL_EXIT_REASONS \ + { TDG_VP_VMCALL_GET_TD_VM_CALL_INFO, \ + "GET_TD_VM_CALL_INFO" }, \ + { TDG_VP_VMCALL_MAP_GPA, "MAP_GPA" }, \ + { TDG_VP_VMCALL_GET_QUOTE, "GET_QUOTE" }, \ + { TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT, \ + "SETUP_EVENT_NOTIFY_INTERRUPT" }, \ + VMX_EXIT_REASONS +/* TDX control structure (TDR/TDCS/TDVPS) field access codes */ #define TDX_CLASS_SHIFT 56#define TDX_FIELD_MASK GENMASK_ULL(31, 0)
arch/x86/kvm/x86.c +2 -0
@@ -12104,6 +12104,8 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_cr);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmrun);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmexit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmexit_inject); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_tdvmcall); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_sept_seamcall);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_intr_vmexit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmenter_failed);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_invlpga);
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 67/69] KVM: TDX: add trace point for TDVMCALL and SEPT operation
Quoted 173 lines · On 03/07/21 00:05, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Signed-off-by Yuan Yao <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/trace.h | 58 +++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.c | 16 ++++++++++ arch/x86/kvm/vmx/tdx_arch.h | 9 ++++++ arch/x86/kvm/x86.c | 2 ++ 4 files changed, 85 insertions(+)arch/x86/kvm/trace.h +58
-0@@ -739,6 +739,64 @@ TRACE_EVENT(kvm_tdx_seamcall_exit, __entry->r9, __entry->r10, __entry->r11) ); +/* +* Tracepoint for TDVMCALL from a TDX guest +*/ +TRACE_EVENT (kvm_tdvmcall, + TP_PROTO(struct kvm_vcpu *vcpu, __u32 exit_reason, + __u64 p1, __u64 p2, __u64 p3, __u64 p4), +TP_ARGS (vcpu, exit_reason, p1, p2, p3, p4), + +TP_STRUCT__entry ( + __field( __u64, rip ) + __field( __u32, exit_reason ) + __field( __u64, p1 ) + __field( __u64, p2 ) + __field( __u64, p3 ) +__field ( __u64, p4 ) + ), + +TP_fast_assign ( + __entry->rip = kvm_rip_read(vcpu); + __entry->exit_reason = exit_reason; + __entry->p1 = p1; + __entry->p2 = p2; + __entry->p3 = p3; + __entry->p4 = p4; + ), + +TP_printk ("rip: %llx reason: %s p1: %llx p2: %llx p3: %llx p4: %llx" , + __entry->rip, +__print_symbolic (__entry->exit_reason, +TDG_VP_VMCALL_EXIT_REASONS ), + __entry->p1, __entry->p2, __entry->p3, __entry->p4) +); + +/* +* Tracepoint for SEPT related SEAMCALLs. +*/ +TRACE_EVENT (kvm_sept_seamcall, + TP_PROTO(__u64 op, __u64 gpa, __u64 hpa, int level), +TP_ARGS (op, gpa, hpa, level), + +TP_STRUCT__entry ( + __field( __u64, op ) + __field( __u64, gpa ) + __field( __u64, hpa ) +__field ( int, level ) + ), + +TP_fast_assign ( + __entry->op = op; + __entry->gpa = gpa; + __entry->hpa = hpa; + __entry->level = level; + ), + +TP_printk ("op: %llu gpa: 0x%llx hpa: 0x%llx level: %u" , + __entry->op, __entry->gpa, __entry->hpa, __entry->level) +); +/* * Tracepoint for nested #vmexit because of interrupt pending */ arch/x86/kvm/vmx/tdx.c +16
-0@@ -934,6 +934,10 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu) exit_reason =tdvmcall_exit_reason (vcpu); +trace_kvm_tdvmcall (vcpu, exit_reason, +tdvmcall_p1_read (vcpu),tdvmcall_p2_read (vcpu), +tdvmcall_p3_read (vcpu),tdvmcall_p4_read (vcpu)); +switch (exit_reason) {case EXIT_REASON_CPUID :return tdx_emulate_cpuid (vcpu);@@ -1011,11 +1015,15 @@ static void tdx_sept_set_private_spte(struct kvm_vcpu *vcpu, gfn_t gfn,/* Build-time faults are induced and handled via TDH_MEM_PAGE_ADD. */ if (is_td_finalized (kvm_tdx)) { +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_PAGE_AUG , gpa, hpa, level); + err =tdh_mem_page_aug (kvm_tdx->tdr.pa, gpa, hpa, &ex_ret);SEPT_ERR (err, &ex_ret,TDH_MEM_PAGE_AUG , vcpu->kvm);return ; } +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_PAGE_ADD , gpa, hpa, level); + source_pa = kvm_tdx->source_pa & ~KVM_TDX_MEASURE_MEMORY_REGION ; err =tdh_mem_page_add (kvm_tdx->tdr.pa, gpa, hpa, source_pa, &ex_ret);@@ -1039,6 +1047,8 @@ static void tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, int level,return ;if (is_hkid_assigned (kvm_tdx)) { +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_PAGE_REMOVE , gpa, hpa, level); + err =tdh_mem_page_remove (kvm_tdx->tdr.pa, gpa, level, &ex_ret);if (SEPT_ERR (err, &ex_ret,TDH_MEM_PAGE_REMOVE , kvm))return ;@@ -1063,6 +1073,8 @@ static int tdx_sept_link_private_sp(struct kvm_vcpu *vcpu, gfn_t gfn,struct tdx_ex_ret ex_ret;u64 err; +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_SEPT_ADD , gpa, hpa, level); + err =tdh_mem_spet_add (kvm_tdx->tdr.pa, gpa, level, hpa, &ex_ret);if (SEPT_ERR (err, &ex_ret,TDH_MEM_SEPT_ADD , vcpu->kvm))return -EIO ;@@ -1077,6 +1089,8 @@ static void tdx_sept_zap_private_spte(struct kvm *kvm, gfn_t gfn, int level)struct tdx_ex_ret ex_ret;u64 err; +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_RANGE_BLOCK , gpa,-1ull , level); + err =tdh_mem_range_block (kvm_tdx->tdr.pa, gpa, level, &ex_ret);SEPT_ERR (err, &ex_ret,TDH_MEM_RANGE_BLOCK , kvm); }@@ -1088,6 +1102,8 @@ static void tdx_sept_unzap_private_spte(struct kvm *kvm, gfn_t gfn, int level)struct tdx_ex_ret ex_ret;u64 err; +trace_kvm_sept_seamcall (SEAMCALL_TDH_MEM_RANGE_UNBLOCK , gpa,-1ull , level); + err =tdh_mem_range_unblock (kvm_tdx->tdr.pa, gpa, level, &ex_ret);SEPT_ERR (err, &ex_ret,TDH_MEM_RANGE_UNBLOCK , kvm); }arch/x86/kvm/vmx/tdx_arch.h +9
-0@@ -104,6 +104,15 @@#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004 +#define TDG_VP_VMCALL_EXIT_REASONS \ + { TDG_VP_VMCALL_GET_TD_VM_CALL_INFO, \ + "GET_TD_VM_CALL_INFO" }, \ + { TDG_VP_VMCALL_MAP_GPA, "MAP_GPA" }, \ + { TDG_VP_VMCALL_GET_QUOTE, "GET_QUOTE" }, \ + { TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT, \ + "SETUP_EVENT_NOTIFY_INTERRUPT" }, \ + VMX_EXIT_REASONS +/* TDX control structure (TDR/TDCS/TDVPS) field access codes */ #define TDX_CLASS_SHIFT 56#define TDX_FIELD_MASK GENMASK_ULL(31, 0)arch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_cr); · line 12111
Quoted 8 lines
arch/x86/kvm/x86.c +2
-0@@ -12104,6 +12104,8 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_cr);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmrun);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmexit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmexit_inject); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_tdvmcall); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_sept_seamcall);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_intr_vmexit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmenter_failed);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_invlpga);
Please split this in two parts for each tracepoint, and squash it in the earlier patches that introduced handle_tdvmcall and tdx_sept_*.
Paolo
RFC PATCH v2 68/69 KVM: TDX: add document on TDX MODULE
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add a document on how to integrate TDX MODULE into initrd so that TDX MODULE can be updated on kernel startup.
--- Documentation/virt/kvm/tdx-module.rst | 48 +++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 Documentation/virt/kvm/tdx-module.rst
Documentation/virt/kvm/tdx-module.rst new file +48 -0
@@ -0,0 +1,48 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========== +TDX MODULE +========== + +Integrating TDX MODULE into initrd +================================== +If TDX is enabled in KVM(CONFIG_KVM_INTEL_TDX=y), kernel is able to load +tdx seam module from initrd. +The related modules (seamldr.ac, libtdx.so and libtdx.so.sigstruct) need to be +stored in initrd. + +tdx-seam is a sample hook script for initramfs-tools. +TDXSEAM_SRCDIR are the directory in the host file system to store files related +to TDX MODULE. + +Since it heavily depends on distro how to prepare initrd, here's a example how +to prepare an initrd. +(Actually this is taken from Documentation/x86/microcode.rst) +:: + #!/bin/bash + + if [ -z "$1" ]; then + echo "You need to supply an initrd file" + exit 1 + fi + + INITRD="$1" + + DSTDIR=lib/firmware/intel-seam + TMPDIR=/tmp/initrd + LIBTDX="/lib/firmware/intel-seam/seamldr.acm /lib/firmware/intel-seam/libtdx.so /lib/firmware/intel-seam/libtdx.so.sigstruct" + + rm -rf $TMPDIR + + mkdir $TMPDIR + cd $TMPDIR + mkdir -p $DSTDIR + + cp ${LIBTDX} ${DSTDIR} + + find . | cpio -o -H newc > ../tdx-seam.cpio + cd .. + mv $INITRD $INITRD.orig + cat tdx-seam.cpio $INITRD.orig > $INITRD + + rm -rf $TMPDIR
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 68/69] KVM: TDX: add document on TDX MODULE
Quoted 56 lines · On 03/07/21 00:05, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add a document on how to integrate TDX MODULE into initrd so that TDX MODULE can be updated on kernel startup.
Signed-off-by Isaku Yamahata <[email protected]>--- Documentation/virt/kvm/tdx-module.rst | 48 +++++++++++++++++++++++++++ 1 file changed, 48 insertions(+)create mode 100644
Documentation/virt/kvm/tdx-module.rstDocumentation/virt/kvm/tdx-module.rst new file +38
-0@@ -0,0 +1,48 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========== +TDX MODULE +========== + +Integrating TDX MODULE into initrd +================================== +If TDX is enabled in KVM(CONFIG_KVM_INTEL_TDX=y), kernel is able to load +tdx seam module from initrd. +The related modules (seamldr.ac, libtdx.so and libtdx.so.sigstruct) need to be +stored in initrd. + +tdx-seam is a sample hook script for initramfs-tools. +TDXSEAM_SRCDIR are the directory in the host file system to store files related +to TDX MODULE. + +Since it heavily depends on distro how to prepare initrd, here's a example how +to prepare an initrd. +(Actually this is taken from Documentation/x86/microcode.rst) +:: + #!/bin/bash + + if [ -z "$1" ]; then + echo "You need to supply an initrd file" + exit 1 + fi + + INITRD="$1" + + DSTDIR=lib/firmware/intel-seam + TMPDIR=/tmp/initrd + LIBTDX="/lib/firmware/intel-seam/seamldr.acm /lib/firmware/intel-seam/libtdx.so /lib/firmware/intel-seam/libtdx.so.sigstruct" + + rm -rf $TMPDIR + + mkdir $TMPDIR + cd $TMPDIR
Documentation/virt/kvm/tdx-module.rst line 48
Quoted 10 lines
Documentation/virt/kvm/tdx-module.rst new file +10
-0+ mkdir -p $DSTDIR + + cp ${LIBTDX} ${DSTDIR} + + find . | cpio -o -H newc > ../tdx-seam.cpio + cd .. + mv $INITRD $INITRD.orig + cat tdx-seam.cpio $INITRD.orig > $INITRD + + rm -rf $TMPDIR
I think this belongs in a different series that adds SEAM loading?
Paolo
RFC PATCH v2 69/69 Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add a documentation to Intel Trusted Docmain Extensions(TDX) support.
--- Documentation/virt/kvm/api.rst | 6 +- Documentation/virt/kvm/intel-tdx.rst | 441 +++++++++++++++++++++++++++ 2 files changed, 446 insertions(+), 1 deletion(-) create mode 100644 Documentation/virt/kvm/intel-tdx.rst
Documentation/virt/kvm/api.rst +5 -1
@@ -4297,7 +4297,7 @@ H_GET_CPU_CHARACTERISTICS hypercall. :Capability: basic :Architectures: x86 -:Type: vm +:Type: vm ioctl, vcpu ioctl :Parameters: an opaque platform specific structure (in/out) :Returns: 0 on success; -1 on error
@@ -4309,6 +4309,10 @@ Currently, this ioctl is used for issuing Secure Encrypted Virtualization (SEV) commands on AMD Processors. The SEV commands are defined in Documentation/virt/kvm/amd-memory-encryption.rst. +Currently, this ioctl is used for issuing Trusted Domain Extensions +(TDX) commands on Intel Processors. The TDX commands are defined in +Documentation/virt/kvm/intel-tdx.rst. + 4.111 KVM_MEMORY_ENCRYPT_REG_REGION -----------------------------------
Documentation/virt/kvm/intel-tdx.rst new file +441 -0
441 lines not on this page · raw message
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerBased on the initial review, I think patches 2-3-17-18-19-20-23-49 can already be merged for 5.15.
The next part should be the introduction of vm_types, blocking ioctls depending on the vm_type (patches 24-31) and possibly applying this block already to SEV-ES.
Quoted 234 lines · On 03/07/21 00:04, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform. [1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.* The goal of this RFC patch
The purpose of this post is to get feedback early on high level design issue of KVM enhancement for TDX. The detailed coding (variable naming etc) is not cared of. This patch series is incomplete (not working). So it's RFC. Although multiple software components, not only KVM but also QEMU, guest Linux and virtual bios, need to be updated, this includes only KVM VMM part. For those who are curious to changes to other component, there are public repositories at github. [8], [9]* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.- 01-12: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 13-40: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 41-65: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 66: main patch to add "basic" support for building/running TDX.
- 67: trace points for
- 68-69: Documentation* TODOs
Those major features are missing from this patch series to keep this patch series small.- load/initialize TDX module
split out from this patch series.
- unmapping private page
Will integrate Kirill's patch to show how kvm will utilize it.
- qemu gdb stub support
- Large page support
- guest PMU support
- TDP MMU support
- and moreChanges from v1:
- rebase to v5.13
- drop load/initialization of TDX module
- catch up the update of related specifications.
- rework on C-wrapper function to invoke seamcall
- various code clean up[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAShttps://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVFIsaku Yamahata (11):
KVM: TDX: introduce config for KVM TDX support
KVM: X86: movekvm_cpu_vmxon()fromvmx.ctovirtext.h
KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
KVM: TDX: add a helper function for kvm to call seamcall
KVM: TDX: add trace point before/after TDX SEAMCALLs
KVM: TDX: Print the name of SEAMCALL status code
KVM: Add per-VM flag to mark read-only memory as unsupported
KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
KVM: TDX: add trace point for TDVMCALL and SEPT operation
KVM: TDX: add document on TDX MODULEDocumentation/virtual/kvm: Add Trust Domain Extensions(TDX)Kai Huang (2):
KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level
routes
cpu/hotplug: Document that TDX also depends on booting CPUs onceRick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bitsSean Christopherson (53):
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add architectural definitions for structures and values
KVM: TDX: define and export helper functions for KVM TDX support
KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
KVM: Exportkvm_io_bus_readfor use by TDX for PV MMIO
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Split core of hypercall emulation to helper function
KVM: x86: Exportkvm_mmiotracepoint for use by TDX for PV MMIO
KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
KVM: Add infrastructure and macro to mark VM as bugged
KVM: Exportkvm_make_all_cpus_request()for use in marking VMs as
bugged
KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the
VM
KVM: x86/mmu: Mark VM as bugged if page fault returnsRET_PF_INVALID
KVM: Addmax_vcpusfield in common 'struct kvm'
KVM: x86: Addvm_typeto differentiate legacy VMs from protected VMs
KVM: x86: Hoistkvm_dirty_regscheck out ofsync_regs()
KVM: x86: Introduce "protected guest" concept and block disallowed
ioctls
KVM: x86: Add per-VM flag to disable direct IRQ injection
KVM: x86: Add flag to disallow #MC injection /KVM_X86_SETUP_MCE
KVM: x86: Add flag to mark TSC as immutable (for TDX)
KVM: Add per-VM flag to disable dirty logging of memslots for TDs
KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
KVM: x86: Addkvm_x86_ops.cache_gprs() and .flush_gprs()
KVM: x86: Add support for vCPU and device-scopedKVM_MEMORY_ENCRYPT_OP
KVM: x86: Introducevm_teardown()hook inkvm_arch_vm_destroy()
KVM: x86: Add aswitch_db_regsflag to handle TDX's auto-switched
behavior
KVM: x86: Check for pending APICv interrupt inkvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: x86: Addguest_supported_xssplacholder
KVM: Exportkvm_is_reserved_pfn()for use by TDX
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Refactor shadow walk in__direct_map()to reduce
indentation
KVM: x86/mmu: Return old SPTE frommmu_spte_clear_track_bits()
KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
KVM: x86/mmu: Move 'pfn' variable to caller ofdirect_page_fault()
KVM: x86/mmu: Introducekvm_mmu_map_tdp_page()for use by TDX
KVM: VMX: Modify NMI and INTR handlers to takeintr_infoas param
KVM: VMX: Move NMI/exception handler to common helper
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Define EPT Violation architectural bits
KVM: VMX: Define VMCS encodings for shared EPT pointer
KVM: VMX: Add 'main.c' to wrap VMX and TDX
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: VMX: Move register caching logic to common code
KVM: TDX: Define TDCALL exit reason
KVM: TDX: Stub intdx.hwith structs, accessors, and VMCS helpers
KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
KVM: VMX: MoveAR_BYTESencoder/decoder helpers tocommon.h
KVM: VMX: MOVE GDT and IDT accessors to common code
KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX
code
KVM: TDX: Add "basic" support for building and running Trust DomainsXiaoyao Li (2):
KVM: TDX: Introducepr_seamcall_ex_ret_info()to print more info when
SEAMCALL fails
KVM: X86: Introduceinitial_tsc_khzin structkvm_archDocumentation/virt/kvm/api.rst | 6 +- Documentation/virt/kvm/intel-tdx.rst | 441 ++++++ Documentation/virt/kvm/tdx-module.rst | 48 + arch/arm64/include/asm/kvm_host.h | 3 - arch/arm64/kvm/arm.c | 7 +- arch/arm64/kvm/vgic/vgic-init.c | 6 +- arch/x86/Kbuild | 1 + arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/kvm-x86-ops.h | 8 + arch/x86/include/asm/kvm_boot.h | 30 + arch/x86/include/asm/kvm_host.h | 55 +- arch/x86/include/asm/virtext.h | 25 + arch/x86/include/asm/vmx.h | 17 + arch/x86/include/uapi/asm/kvm.h | 60 + arch/x86/include/uapi/asm/vmx.h | 7 +- arch/x86/kernel/asm-offsets_64.c | 15 + arch/x86/kvm/Kconfig | 11 + arch/x86/kvm/Makefile | 3 +- arch/x86/kvm/boot/Makefile | 6 + arch/x86/kvm/boot/seam/tdx_common.c | 242 +++ arch/x86/kvm/boot/seam/tdx_common.h | 13 + arch/x86/kvm/ioapic.c | 4 + arch/x86/kvm/irq_comm.c | 13 +- arch/x86/kvm/lapic.c | 7 +- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/mmu.h | 31 +- arch/x86/kvm/mmu/mmu.c | 526 +++++-- arch/x86/kvm/mmu/mmu_internal.h | 3 + arch/x86/kvm/mmu/paging_tmpl.h | 25 +- arch/x86/kvm/mmu/spte.c | 15 +- arch/x86/kvm/mmu/spte.h | 18 +- arch/x86/kvm/svm/svm.c | 18 +- arch/x86/kvm/trace.h | 138 ++ arch/x86/kvm/vmx/common.h | 178 +++ arch/x86/kvm/vmx/main.c | 1098 ++++++++++++++ arch/x86/kvm/vmx/posted_intr.c | 6 + arch/x86/kvm/vmx/seamcall.S | 64 + arch/x86/kvm/vmx/seamcall.h | 68 + arch/x86/kvm/vmx/tdx.c | 1958 +++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 267 ++++ arch/x86/kvm/vmx/tdx_arch.h | 370 +++++ arch/x86/kvm/vmx/tdx_errno.h | 202 +++ arch/x86/kvm/vmx/tdx_ops.h | 218 +++ arch/x86/kvm/vmx/tdx_stubs.c | 45 + arch/x86/kvm/vmx/vmcs.h | 11 - arch/x86/kvm/vmx/vmenter.S | 146 ++ arch/x86/kvm/vmx/vmx.c | 509 ++----- arch/x86/kvm/x86.c | 285 +++- include/linux/kvm_host.h | 51 +- include/uapi/linux/kvm.h | 2 + kernel/cpu.c | 4 + tools/arch/x86/include/uapi/asm/kvm.h | 55 + tools/include/uapi/linux/kvm.h | 2 + virt/kvm/kvm_main.c | 44 +- 54 files changed, 6717 insertions(+), 672 deletions(-)create mode 100644
Documentation/virt/kvm/intel-tdx.rstcreate mode 100644Documentation/virt/kvm/tdx-module.rstcreate mode 100644arch/x86/include/asm/kvm_boot.hcreate mode 100644arch/x86/kvm/boot/Makefilecreate mode 100644arch/x86/kvm/boot/seam/tdx_common.ccreate mode 100644arch/x86/kvm/boot/seam/tdx_common.hcreate mode 100644arch/x86/kvm/vmx/common.hcreate mode 100644arch/x86/kvm/vmx/main.ccreate mode 100644arch/x86/kvm/vmx/seamcall.Screate mode 100644arch/x86/kvm/vmx/seamcall.hcreate mode 100644arch/x86/kvm/vmx/tdx.ccreate mode 100644arch/x86/kvm/vmx/tdx.hcreate mode 100644arch/x86/kvm/vmx/tdx_arch.hcreate mode 100644arch/x86/kvm/vmx/tdx_errno.hcreate mode 100644arch/x86/kvm/vmx/tdx_ops.hcreate mode 100644arch/x86/kvm/vmx/tdx_stubs.c
Paolo Bonzini
maintainerBased on the initial review, I think patches 2-3-17-18-19-20-23-49 can already be merged for 5.15.
The next part should be the introduction of vm_types, blocking ioctls depending on the vm_type (patches 24-31). Perhaps this blocking should be applied already to SEV-ES, so that the corresponding code in QEMU can be added early.
Paolo
Quoted 234 lines · On 03/07/21 00:04, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform. [1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.* The goal of this RFC patch
The purpose of this post is to get feedback early on high level design issue of KVM enhancement for TDX. The detailed coding (variable naming etc) is not cared of. This patch series is incomplete (not working). So it's RFC. Although multiple software components, not only KVM but also QEMU, guest Linux and virtual bios, need to be updated, this includes only KVM VMM part. For those who are curious to changes to other component, there are public repositories at github. [8], [9]* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.- 01-12: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 13-40: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 41-65: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 66: main patch to add "basic" support for building/running TDX.
- 67: trace points for
- 68-69: Documentation* TODOs
Those major features are missing from this patch series to keep this patch series small.- load/initialize TDX module
split out from this patch series.
- unmapping private page
Will integrate Kirill's patch to show how kvm will utilize it.
- qemu gdb stub support
- Large page support
- guest PMU support
- TDP MMU support
- and moreChanges from v1:
- rebase to v5.13
- drop load/initialization of TDX module
- catch up the update of related specifications.
- rework on C-wrapper function to invoke seamcall
- various code clean up[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAShttps://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVFIsaku Yamahata (11):
KVM: TDX: introduce config for KVM TDX support
KVM: X86: movekvm_cpu_vmxon()fromvmx.ctovirtext.h
KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
KVM: TDX: add a helper function for kvm to call seamcall
KVM: TDX: add trace point before/after TDX SEAMCALLs
KVM: TDX: Print the name of SEAMCALL status code
KVM: Add per-VM flag to mark read-only memory as unsupported
KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
KVM: TDX: add trace point for TDVMCALL and SEPT operation
KVM: TDX: add document on TDX MODULEDocumentation/virtual/kvm: Add Trust Domain Extensions(TDX)Kai Huang (2):
KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level
routes
cpu/hotplug: Document that TDX also depends on booting CPUs onceRick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bitsSean Christopherson (53):
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add architectural definitions for structures and values
KVM: TDX: define and export helper functions for KVM TDX support
KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
KVM: Exportkvm_io_bus_readfor use by TDX for PV MMIO
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Split core of hypercall emulation to helper function
KVM: x86: Exportkvm_mmiotracepoint for use by TDX for PV MMIO
KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
KVM: Add infrastructure and macro to mark VM as bugged
KVM: Exportkvm_make_all_cpus_request()for use in marking VMs as
bugged
KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the
VM
KVM: x86/mmu: Mark VM as bugged if page fault returnsRET_PF_INVALID
KVM: Addmax_vcpusfield in common 'struct kvm'
KVM: x86: Addvm_typeto differentiate legacy VMs from protected VMs
KVM: x86: Hoistkvm_dirty_regscheck out ofsync_regs()
KVM: x86: Introduce "protected guest" concept and block disallowed
ioctls
KVM: x86: Add per-VM flag to disable direct IRQ injection
KVM: x86: Add flag to disallow #MC injection /KVM_X86_SETUP_MCE
KVM: x86: Add flag to mark TSC as immutable (for TDX)
KVM: Add per-VM flag to disable dirty logging of memslots for TDs
KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
KVM: x86: Addkvm_x86_ops.cache_gprs() and .flush_gprs()
KVM: x86: Add support for vCPU and device-scopedKVM_MEMORY_ENCRYPT_OP
KVM: x86: Introducevm_teardown()hook inkvm_arch_vm_destroy()
KVM: x86: Add aswitch_db_regsflag to handle TDX's auto-switched
behavior
KVM: x86: Check for pending APICv interrupt inkvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: x86: Addguest_supported_xssplacholder
KVM: Exportkvm_is_reserved_pfn()for use by TDX
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Refactor shadow walk in__direct_map()to reduce
indentation
KVM: x86/mmu: Return old SPTE frommmu_spte_clear_track_bits()
KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
KVM: x86/mmu: Move 'pfn' variable to caller ofdirect_page_fault()
KVM: x86/mmu: Introducekvm_mmu_map_tdp_page()for use by TDX
KVM: VMX: Modify NMI and INTR handlers to takeintr_infoas param
KVM: VMX: Move NMI/exception handler to common helper
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Define EPT Violation architectural bits
KVM: VMX: Define VMCS encodings for shared EPT pointer
KVM: VMX: Add 'main.c' to wrap VMX and TDX
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: VMX: Move register caching logic to common code
KVM: TDX: Define TDCALL exit reason
KVM: TDX: Stub intdx.hwith structs, accessors, and VMCS helpers
KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
KVM: VMX: MoveAR_BYTESencoder/decoder helpers tocommon.h
KVM: VMX: MOVE GDT and IDT accessors to common code
KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX
code
KVM: TDX: Add "basic" support for building and running Trust DomainsXiaoyao Li (2):
KVM: TDX: Introducepr_seamcall_ex_ret_info()to print more info when
SEAMCALL fails
KVM: X86: Introduceinitial_tsc_khzin structkvm_archDocumentation/virt/kvm/api.rst | 6 +- Documentation/virt/kvm/intel-tdx.rst | 441 ++++++ Documentation/virt/kvm/tdx-module.rst | 48 + arch/arm64/include/asm/kvm_host.h | 3 - arch/arm64/kvm/arm.c | 7 +- arch/arm64/kvm/vgic/vgic-init.c | 6 +- arch/x86/Kbuild | 1 + arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/kvm-x86-ops.h | 8 + arch/x86/include/asm/kvm_boot.h | 30 + arch/x86/include/asm/kvm_host.h | 55 +- arch/x86/include/asm/virtext.h | 25 + arch/x86/include/asm/vmx.h | 17 + arch/x86/include/uapi/asm/kvm.h | 60 + arch/x86/include/uapi/asm/vmx.h | 7 +- arch/x86/kernel/asm-offsets_64.c | 15 + arch/x86/kvm/Kconfig | 11 + arch/x86/kvm/Makefile | 3 +- arch/x86/kvm/boot/Makefile | 6 + arch/x86/kvm/boot/seam/tdx_common.c | 242 +++ arch/x86/kvm/boot/seam/tdx_common.h | 13 + arch/x86/kvm/ioapic.c | 4 + arch/x86/kvm/irq_comm.c | 13 +- arch/x86/kvm/lapic.c | 7 +- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/mmu.h | 31 +- arch/x86/kvm/mmu/mmu.c | 526 +++++-- arch/x86/kvm/mmu/mmu_internal.h | 3 + arch/x86/kvm/mmu/paging_tmpl.h | 25 +- arch/x86/kvm/mmu/spte.c | 15 +- arch/x86/kvm/mmu/spte.h | 18 +- arch/x86/kvm/svm/svm.c | 18 +- arch/x86/kvm/trace.h | 138 ++ arch/x86/kvm/vmx/common.h | 178 +++ arch/x86/kvm/vmx/main.c | 1098 ++++++++++++++ arch/x86/kvm/vmx/posted_intr.c | 6 + arch/x86/kvm/vmx/seamcall.S | 64 + arch/x86/kvm/vmx/seamcall.h | 68 + arch/x86/kvm/vmx/tdx.c | 1958 +++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 267 ++++ arch/x86/kvm/vmx/tdx_arch.h | 370 +++++ arch/x86/kvm/vmx/tdx_errno.h | 202 +++ arch/x86/kvm/vmx/tdx_ops.h | 218 +++ arch/x86/kvm/vmx/tdx_stubs.c | 45 + arch/x86/kvm/vmx/vmcs.h | 11 - arch/x86/kvm/vmx/vmenter.S | 146 ++ arch/x86/kvm/vmx/vmx.c | 509 ++----- arch/x86/kvm/x86.c | 285 +++- include/linux/kvm_host.h | 51 +- include/uapi/linux/kvm.h | 2 + kernel/cpu.c | 4 + tools/arch/x86/include/uapi/asm/kvm.h | 55 + tools/include/uapi/linux/kvm.h | 2 + virt/kvm/kvm_main.c | 44 +- 54 files changed, 6717 insertions(+), 672 deletions(-)create mode 100644
Documentation/virt/kvm/intel-tdx.rstcreate mode 100644Documentation/virt/kvm/tdx-module.rstcreate mode 100644arch/x86/include/asm/kvm_boot.hcreate mode 100644arch/x86/kvm/boot/Makefilecreate mode 100644arch/x86/kvm/boot/seam/tdx_common.ccreate mode 100644arch/x86/kvm/boot/seam/tdx_common.hcreate mode 100644arch/x86/kvm/vmx/common.hcreate mode 100644arch/x86/kvm/vmx/main.ccreate mode 100644arch/x86/kvm/vmx/seamcall.Screate mode 100644arch/x86/kvm/vmx/seamcall.hcreate mode 100644arch/x86/kvm/vmx/tdx.ccreate mode 100644arch/x86/kvm/vmx/tdx.hcreate mode 100644arch/x86/kvm/vmx/tdx_arch.hcreate mode 100644arch/x86/kvm/vmx/tdx_errno.hcreate mode 100644arch/x86/kvm/vmx/tdx_ops.hcreate mode 100644arch/x86/kvm/vmx/tdx_stubs.c
Quoted 23 lines · On 03/07/21 00:04, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform. [1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.* The goal of this RFC patch
The purpose of this post is to get feedback early on high level design issue of KVM enhancement for TDX. The detailed coding (variable naming etc) is not cared of. This patch series is incomplete (not working). So it's RFC. Although multiple software components, not only KVM but also QEMU, guest Linux and virtual bios, need to be updated, this includes only KVM VMM part. For those who are curious to changes to other component, there are public repositories at github. [8], [9]* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.
Quoted 9 lines
- 01-12: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 13-40: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 41-65: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 66: main patch to add "basic" support for building/running TDX.
- 67: trace points for
- 68-69: Documentation
Queued 2,3,17-20,23,44-45, thanks.
Paolo
Quoted 201 lines
* TODOs
Those major features are missing from this patch series to keep this patch series small.- load/initialize TDX module
split out from this patch series.
- unmapping private page
Will integrate Kirill's patch to show how kvm will utilize it.
- qemu gdb stub support
- Large page support
- guest PMU support
- TDP MMU support
- and moreChanges from v1:
- rebase to v5.13
- drop load/initialization of TDX module
- catch up the update of related specifications.
- rework on C-wrapper function to invoke seamcall
- various code clean up[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAShttps://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVFIsaku Yamahata (11):
KVM: TDX: introduce config for KVM TDX support
KVM: X86: movekvm_cpu_vmxon()fromvmx.ctovirtext.h
KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
KVM: TDX: add a helper function for kvm to call seamcall
KVM: TDX: add trace point before/after TDX SEAMCALLs
KVM: TDX: Print the name of SEAMCALL status code
KVM: Add per-VM flag to mark read-only memory as unsupported
KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
KVM: TDX: add trace point for TDVMCALL and SEPT operation
KVM: TDX: add document on TDX MODULEDocumentation/virtual/kvm: Add Trust Domain Extensions(TDX)Kai Huang (2):
KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level
routes
cpu/hotplug: Document that TDX also depends on booting CPUs onceRick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bitsSean Christopherson (53):
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add architectural definitions for structures and values
KVM: TDX: define and export helper functions for KVM TDX support
KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
KVM: Exportkvm_io_bus_readfor use by TDX for PV MMIO
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Split core of hypercall emulation to helper function
KVM: x86: Exportkvm_mmiotracepoint for use by TDX for PV MMIO
KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
KVM: Add infrastructure and macro to mark VM as bugged
KVM: Exportkvm_make_all_cpus_request()for use in marking VMs as
bugged
KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the
VM
KVM: x86/mmu: Mark VM as bugged if page fault returnsRET_PF_INVALID
KVM: Addmax_vcpusfield in common 'struct kvm'
KVM: x86: Addvm_typeto differentiate legacy VMs from protected VMs
KVM: x86: Hoistkvm_dirty_regscheck out ofsync_regs()
KVM: x86: Introduce "protected guest" concept and block disallowed
ioctls
KVM: x86: Add per-VM flag to disable direct IRQ injection
KVM: x86: Add flag to disallow #MC injection /KVM_X86_SETUP_MCE
KVM: x86: Add flag to mark TSC as immutable (for TDX)
KVM: Add per-VM flag to disable dirty logging of memslots for TDs
KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
KVM: x86: Addkvm_x86_ops.cache_gprs() and .flush_gprs()
KVM: x86: Add support for vCPU and device-scopedKVM_MEMORY_ENCRYPT_OP
KVM: x86: Introducevm_teardown()hook inkvm_arch_vm_destroy()
KVM: x86: Add aswitch_db_regsflag to handle TDX's auto-switched
behavior
KVM: x86: Check for pending APICv interrupt inkvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: x86: Addguest_supported_xssplacholder
KVM: Exportkvm_is_reserved_pfn()for use by TDX
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Refactor shadow walk in__direct_map()to reduce
indentation
KVM: x86/mmu: Return old SPTE frommmu_spte_clear_track_bits()
KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
KVM: x86/mmu: Move 'pfn' variable to caller ofdirect_page_fault()
KVM: x86/mmu: Introducekvm_mmu_map_tdp_page()for use by TDX
KVM: VMX: Modify NMI and INTR handlers to takeintr_infoas param
KVM: VMX: Move NMI/exception handler to common helper
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Define EPT Violation architectural bits
KVM: VMX: Define VMCS encodings for shared EPT pointer
KVM: VMX: Add 'main.c' to wrap VMX and TDX
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: VMX: Move register caching logic to common code
KVM: TDX: Define TDCALL exit reason
KVM: TDX: Stub intdx.hwith structs, accessors, and VMCS helpers
KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
KVM: VMX: MoveAR_BYTESencoder/decoder helpers tocommon.h
KVM: VMX: MOVE GDT and IDT accessors to common code
KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX
code
KVM: TDX: Add "basic" support for building and running Trust DomainsXiaoyao Li (2):
KVM: TDX: Introducepr_seamcall_ex_ret_info()to print more info when
SEAMCALL fails
KVM: X86: Introduceinitial_tsc_khzin structkvm_archDocumentation/virt/kvm/api.rst | 6 +- Documentation/virt/kvm/intel-tdx.rst | 441 ++++++ Documentation/virt/kvm/tdx-module.rst | 48 + arch/arm64/include/asm/kvm_host.h | 3 - arch/arm64/kvm/arm.c | 7 +- arch/arm64/kvm/vgic/vgic-init.c | 6 +- arch/x86/Kbuild | 1 + arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/kvm-x86-ops.h | 8 + arch/x86/include/asm/kvm_boot.h | 30 + arch/x86/include/asm/kvm_host.h | 55 +- arch/x86/include/asm/virtext.h | 25 + arch/x86/include/asm/vmx.h | 17 + arch/x86/include/uapi/asm/kvm.h | 60 + arch/x86/include/uapi/asm/vmx.h | 7 +- arch/x86/kernel/asm-offsets_64.c | 15 + arch/x86/kvm/Kconfig | 11 + arch/x86/kvm/Makefile | 3 +- arch/x86/kvm/boot/Makefile | 6 + arch/x86/kvm/boot/seam/tdx_common.c | 242 +++ arch/x86/kvm/boot/seam/tdx_common.h | 13 + arch/x86/kvm/ioapic.c | 4 + arch/x86/kvm/irq_comm.c | 13 +- arch/x86/kvm/lapic.c | 7 +- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/mmu.h | 31 +- arch/x86/kvm/mmu/mmu.c | 526 +++++-- arch/x86/kvm/mmu/mmu_internal.h | 3 + arch/x86/kvm/mmu/paging_tmpl.h | 25 +- arch/x86/kvm/mmu/spte.c | 15 +- arch/x86/kvm/mmu/spte.h | 18 +- arch/x86/kvm/svm/svm.c | 18 +- arch/x86/kvm/trace.h | 138 ++ arch/x86/kvm/vmx/common.h | 178 +++ arch/x86/kvm/vmx/main.c | 1098 ++++++++++++++ arch/x86/kvm/vmx/posted_intr.c | 6 + arch/x86/kvm/vmx/seamcall.S | 64 + arch/x86/kvm/vmx/seamcall.h | 68 + arch/x86/kvm/vmx/tdx.c | 1958 +++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 267 ++++ arch/x86/kvm/vmx/tdx_arch.h | 370 +++++ arch/x86/kvm/vmx/tdx_errno.h | 202 +++ arch/x86/kvm/vmx/tdx_ops.h | 218 +++ arch/x86/kvm/vmx/tdx_stubs.c | 45 + arch/x86/kvm/vmx/vmcs.h | 11 - arch/x86/kvm/vmx/vmenter.S | 146 ++ arch/x86/kvm/vmx/vmx.c | 509 ++----- arch/x86/kvm/x86.c | 285 +++- include/linux/kvm_host.h | 51 +- include/uapi/linux/kvm.h | 2 + kernel/cpu.c | 4 + tools/arch/x86/include/uapi/asm/kvm.h | 55 + tools/include/uapi/linux/kvm.h | 2 + virt/kvm/kvm_main.c | 44 +- 54 files changed, 6717 insertions(+), 672 deletions(-)create mode 100644
Documentation/virt/kvm/intel-tdx.rstcreate mode 100644Documentation/virt/kvm/tdx-module.rstcreate mode 100644arch/x86/include/asm/kvm_boot.hcreate mode 100644arch/x86/kvm/boot/Makefilecreate mode 100644arch/x86/kvm/boot/seam/tdx_common.ccreate mode 100644arch/x86/kvm/boot/seam/tdx_common.hcreate mode 100644arch/x86/kvm/vmx/common.hcreate mode 100644arch/x86/kvm/vmx/main.ccreate mode 100644arch/x86/kvm/vmx/seamcall.Screate mode 100644arch/x86/kvm/vmx/seamcall.hcreate mode 100644arch/x86/kvm/vmx/tdx.ccreate mode 100644arch/x86/kvm/vmx/tdx.hcreate mode 100644arch/x86/kvm/vmx/tdx_arch.hcreate mode 100644arch/x86/kvm/vmx/tdx_errno.hcreate mode 100644arch/x86/kvm/vmx/tdx_ops.hcreate mode 100644arch/x86/kvm/vmx/tdx_stubs.c
Sean Christopherson
maintainerQuoted 15 lines · Paolo Bonzini
Quoted 4 lines · On 03/07/21 00:04, [email protected] wrote:
* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.Quoted 9 lines
- 01-12: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 13-40: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 41-65: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 66: main patch to add "basic" support for building/running TDX.
- 67: trace points for
- 68-69: Documentation
Quoted 1 line
Queued 2,3,17-20,23,44-45, thanks.
I strongly object to merging these two until we see the new SEAMLDR code:
[RFC PATCH v2 02/69] KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h [RFC PATCH v2 03/69] KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
If the SEAMLDR code ends up being fully contained in KVM, then this is unnecessary churn and exposes code outside of KVM that we may not want exposed (yet). E.g. setting and clearing CR4.VMXE (in the fault path) in cpu_vmxon() may not be necessary/desirable for SEAMLDR, we simply can't tell without seeing the code.
Paolo Bonzini
maintainerQuoted 9 lines · Sean Christopherson
I strongly object to merging these two until we see the new SEAMLDR code:
[RFC PATCH v2 02/69] KVM: X86: move
kvm_cpu_vmxon()fromvmx.ctovirtext.h[RFC PATCH v2 03/69] KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86If the SEAMLDR code ends up being fully contained in KVM, then this is unnecessary churn and exposes code outside of KVM that we may not want exposed (yet). E.g. setting and clearing CR4.VMXE (in the fault path) in
cpu_vmxon()may not be necessary/desirable for SEAMLDR, we simply can't tell without seeing the code.
Fair enough (though, for patch 2, it's a bit weird to have vmxoff in virtext.h and not vmxon).
Paolo
Sean Christopherson
maintainerQuoted 6 lines · Paolo Bonzini
Quoted 5 lines · Sean Christopherson
I strongly object to merging these two until we see the new SEAMLDR code:
[RFC PATCH v2 02/69] KVM: X86: move
kvm_cpu_vmxon()fromvmx.ctovirtext.h[RFC PATCH v2 03/69] KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
Quoted 7 lines
Quoted 4 lines
If the SEAMLDR code ends up being fully contained in KVM, then this is unnecessary churn and exposes code outside of KVM that we may not want exposed (yet). E.g. setting and clearing CR4.VMXE (in the fault path) in
cpu_vmxon()may not be necessary/desirable for SEAMLDR, we simply can't tell without seeing the code.Fair enough (though, for patch 2, it's a bit weird to have vmxoff in
virtext.hand not vmxon).
I don't really disagree, but vmxoff is already in virtext.h for the emergency reboot stuff. This series only touches the vmxon code.
Paolo Bonzini
maintainerQuoted 5 lines · Sean Christopherson
Quoted 2 lines
Fair enough (though, for patch 2, it's a bit weird to have vmxoff in
virtext.hand not vmxon).I don't really disagree, but vmxoff is already in
virtext.hfor the emergency reboot stuff. This series only touches the vmxon code.
Yes, that's what I meant. But anyway I've dequeued them.
Paolo