KVM TDX basic feature support

---
 arch/x86/kvm/Makefile      |   2 +-
 arch/x86/kvm/vmx/main.c    | 167 +++++++++++++++++
 arch/x86/kvm/vmx/vmx.c     | 370 ++++++++++---------------------------
 arch/x86/kvm/vmx/x86_ops.h | 125 +++++++++++++
 4 files changed, 394 insertions(+), 270 deletions(-)
 create mode 100644 arch/x86/kvm/vmx/main.c
 create mode 100644 arch/x86/kvm/vmx/x86_ops.h

arch/x86/kvm/Makefile +1 -1

index 80e3fe184d17..0e894ae23cbc 100644
@@ -23,7 +23,7 @@ kvm-$(CONFIG_KVM_XEN)	+= xen.o
 kvm-$(CONFIG_KVM_SMM)	+= smm.o
 
 kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \
-			   vmx/hyperv.o vmx/nested.o vmx/posted_intr.o
+			   vmx/hyperv.o vmx/nested.o vmx/posted_intr.o vmx/main.o
 kvm-intel-$(CONFIG_X86_SGX_KVM)	+= vmx/sgx.o
 
 kvm-amd-y		+= svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \

arch/x86/kvm/vmx/main.c new file +167 -0

167 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.c +101 ~~-269~~

913 lines not on this page · raw message

arch/x86/kvm/vmx/x86_ops.h new file +125 -0

125 lines not on this page · raw message

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 17:57 UTC #2 lore raw

PATCH v13 2/113 KVM: x86/vmx: Refactor KVM VMX module init/exit functions

From: Isaku Yamahata <[email protected]>

Currently, KVM VMX module initialization/exit functions are a single function each. Refactor KVM VMX module initialization functions into KVM common part and VMX part so that TDX specific part can be added cleanly. Opportunistically refactor module exit function as well.

The current module initialization flow is,
0.) Check if VMX is supported,
1.) hyper-v specific initialization,
2.) system-wide x86 specific and vendor specific initialization, 3.) Final VMX specific system-wide initialization,
4.) calculate the sizes of VMX kvm structure and VMX vcpu structure, 5.) report those sizes to the KVM common layer and KVM common
initialization

Refactor the KVM VMX module initialization function into functions with a wrapper function to separate VMX logic in vmx.c from a file, main.c, common among VMX and TDX. Introduce a wrapper function for vmx_init().

The KVM architecture common layer allocates struct kvm with reported size for architecture-specific code. The KVM VMX module defines its structure as struct vmx_kvm { struct kvm; VMX specific members;} and uses it as struct vmx kvm. Similar for vcpu structure. TDX KVM patches will define TDX specific kvm and vcpu structures.

The current module exit function is also a single function, a combination of VMX specific logic and common KVM logic. Refactor it into VMX specific logic and KVM common logic. This is just refactoring to keep the VMX specific logic in vmx.c from main.c.

---
 arch/x86/kvm/vmx/main.c    | 51 +++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c     | 54 +++++---------------------------------
 arch/x86/kvm/vmx/x86_ops.h | 13 ++++++++-
 3 files changed, 69 insertions(+), 49 deletions(-)

arch/x86/kvm/vmx/main.c +51 -0

index a59559ff140e..3f49e8e38b6b 100644
@@ -165,3 +165,54 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {
 	.runtime_ops = &vt_x86_ops,
 	.pmu_ops = &intel_pmu_ops,
 };
+
+static int __init vt_init(void)
+{
+	unsigned int vcpu_size, vcpu_align;
+	int r;
+
+	if (!kvm_is_vmx_supported())
+		return -EOPNOTSUPP;
+
+	/*
+	 * Note, hv_init_evmcs() touches only VMX knobs, i.e. there's nothing
+	 * to unwind if a later step fails.
+	 */
+	hv_init_evmcs();
+
+	r = kvm_x86_vendor_init(&vt_init_ops);
+	if (r)
+		return r;
+
+	r = vmx_init();
+	if (r)
+		goto err_vmx_init;
+
+	/*
+	 * Common KVM initialization _must_ come last, after this, /dev/kvm is
+	 * exposed to userspace!
+	 */
+	vt_x86_ops.vm_size = sizeof(struct kvm_vmx);
+	vcpu_size = sizeof(struct vcpu_vmx);
+	vcpu_align = __alignof__(struct vcpu_vmx);
+	r = kvm_init(vcpu_size, vcpu_align, THIS_MODULE);
+	if (r)
+		goto err_kvm_init;
+
+	return 0;
+
+err_kvm_init:
+	vmx_exit();
+err_vmx_init:
+	kvm_x86_vendor_exit();
+	return r;
+}
+module_init(vt_init);
+
+static void vt_exit(void)
+{
+	kvm_exit();
+	kvm_x86_vendor_exit();
+	vmx_exit();
+}
+module_exit(vt_exit);

arch/x86/kvm/vmx/vmx.c +6 ~~-48~~

104 lines not on this page · raw message

arch/x86/kvm/vmx/x86_ops.h +12 -1

index e9ec4d259ff5..051b5c4b5c2f 100644
@@ -8,11 +8,22 @@
 
 #include "x86.h"
 
-__init int vmx_hardware_setup(void);
+#if IS_ENABLED(CONFIG_HYPERV)
+__init void hv_init_evmcs(void);
+void hv_reset_evmcs(void);
+#else /* IS_ENABLED(CONFIG_HYPERV) */
+static inline void hv_init_evmcs(void) {}
+static inline void hv_reset_evmcs(void) {}
+#endif /* IS_ENABLED(CONFIG_HYPERV) */
+
+bool kvm_is_vmx_supported(void);
+int __init vmx_init(void);
+void vmx_exit(void);
 
 extern struct kvm_x86_ops vt_x86_ops __initdata;
 extern struct kvm_x86_init_ops vt_init_ops __initdata;
 
+__init int vmx_hardware_setup(void);
 void vmx_hardware_unsetup(void);
 int vmx_check_processor_compat(void);
 int vmx_hardware_enable(void);

Signature · 1 line

2.25.1

Wang, Wei W

2023-03-13 14:49 UTC #3 lore raw

RE: [PATCH v13 002/113] KVM: x86/vmx: Refactor KVM VMX module init/exit functions

Quoted 77 lines · On Monday, March 13, 2023 1:55 AM, [email protected] wrote:

Currently, KVM VMX module initialization/exit functions are a single function each. Refactor KVM VMX module initialization functions into KVM common part and VMX part so that TDX specific part can be added cleanly. Opportunistically refactor module exit function as well.
The current module initialization flow is,
0.) Check if VMX is supported,
1.) hyper-v specific initialization,
2.) system-wide x86 specific and vendor specific initialization, 3.) Final VMX specific system-wide initialization,
4.) calculate the sizes of VMX kvm structure and VMX vcpu structure, 5.) report those sizes to the KVM common layer and KVM common
initialization
Refactor the KVM VMX module initialization function into functions with a wrapper function to separate VMX logic in vmx.c from a file, main.c, common among VMX and TDX. Introduce a wrapper function for vmx_init().
The KVM architecture common layer allocates struct kvm with reported size for architecture-specific code. The KVM VMX module defines its structure as struct vmx_kvm { struct kvm; VMX specific members;} and uses it as struct vmx kvm. Similar for vcpu structure. TDX KVM patches will define TDX specific kvm and vcpu structures.
The current module exit function is also a single function, a combination of VMX specific logic and common KVM logic. Refactor it into VMX specific logic and KVM common logic. This is just refactoring to keep the VMX specific logic in vmx.c from main.c.
Signed-off-by Isaku Yamahata <[email protected]>
---
 arch/x86/kvm/vmx/main.c    | 51 +++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c     | 54 +++++---------------------------------
 arch/x86/kvm/vmx/x86_ops.h | 13 ++++++++-
 3 files changed, 69 insertions(+), 49 deletions(-)
arch/x86/kvm/vmx/main.c index renamed from arch/x86/kvm/vmx/main.c +0 -0
a59559ff140e..3f49e8e38b6b 100644
arch/x86/kvm/vmx/main.c +33 -0
@@ -165,3 +165,54 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {
 	.runtime_ops = &vt_x86_ops,
 	.pmu_ops = &intel_pmu_ops,
 };
+
+static int __init vt_init(void)
+{
+	unsigned int vcpu_size, vcpu_align;
+	int r;
+
+	if (!kvm_is_vmx_supported())
+		return -EOPNOTSUPP;
+
+	/*
+	 * Note, hv_init_evmcs() touches only VMX knobs, i.e. there's nothing
+	 * to unwind if a later step fails.
+	 */
+	hv_init_evmcs();
+
+	r = kvm_x86_vendor_init(&vt_init_ops);
+	if (r)
+		return r;
+
+	r = vmx_init();
+	if (r)
+		goto err_vmx_init;
+
+	/*
+	 * Common KVM initialization _must_ come last, after this, /dev/kvm is
+	 * exposed to userspace!
+	 */
+	vt_x86_ops.vm_size = sizeof(struct kvm_vmx);
+	vcpu_size = sizeof(struct vcpu_vmx);
+	vcpu_align = __alignof__(struct vcpu_vmx);
+	r = kvm_init(vcpu_size, vcpu_align, THIS_MODULE);
+	if (r)
+		goto err_kvm_init;

arch/x86/kvm/vmx/main.c struct kvm_x86_init_ops vt_init_ops __initdata = { · line 210

Quoted 10 lines

arch/x86/kvm/vmx/main.c +10 -0

+
+	return 0;
+
+err_kvm_init:
+	vmx_exit();
+err_vmx_init:
+	kvm_x86_vendor_exit();
+	return r;
+}
+module_init(vt_init);

I had a patch to fix a bug here, maybe you can take it:

kvm_x86_vendor_init copies vt_x86_ops to kvm_x86_ops. vt_x86_ops.vm_size needs to be updated before calling kvm_x86_vendor_init so that kvm_x86_ops can get the correct vm_size.

arch/x86/kvm/vmx/main.c +13 ~~-12~~

index 41f07d92aaf6..37a8c212c653 100644
@@ -1126,18 +1126,6 @@ static int __init vt_init(void)
         */
        hv_init_evmcs();
 
-       r = kvm_x86_vendor_init(&vt_init_ops);
-       if (r)
-               return r;
-
-       r = vmx_init();
-       if (r)
-               goto err_vmx_init;
-
-       r = tdx_init();
-       if (r)
-               goto err_tdx_init;
-
        /*
         * Common KVM initialization _must_ come last, after this, /dev/kvm is
         * exposed to userspace!

@@ -1153,6 +1141,19 @@ static int __init vt_init(void)
                vcpu_align = max_t(unsigned int, vcpu_align,
                                   __alignof__(struct vcpu_tdx));
        }
+
+       r = kvm_x86_vendor_init(&vt_init_ops);
+       if (r)
+               return r;
+
+       r = vmx_init();
+       if (r)
+               goto err_vmx_init;
+
+       r = tdx_init();
+       if (r)
+               goto err_tdx_init;
+
        r = kvm_init(vcpu_size, vcpu_align, THIS_MODULE);
        if (r)
                goto err_kvm_init;

Isaku Yamahata

2023-03-13 18:41 UTC #4 lore raw

Re: [PATCH v13 002/113] KVM: x86/vmx: Refactor KVM VMX module init/exit functions

Quoted 91 lines · Wei W

Quoted 77 lines · On Monday, March 13, 2023 1:55 AM, [email protected] wrote:
Currently, KVM VMX module initialization/exit functions are a single function each. Refactor KVM VMX module initialization functions into KVM common part and VMX part so that TDX specific part can be added cleanly. Opportunistically refactor module exit function as well.
The current module initialization flow is,
0.) Check if VMX is supported,
1.) hyper-v specific initialization,
2.) system-wide x86 specific and vendor specific initialization, 3.) Final VMX specific system-wide initialization,
4.) calculate the sizes of VMX kvm structure and VMX vcpu structure, 5.) report those sizes to the KVM common layer and KVM common
initialization
Refactor the KVM VMX module initialization function into functions with a wrapper function to separate VMX logic in vmx.c from a file, main.c, common among VMX and TDX. Introduce a wrapper function for vmx_init().
The KVM architecture common layer allocates struct kvm with reported size for architecture-specific code. The KVM VMX module defines its structure as struct vmx_kvm { struct kvm; VMX specific members;} and uses it as struct vmx kvm. Similar for vcpu structure. TDX KVM patches will define TDX specific kvm and vcpu structures.
The current module exit function is also a single function, a combination of VMX specific logic and common KVM logic. Refactor it into VMX specific logic and KVM common logic. This is just refactoring to keep the VMX specific logic in vmx.c from main.c.
Signed-off-by Isaku Yamahata <[email protected]>
---
 arch/x86/kvm/vmx/main.c    | 51 +++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c     | 54 +++++---------------------------------
 arch/x86/kvm/vmx/x86_ops.h | 13 ++++++++-
 3 files changed, 69 insertions(+), 49 deletions(-)
arch/x86/kvm/vmx/main.c index renamed from arch/x86/kvm/vmx/main.c +0 -0
a59559ff140e..3f49e8e38b6b 100644
arch/x86/kvm/vmx/main.c +33 -0
@@ -165,3 +165,54 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {
 	.runtime_ops = &vt_x86_ops,
 	.pmu_ops = &intel_pmu_ops,
 };
+
+static int __init vt_init(void)
+{
+	unsigned int vcpu_size, vcpu_align;
+	int r;
+
+	if (!kvm_is_vmx_supported())
+		return -EOPNOTSUPP;
+
+	/*
+	 * Note, hv_init_evmcs() touches only VMX knobs, i.e. there's nothing
+	 * to unwind if a later step fails.
+	 */
+	hv_init_evmcs();
+
+	r = kvm_x86_vendor_init(&vt_init_ops);
+	if (r)
+		return r;
+
+	r = vmx_init();
+	if (r)
+		goto err_vmx_init;
+
+	/*
+	 * Common KVM initialization _must_ come last, after this, /dev/kvm is
+	 * exposed to userspace!
+	 */
+	vt_x86_ops.vm_size = sizeof(struct kvm_vmx);
+	vcpu_size = sizeof(struct vcpu_vmx);
+	vcpu_align = __alignof__(struct vcpu_vmx);
+	r = kvm_init(vcpu_size, vcpu_align, THIS_MODULE);
+	if (r)
+		goto err_kvm_init;
arch/x86/kvm/vmx/main.c struct kvm_x86_init_ops vt_init_ops __initdata = { · line 210
Quoted 10 lines
arch/x86/kvm/vmx/main.c +10 -0
+
+	return 0;
+
+err_kvm_init:
+	vmx_exit();
+err_vmx_init:
+	kvm_x86_vendor_exit();
+	return r;
+}
+module_init(vt_init);
I had a patch to fix a bug here, maybe you can take it:

Quoted 3 lines

kvm_x86_vendor_init copies vt_x86_ops to kvm_x86_ops. vt_x86_ops.vm_size needs to be updated before calling kvm_x86_vendor_init so that kvm_x86_ops can get the correct vm_size.

Thanks for catching it. With your patch, vm_size is always max(sizeof struct kvm_vmx, sizeof strut kvm_tdx) even when the admin sets kvm_intel.tdx=true and tdx is disabled by error.

option 1: Ignore such waste. Your patch. The difference is small and it's only

          the error case. Locally I have the following values.
          sizeof(struct kvm_vmx) = 44576
          sizeof(struct vcpu_vmx) = 10432
          sizeof(struct kvm_tdx)= 44632
          sizeof(struct vcpu_tdx) = 8192
          I suspect the actual allocation size for struct kvm is same.  That's
          the reason why I didn't hit problem.

option 2: Explicitly update vm_size after kvm_x86_vendor_init()
struct kvm_x86_ops isn't exported. It would be ugly.

option 3: Allow setup_hardware() to update vm_size.
setup_hardware(void) => setup_hardware(unsigned int *vm_size) It's confusing because kvm_x86_ops.vm_size is already initialized.

Let's go with option 1(your patch).

Signature · 1 line

Isaku Yamahata <[email protected]>

Wang, Wei W

2023-03-14 01:58 UTC #5 lore raw

RE: [PATCH v13 002/113] KVM: x86/vmx: Refactor KVM VMX module init/exit functions

Quoted 9 lines · Isaku Yamahata

Quoted 5 lines
I had a patch to fix a bug here, maybe you can take it:
kvm_x86_vendor_init copies vt_x86_ops to kvm_x86_ops. vt_x86_ops.vm_size needs to be updated before calling kvm_x86_vendor_init so that kvm_x86_ops can get the correct vm_size.
Thanks for catching it. With your patch, vm_size is always max(sizeof struct kvm_vmx, sizeof strut kvm_tdx) even when the admin sets kvm_intel.tdx=true and tdx is disabled by error.

This seems to be another bug, where enable_tdx should be set to false if tdx fails to be enabled:

arch/x86/kvm/vmx/tdx.c +1 -0

index d6a9f705a2a1..ef1e794efb97 100644
@@ -4552,6 +4552,7 @@ static int tdx_module_setup(void)
 
        ret = tdx_enable();
        if (ret) {
+               enable_tdx = false;
                pr_info("Failed to initialize TDX module.\n");
                return ret;
        }

Quoted 9 lines

option 1: Ignore such waste. Your patch. The difference is small and it's only

          the error case. Locally I have the following values.
          sizeof(struct kvm_vmx) = 44576
          sizeof(struct vcpu_vmx) = 10432
          sizeof(struct kvm_tdx)= 44632
          sizeof(struct vcpu_tdx) = 8192
          I suspect the actual allocation size for struct kvm is same.  That's
          the reason why I didn't hit problem.

No, the actual allocation size isn't same.
You didn’t get error notices because the kvm_tdx fields are still located in the same page as that allocated for kvm_vmx, though that part of memory isn’t explicitly allocated. If kvm_tdx size grows and crosses the page boundary, you will see unexpected faults. Or if this part of memory (illegally used by kvm_tdx) later happens to be given to other kernel components to use, you would see random errors somewhere.

Quoted 9 lines

option 2: Explicitly update vm_size after kvm_x86_vendor_init()
struct kvm_x86_ops isn't exported. It would be ugly.
option 3: Allow setup_hardware() to update vm_size.
setup_hardware(void) => setup_hardware(unsigned int *vm_size) It's confusing because kvm_x86_ops.vm_size is already initialized.
Let's go with option 1(your patch).

Sounds good.

Wen, Qian

2023-05-23 02:24 UTC #6 lore raw

Re: [PATCH v13 002/113] KVM: x86/vmx: Refactor KVM VMX module init/exit functions

Quoted 64 lines · On 3/13/2023 1:55 AM, [email protected] wrote:

From: Isaku Yamahata <[email protected]>
Currently, KVM VMX module initialization/exit functions are a single function each. Refactor KVM VMX module initialization functions into KVM common part and VMX part so that TDX specific part can be added cleanly. Opportunistically refactor module exit function as well.
The current module initialization flow is,
0.) Check if VMX is supported,
1.) hyper-v specific initialization,
2.) system-wide x86 specific and vendor specific initialization, 3.) Final VMX specific system-wide initialization,
4.) calculate the sizes of VMX kvm structure and VMX vcpu structure, 5.) report those sizes to the KVM common layer and KVM common
initialization
Refactor the KVM VMX module initialization function into functions with a wrapper function to separate VMX logic in vmx.c from a file, main.c, common among VMX and TDX. Introduce a wrapper function for vmx_init().
The KVM architecture common layer allocates struct kvm with reported size for architecture-specific code. The KVM VMX module defines its structure as struct vmx_kvm { struct kvm; VMX specific members;} and uses it as struct vmx kvm. Similar for vcpu structure. TDX KVM patches will define TDX specific kvm and vcpu structures.
The current module exit function is also a single function, a combination of VMX specific logic and common KVM logic. Refactor it into VMX specific logic and KVM common logic. This is just refactoring to keep the VMX specific logic in vmx.c from main.c.
Signed-off-by Isaku Yamahata <[email protected]>
---
 arch/x86/kvm/vmx/main.c    | 51 +++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c     | 54 +++++---------------------------------
 arch/x86/kvm/vmx/x86_ops.h | 13 ++++++++-
 3 files changed, 69 insertions(+), 49 deletions(-)
arch/x86/kvm/vmx/main.c +18 -0
index a59559ff140e..3f49e8e38b6b 100644
@@ -165,3 +165,54 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {
 	.runtime_ops = &vt_x86_ops,
 	.pmu_ops = &intel_pmu_ops,
 };
+
+static int __init vt_init(void)
+{
+	unsigned int vcpu_size, vcpu_align;
+	int r;
+
+	if (!kvm_is_vmx_supported())
+		return -EOPNOTSUPP;
+
+	/*
+	 * Note, hv_init_evmcs() touches only VMX knobs, i.e. there's nothing
+	 * to unwind if a later step fails.
+	 */
+	hv_init_evmcs();
+
+	r = kvm_x86_vendor_init(&vt_init_ops);
+	if (r)
+		return r;

arch/x86/kvm/vmx/main.c struct kvm_x86_init_ops vt_init_ops __initdata = { · line 195

Quoted 10 lines

arch/x86/kvm/vmx/main.c +10 -0

+
+	r = vmx_init();
+	if (r)
+		goto err_vmx_init;
+
+	/*
+	 * Common KVM initialization _must_ come last, after this, /dev/kvm is
+	 * exposed to userspace!
+	 */
+	vt_x86_ops.vm_size = sizeof(struct kvm_vmx);

nit: why initialize vm_size again? I noticed that it has already been assigned a value when create vt_x86_ops.

+struct kvm_x86_ops vt_x86_ops __initdata = {
...
+ .vm_size = sizeof(struct kvm_vmx),
+ .vm_init = vmx_vm_init,

Quoted 167 lines

arch/x86/kvm/vmx/main.c +23 -0

+	vcpu_size = sizeof(struct vcpu_vmx);
+	vcpu_align = __alignof__(struct vcpu_vmx);
+	r = kvm_init(vcpu_size, vcpu_align, THIS_MODULE);
+	if (r)
+		goto err_kvm_init;
+
+	return 0;
+
+err_kvm_init:
+	vmx_exit();
+err_vmx_init:
+	kvm_x86_vendor_exit();
+	return r;
+}
+module_init(vt_init);
+
+static void vt_exit(void)
+{
+	kvm_exit();
+	kvm_x86_vendor_exit();
+	vmx_exit();
+}
+module_exit(vt_exit);

arch/x86/kvm/vmx/vmx.c +6 ~~-48~~

index 47a9a647ae3a..3bbd07412f00 100644
@@ -553,7 +553,7 @@ static int hv_enable_l2_tlb_flush(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
-static __init void hv_init_evmcs(void)
+__init void hv_init_evmcs(void)
 {
 	int cpu;

@@ -589,7 +589,7 @@ static __init void hv_init_evmcs(void)
 	}
 }
 
-static void hv_reset_evmcs(void)
+void hv_reset_evmcs(void)
 {
 	struct hv_vp_assist_page *vp_ap;

@@ -613,10 +613,6 @@ static void hv_reset_evmcs(void)
 	vp_ap->current_nested_vmcs = 0;
 	vp_ap->enlighten_vmentry = 0;
 }
-
-#else /* IS_ENABLED(CONFIG_HYPERV) */
-static void hv_init_evmcs(void) {}
-static void hv_reset_evmcs(void) {}
 #endif /* IS_ENABLED(CONFIG_HYPERV) */
 
 /*

@@ -2741,7 +2737,7 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 	return 0;
 }
 
-static bool kvm_is_vmx_supported(void)
+bool kvm_is_vmx_supported(void)
 {
 	int cpu = raw_smp_processor_id();

@@ -8381,7 +8377,7 @@ static void vmx_cleanup_l1d_flush(void)
 	l1tf_vmx_mitigation = VMENTER_L1D_FLUSH_AUTO;
 }
 
-static void __vmx_exit(void)
+void vmx_exit(void)
 {
 	allow_smaller_maxphyaddr = false;

@@ -8392,32 +8388,10 @@ static void __vmx_exit(void)
 	vmx_cleanup_l1d_flush();
 }
 
-static void vmx_exit(void)
-{
-	kvm_exit();
-	kvm_x86_vendor_exit();
-
-	__vmx_exit();
-}
-module_exit(vmx_exit);
-
-static int __init vmx_init(void)
+int __init vmx_init(void)
 {
 	int r, cpu;
 
-	if (!kvm_is_vmx_supported())
-		return -EOPNOTSUPP;
-
-	/*
-	 * Note, hv_init_evmcs() touches only VMX knobs, i.e. there's nothing
-	 * to unwind if a later step fails.
-	 */
-	hv_init_evmcs();
-
-	r = kvm_x86_vendor_init(&vt_init_ops);
-	if (r)
-		return r;
-
 	/*
 	 * Must be called after common x86 init so enable_ept is properly set
 	 * up. Hand the parameter mitigation value in which was stored in

@@ -8427,7 +8401,7 @@ static int __init vmx_init(void)
 	 */
 	r = vmx_setup_l1d_flush(vmentry_l1d_flush_param);
 	if (r)
-		goto err_l1d_flush;
+		return r;
 
 	vmx_setup_fb_clear_ctrl();

@@ -8451,21 +8425,5 @@ static int __init vmx_init(void)
 	if (!enable_ept)
 		allow_smaller_maxphyaddr = true;
 
-	/*
-	 * Common KVM initialization _must_ come last, after this, /dev/kvm is
-	 * exposed to userspace!
-	 */
-	r = kvm_init(sizeof(struct vcpu_vmx), __alignof__(struct vcpu_vmx),
-		     THIS_MODULE);
-	if (r)
-		goto err_kvm_init;
-
 	return 0;
-
-err_kvm_init:
-	__vmx_exit();
-err_l1d_flush:
-	kvm_x86_vendor_exit();
-	return r;
 }
-module_init(vmx_init);

arch/x86/kvm/vmx/x86_ops.h +12 -1

index e9ec4d259ff5..051b5c4b5c2f 100644
@@ -8,11 +8,22 @@
 
 #include "x86.h"
 
-__init int vmx_hardware_setup(void);
+#if IS_ENABLED(CONFIG_HYPERV)
+__init void hv_init_evmcs(void);
+void hv_reset_evmcs(void);
+#else /* IS_ENABLED(CONFIG_HYPERV) */
+static inline void hv_init_evmcs(void) {}
+static inline void hv_reset_evmcs(void) {}
+#endif /* IS_ENABLED(CONFIG_HYPERV) */
+
+bool kvm_is_vmx_supported(void);
+int __init vmx_init(void);
+void vmx_exit(void);
 
 extern struct kvm_x86_ops vt_x86_ops __initdata;
 extern struct kvm_x86_init_ops vt_init_ops __initdata;
 
+__init int vmx_hardware_setup(void);
 void vmx_hardware_unsetup(void);
 int vmx_check_processor_compat(void);
 int vmx_hardware_enable(void);

Isaku Yamahata

2023-05-28 06:10 UTC #7 lore raw

Re: [PATCH v13 002/113] KVM: x86/vmx: Refactor KVM VMX module init/exit functions

Quoted 30 lines · Qian

Quoted 30 lines

arch/x86/kvm/vmx/main.c +22 -0

index a59559ff140e..3f49e8e38b6b 100644
@@ -165,3 +165,54 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {
 	.runtime_ops = &vt_x86_ops,
 	.pmu_ops = &intel_pmu_ops,
 };
+
+static int __init vt_init(void)
+{
+	unsigned int vcpu_size, vcpu_align;
+	int r;
+
+	if (!kvm_is_vmx_supported())
+		return -EOPNOTSUPP;
+
+	/*
+	 * Note, hv_init_evmcs() touches only VMX knobs, i.e. there's nothing
+	 * to unwind if a later step fails.
+	 */
+	hv_init_evmcs();
+
+	r = kvm_x86_vendor_init(&vt_init_ops);
+	if (r)
+		return r;
+
+	r = vmx_init();
+	if (r)
+		goto err_vmx_init;

Quoted 8 lines

arch/x86/kvm/vmx/main.c struct kvm_x86_init_ops vt_init_ops __initdata = { · line 195
Quoted 6 lines
arch/x86/kvm/vmx/main.c +6 -0
+
+	/*
+	 * Common KVM initialization _must_ come last, after this, /dev/kvm is
+	 * exposed to userspace!
+	 */
+	vt_x86_ops.vm_size = sizeof(struct kvm_vmx);
nit: why initialize vm_size again? I noticed that it has already been assigned a value when create vt_x86_ops.

Thanks, will remove the line.

Signature · 1 line

Isaku Yamahata <[email protected]>

[email protected]

author

2023-03-12 17:57 UTC #8 lore raw

PATCH v13 3/113 KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

From: Isaku Yamahata <[email protected]>

TDX requires several initialization steps for KVM to create guest TDs. Detect CPU feature, enable VMX (TDX is based on VMX), detect the TDX module availability, and initialize it.

There are several options on when to initialize the TDX module. A.) kernel module loading time, B.) the first guest TD creation time. A.) was chosen. With B.), a user may hit an error of the TDX initialization when trying to create the first guest TD. The machine that fails to initialize the TDX module can't boot any guest TD further. Such failure is undesirable and a surprise because the user expects that the machine can accommodate guest TD, but actually not. So A.) is better than B.).

Introduce a module parameter, enable_tdx, to explicitly enable TDX KVM support. It's off by default to keep same behavior for those who don't use TDX. Implement hardware_setup method to detect TDX feature of CPU. Because TDX requires all present CPUs to enable VMX (VMXON), the x86 specific kvm_arch_post_hardware_enable_setup overrides the existing weak symbol of kvm_arch_post_hardware_enable_setup which is called at the KVM module initialization.

---
 arch/x86/kvm/Makefile      |  1 +
 arch/x86/kvm/vmx/main.c    | 18 ++++++++++-
 arch/x86/kvm/vmx/tdx.c     | 63 ++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c     | 39 +++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h | 10 ++++++
 5 files changed, 130 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kvm/vmx/tdx.c

arch/x86/kvm/Makefile +1 -0

index 0e894ae23cbc..4b01ab842ab7 100644
@@ -25,6 +25,7 @@ kvm-$(CONFIG_KVM_SMM)	+= smm.o
 kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \
 			   vmx/hyperv.o vmx/nested.o vmx/posted_intr.o vmx/main.o
 kvm-intel-$(CONFIG_X86_SGX_KVM)	+= vmx/sgx.o
+kvm-intel-$(CONFIG_INTEL_TDX_HOST)	+= vmx/tdx.o
 
 kvm-amd-y		+= svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \
 			   svm/sev.o svm/hyperv.o

arch/x86/kvm/vmx/main.c +17 -1

index 3f49e8e38b6b..5c9f5e00b3c4 100644
@@ -6,6 +6,22 @@
 #include "nested.h"
 #include "pmu.h"
 
+static bool enable_tdx __ro_after_init;
+module_param_named(tdx, enable_tdx, bool, 0444);
+
+static __init int vt_hardware_setup(void)
+{
+	int ret;
+
+	ret = vmx_hardware_setup();
+	if (ret)
+		return ret;
+
+	enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops);
+
+	return 0;
+}
+
 #define VMX_REQUIRED_APICV_INHIBITS		       \
 (						       \
        BIT(APICV_INHIBIT_REASON_DISABLE)|	       \

@@ -159,7 +175,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {
-	.hardware_setup = vmx_hardware_setup,
+	.hardware_setup = vt_hardware_setup,
 	.handle_intel_pt_intr = NULL,
 
 	.runtime_ops = &vt_x86_ops,

arch/x86/kvm/vmx/tdx.c new file +63 -0

63 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.c +39 -0

index 3bbd07412f00..d23830d92f61 100644
@@ -8123,6 +8123,45 @@ static unsigned int vmx_handle_intel_pt_intr(void)
 	return 1;
 }
 
+static __init void vmxon(void *arg)
+{
+	int cpu = raw_smp_processor_id();
+	u64 phys_addr = __pa(per_cpu(vmxarea, cpu));
+	atomic_t *failed = arg;
+	int r;
+
+	if (cr4_read_shadow() & X86_CR4_VMXE) {
+		r = -EBUSY;
+		goto out;
+	}
+
+	r = kvm_cpu_vmxon(phys_addr);
+out:
+	if (r)
+		atomic_inc(failed);
+}
+
+__init int vmxon_all(void)
+{
+	atomic_t failed = ATOMIC_INIT(0);
+
+	on_each_cpu(vmxon, &failed, 1);
+
+	if (atomic_read(&failed))
+		return -EBUSY;
+	return 0;
+}
+
+static __init void vmxoff(void *junk)
+{
+	cpu_vmxoff();
+}
+
+__init void vmxoff_all(void)
+{
+	on_each_cpu(vmxoff, NULL, 1);
+}
+
 static __init void vmx_setup_user_return_msrs(void)
 {

arch/x86/kvm/vmx/x86_ops.h +10 -0

index 051b5c4b5c2f..0f200aead411 100644
@@ -20,6 +20,10 @@ bool kvm_is_vmx_supported(void);
 int __init vmx_init(void);
 void vmx_exit(void);
 
+__init int vmxon_all(void);
+__init void vmxoff_all(void);
+__init int vmx_hardware_setup(void);
+
 extern struct kvm_x86_ops vt_x86_ops __initdata;
 extern struct kvm_x86_init_ops vt_init_ops __initdata;

@@ -133,4 +137,10 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu);
 #endif
 void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
+#ifdef CONFIG_INTEL_TDX_HOST
+int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
+#else
+static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
+#endif
+
 #endif /* __KVM_X86_VMX_X86_OPS_H */

Signature · 1 line

2.25.1

Huang, Kai

2023-03-14 02:38 UTC #9 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Review

Quoted 13 lines · On Sun, 2023-03-12 at 10:55 -0700, [email protected] wrote:

(quoted diff) +13 -0

+int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
+{
+	int r;
+
+	if (!enable_ept) {
+		pr_warn("Cannot enable TDX with EPT disabled\n");
+		return -EINVAL;
+	}
+
+	/* tdx_enable() in tdx_module_setup() requires cpus lock. */
+	cpus_read_lock();
+	/* TDX requires VMX. */
+	r = vmxon_all();

Why not using hardware_enable_all()?

Quoted 9 lines

(quoted diff) +9 -0

+	if (!r) {
+		int cpu;
+
+		/*
+		 * Because tdx_cpu_enabel() acquire spin locks, on_each_cpu()
+		 * can't be used.
+		 */
+		for_each_online_cpu(cpu) {
+			if (smp_call_on_cpu(cpu, tdx_cpu_enable_cpu, NULL, false))

Review

Quoted 10 lines

(quoted diff) +10 -0

+				r = -EIO;
+		}
+		if (!r)
+			r = tdx_module_setup();
+	}
+	vmxoff_all();
+	cpus_read_unlock();
+
+	return r;
+}

I think you should use hardware_enable_all(), and just do something similar to below in vmx_hardware_enable():

arch/x86/kvm/vmx/vmx.c +9 -0

index 7eec0226d56a..b7b3f99c0095 100644
@@ -2504,6 +2504,15 @@ static int vmx_hardware_enable(void)
                return r;
        }
 
+       if (enable_tdx) {
+               r = tdx_cpu_enable();
+               if (r) {
+                       cpu_vmxoff();
+                       intel_pt_handle_vmx(0);
+                       return r;
+               }
+       }
+
        if (enable_ept)
                ept_sync_global();

It handles two cases automatically:

1) When user wants to use TDX while loading KVM module, if above fails, then hardware_enable_all() returns error, and you just give up initializing TDX module and mark enable_tdx as false.

2) When a new cpu becomes online, and when TDX is being used by KVM, then if above fails, it automatically rejects the CPU which isn't TDX-runnable although it is VMX-runnable.

Isaku Yamahata

2023-03-15 07:27 UTC #10 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Quoted 31 lines · Kai

Review

Quoted 13 lines · On Sun, 2023-03-12 at 10:55 -0700, [email protected] wrote:

(quoted diff) +13 -0

+int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
+{
+	int r;
+
+	if (!enable_ept) {
+		pr_warn("Cannot enable TDX with EPT disabled\n");
+		return -EINVAL;
+	}
+
+	/* tdx_enable() in tdx_module_setup() requires cpus lock. */
+	cpus_read_lock();
+	/* TDX requires VMX. */
+	r = vmxon_all();

Why not using hardware_enable_all()?

Quoted 14 lines

(quoted diff) +14 -0

+	if (!r) {
+		int cpu;
+
+		/*
+		 * Because tdx_cpu_enabel() acquire spin locks, on_each_cpu()
+		 * can't be used.
+		 */
+		for_each_online_cpu(cpu) {
+			if (smp_call_on_cpu(cpu, tdx_cpu_enable_cpu, NULL, false))
+				r = -EIO;
+		}
+		if (!r)
+			r = tdx_module_setup();
+	}

Quoted 8 lines

Review
Quoted 5 lines
(quoted diff) +5 -0
+	vmxoff_all();
+	cpus_read_unlock();
+
+	return r;
+}
I think you should use hardware_enable_all(), and just do something similar to below in vmx_hardware_enable():

The use of hardware_enable_all() make us circle back to refactoring KVM hardware initialization topic. I'd like to stay away from it for now for TDX. I find that vmxon_all() is unnecessary and we can move VMXON to tdx_cpu_enable_cpu().
Here is the version of dropping vmxon_all().

From f8fa8fe9786f1c4d4a3b0af0d0228d2842984cba Mon Sep 17 00:00:00 2001
Message-Id: <f8fa8fe9786f1c4d4a3b0af0d0228d2842984cba.1678864879.git.isaku.yamahata@intel.com>
In-Reply-To: <d2aa2142665b8204b628232ab615c98090371c99.1678864879.git.isaku.yamahata@intel.com>
References: <d2aa2142665b8204b628232ab615c98090371c99.1678864879.git.isaku.yamahata@intel.com>
From: Isaku Yamahata <[email protected]>
Date: Tue, 22 Feb 2022 14:44:15 -0800
Subject: [PATCH] KVM: TDX: Initialize the TDX module when loading the
KVM intel kernel module

TDX requires several initialization steps for KVM to create guest TDs. Detect CPU feature, enable VMX (TDX is based on VMX), detect the TDX module availability, and initialize it.

Introduce a module parameter, kvm_intel.tdx, to explicitly enable TDX KVM support. It's off by default to keep same behavior for those who don't use TDX. Implement hardware_setup method to detect TDX feature of CPU and initialize TDX module.

---
 arch/x86/kvm/Makefile      |  1 +
 arch/x86/kvm/vmx/main.c    | 18 +++++++++-
 arch/x86/kvm/vmx/tdx.c     | 74 ++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c     |  7 ++++
 arch/x86/kvm/vmx/x86_ops.h |  9 +++++
 5 files changed, 108 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kvm/vmx/tdx.c

arch/x86/kvm/Makefile +1 -0

index 0e894ae23cbc..4b01ab842ab7 100644
@@ -25,6 +25,7 @@ kvm-$(CONFIG_KVM_SMM)	+= smm.o
 kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \
 			   vmx/hyperv.o vmx/nested.o vmx/posted_intr.o vmx/main.o
 kvm-intel-$(CONFIG_X86_SGX_KVM)	+= vmx/sgx.o
+kvm-intel-$(CONFIG_INTEL_TDX_HOST)	+= vmx/tdx.o
 
 kvm-amd-y		+= svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \
 			   svm/sev.o svm/hyperv.o

arch/x86/kvm/vmx/main.c +17 -1

index 3f49e8e38b6b..5c9f5e00b3c4 100644
@@ -6,6 +6,22 @@
 #include "nested.h"
 #include "pmu.h"
 
+static bool enable_tdx __ro_after_init;
+module_param_named(tdx, enable_tdx, bool, 0444);
+
+static __init int vt_hardware_setup(void)
+{
+	int ret;
+
+	ret = vmx_hardware_setup();
+	if (ret)
+		return ret;
+
+	enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops);
+
+	return 0;
+}
+
 #define VMX_REQUIRED_APICV_INHIBITS		       \
 (						       \
        BIT(APICV_INHIBIT_REASON_DISABLE)|	       \

@@ -159,7 +175,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {
-	.hardware_setup = vmx_hardware_setup,
+	.hardware_setup = vt_hardware_setup,
 	.handle_intel_pt_intr = NULL,
 
 	.runtime_ops = &vt_x86_ops,

arch/x86/kvm/vmx/tdx.c new file +74 -0

74 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.c +7 -0

index 3bbd07412f00..ce48e0bc9e00 100644
@@ -8123,6 +8123,13 @@ static unsigned int vmx_handle_intel_pt_intr(void)
 	return 1;
 }
 
+__init int vmxon(void)
+{
+	if (cr4_read_shadow() & X86_CR4_VMXE)
+		return -EBUSY;
+	return kvm_cpu_vmxon(__pa(this_cpu_read(vmxarea)));
+}
+
 static __init void vmx_setup_user_return_msrs(void)
 {

arch/x86/kvm/vmx/x86_ops.h +9 -0

index 051b5c4b5c2f..892c39f7d771 100644
@@ -20,6 +20,9 @@ bool kvm_is_vmx_supported(void);
 int __init vmx_init(void);
 void vmx_exit(void);
 
+__init int vmxon(void);
+__init int vmx_hardware_setup(void);
+
 extern struct kvm_x86_ops vt_x86_ops __initdata;
 extern struct kvm_x86_init_ops vt_init_ops __initdata;

@@ -133,4 +136,10 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu);
 #endif
 void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
+#ifdef CONFIG_INTEL_TDX_HOST
+int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
+#else
+static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
+#endif
+
 #endif /* __KVM_X86_VMX_X86_OPS_H */

Signature · 6 lines

2.25.1



-- 
Isaku Yamahata <[email protected]>

Huang, Kai

2023-03-15 09:48 UTC #11 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Quoted 7 lines

Quoted 4 lines
I think you should use hardware_enable_all(), and just do something similar to below in vmx_hardware_enable():
The use of hardware_enable_all() make us circle back to refactoring KVM hardware initialization topic. I'd like to stay away from it for now for TDX.

Sean's series to improve hardware enable has been merged to upstream already.

Can you elaborate what's the issue here?

[...]

arch/x86/kvm/vmx/main.c line 20

Quoted 12 lines

arch/x86/kvm/vmx/main.c +12 -0

+static bool enable_tdx __ro_after_init;
+module_param_named(tdx, enable_tdx, bool, 0444);
+
+static __init int vt_hardware_setup(void)
+{
+	int ret;
+
+	ret = vmx_hardware_setup();
+	if (ret)
+		return ret;
+
+	enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops);

Unfortunately, the enable_tdx should also be protected by the cpus_read_lock(), because CPU hotplug code path checks it too (as seen in your next patch).

Quoted 46 lines

arch/x86/kvm/vmx/main.c +5 -1

+
+	return 0;
+}
+
 #define VMX_REQUIRED_APICV_INHIBITS		       \
 (						       \
        BIT(APICV_INHIBIT_REASON_DISABLE)|	       \

@@ -159,7 +175,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {
-	.hardware_setup = vmx_hardware_setup,
+	.hardware_setup = vt_hardware_setup,
 	.handle_intel_pt_intr = NULL,
 
 	.runtime_ops = &vt_x86_ops,

arch/x86/kvm/vmx/tdx.c new file +24 -0

new file mode 100644
index 000000000000..8d265d7ae6fb
@@ -0,0 +1,74 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/cpu.h>
+
+#include <asm/tdx.h>
+
+#include "capabilities.h"
+#include "x86_ops.h"
+#include "x86.h"
+
+#undef pr_fmt
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+static int __init tdx_module_setup(void)
+{
+	int ret;
+
+	ret = tdx_enable();
+	if (ret) {
+		pr_info("Failed to initialize TDX module.\n");
+		return ret;
+	}
+
+	pr_info("TDX is supported.\n");
+	return 0;

arch/x86/kvm/vmx/tdx.c line 34

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c new file +10 -0

+}
+
+static int __init tdx_cpu_enable_cpu(void *unused)
+{
+	int r;
+
+	/*
+	 * TDX requires VMX. Because thread can be migrated, keep VMXON on
+	 * all online cpus until all TDX module initialization is done.
+	 */

The second sentence in this comment should be somewhere else, but not here.

Quoted 17 lines

arch/x86/kvm/vmx/tdx.c new file +17 -0

+	r = vmxon();
+	if (r)
+		return r;
+	return tdx_cpu_enable();
+}
+
+static void __init tdx_vmxoff_cpu(void *unused)
+{
+	WARN_ON_ONCE(cpu_vmxoff());
+}
+
+int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
+{
+	int cpu;
+	int r = 0;
+
+	if (!enable_ept) {

arch/x86/kvm/vmx/tdx.c line 61

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c new file +10 -0

+		pr_warn("Cannot enable TDX with EPT disabled\n");
+		return -EINVAL;
+	}
+
+	/* tdx_enable() in tdx_module_setup() requires cpus lock. */
+	cpus_read_lock();
+	/*
+	 * Because tdx_cpu_enable() acquires spin locks, on_each_cpu()
+	 * can't be used.
+	 */

Here you have cpus_read_lock() protection, so tdx_cpu_enable() cannot be called from CPU hotplug code path when you are doing here.

So, using on_each_cpu() to do tdx_cpu_enable() is OK here, because on one particular cpu, when it already has taken the spinlock, it cannot receive IPI anymore which can try to take the spinlock again.

arch/x86/kvm/vmx/tdx.c line 74

Quoted 13 lines

arch/x86/kvm/vmx/tdx.c new file +13 -0

+	for_each_online_cpu(cpu) {
+		if (smp_call_on_cpu(cpu, tdx_cpu_enable_cpu, NULL, false)) {
+			r = -EIO;
+			break;
+		}
+	}
+	if (!r)
+		r = tdx_module_setup();
+	on_each_cpu(tdx_vmxoff_cpu, NULL, 1);
+	cpus_read_unlock();
+
+	return r;
+}

I think you can merge next patch with this one because they are kinda related.

Putting them together allows people to review more easily.

Huang, Kai

2023-03-15 10:35 UTC #12 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Quoted 17 lines

arch/x86/kvm/vmx/main.c line 20
Quoted 13 lines
arch/x86/kvm/vmx/main.c +12 -0
+static bool enable_tdx __ro_after_init;
+module_param_named(tdx, enable_tdx, bool, 0444);
+
+static __init int vt_hardware_setup(void)
+{
+	int ret;
+
+	ret = vmx_hardware_setup();
+	if (ret)
+		return ret;
+
+	enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops);
Unfortunately, the enable_tdx should also be protected by the cpus_read_lock(),
because CPU hotplug code path checks it too (as seen in your next patch).

Sorry I was wrong. I forgot the CPU hotplug callbacks are registered in kvm_init(), which happens after hardware_setup(), so the CPU hotplug code cannot race with this hardware_setup() code here.

[...]

Quoted 5 lines

I think you can merge next patch with this one because they are kinda related.
Putting them together allows people to review more easily.

And perhaps this isn't necessary either.

Isaku Yamahata

2023-03-16 00:27 UTC #13 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

In reply to Huang, Kai

Quoted 9 lines · Kai

Quoted 7 lines
Quoted 4 lines
I think you should use hardware_enable_all(), and just do something similar to below in vmx_hardware_enable():
The use of hardware_enable_all() make us circle back to refactoring KVM hardware initialization topic. I'd like to stay away from it for now for TDX.
Sean's series to improve hardware enable has been merged to upstream already.

I revised the patch and I can use hardware_enable_all().

Quoted 16 lines

Review

Quoted 13 lines

(quoted diff) +13 -0

+	for_each_online_cpu(cpu) {
+		if (smp_call_on_cpu(cpu, tdx_cpu_enable_cpu, NULL, false)) {
+			r = -EIO;
+			break;
+		}
+	}
+	if (!r)
+		r = tdx_module_setup();
+	on_each_cpu(tdx_vmxoff_cpu, NULL, 1);
+	cpus_read_unlock();
+
+	return r;
+}

I think you can merge next patch with this one because they are kinda related.

Quoted 1 line

Putting them together allows people to review more easily.

Finally I come up with the version that uses kvm_hardware_enable_all(). that doesn't issue cpus_read_lock().
I had to patch tdx_cpu_enable(). I'll reply to the patch itself.

From 38774fc6e2bc5f0eddd0a0ab035ba8e712ee5ff2 Mon Sep 17 00:00:00 2001
Message-Id: <38774fc6e2bc5f0eddd0a0ab035ba8e712ee5ff2.1678926122.git.isaku.yamahata@intel.com>
In-Reply-To: <d2aa2142665b8204b628232ab615c98090371c99.1678926122.git.isaku.yamahata@intel.com>
References: <d2aa2142665b8204b628232ab615c98090371c99.1678926122.git.isaku.yamahata@intel.com>
From: Isaku Yamahata <[email protected]>
Date: Tue, 22 Feb 2022 14:44:15 -0800
Subject: [PATCH] KVM: TDX: Initialize the TDX module when loading the
KVM intel kernel module

TDX requires several initialization steps for KVM to create guest TDs. Detect CPU feature, enable VMX (TDX is based on VMX) on all online CPUs, detect the TDX module availability, initialize it and disable VMX.

To enable VMX on all online CPUs, utilize kvm_hardware_enable_all() that calls hardware_enable() methods. The method also initialize each CPU for TDX. TDX requires to call a TDX initialization function per logical processor (LP) before the LP uses TDX. When CPU is onlined, call the TDX LP initialization API when cpu is onlined. If it failed refuse onlininig of the cpu for simplicity instead of TDX avoiding the LP.

---
 arch/x86/kvm/Makefile      |  1 +
 arch/x86/kvm/vmx/main.c    | 34 ++++++++++++++++++++++++++--
 arch/x86/kvm/vmx/tdx.c     | 46 ++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h |  8 +++++++
 arch/x86/kvm/x86.c         | 10 +++++++++
 5 files changed, 97 insertions(+), 2 deletions(-)
 create mode 100644 arch/x86/kvm/vmx/tdx.c

arch/x86/kvm/Makefile +1 -0

index 0e894ae23cbc..4b01ab842ab7 100644
@@ -25,6 +25,7 @@ kvm-$(CONFIG_KVM_SMM)	+= smm.o
 kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \
 			   vmx/hyperv.o vmx/nested.o vmx/posted_intr.o vmx/main.o
 kvm-intel-$(CONFIG_X86_SGX_KVM)	+= vmx/sgx.o
+kvm-intel-$(CONFIG_INTEL_TDX_HOST)	+= vmx/tdx.o
 
 kvm-amd-y		+= svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \
 			   svm/sev.o svm/hyperv.o

arch/x86/kvm/vmx/main.c +32 -2

index 3f49e8e38b6b..f7bfa9888d9e 100644
@@ -6,6 +6,36 @@
 #include "nested.h"
 #include "pmu.h"
 
+static bool enable_tdx __ro_after_init;
+module_param_named(tdx, enable_tdx, bool, 0444);
+
+static int vt_hardware_enable(void)
+{
+	int ret;
+
+	ret = vmx_hardware_enable();
+	if (ret || !enable_tdx)
+		return ret;
+
+	ret = tdx_cpu_enable();
+	if (ret)
+		vmx_hardware_disable();
+	return ret;
+}
+
+static __init int vt_hardware_setup(void)
+{
+	int ret;
+
+	ret = vmx_hardware_setup();
+	if (ret)
+		return ret;
+
+	enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops);
+
+	return 0;
+}
+
 #define VMX_REQUIRED_APICV_INHIBITS		       \
 (						       \
        BIT(APICV_INHIBIT_REASON_DISABLE)|	       \

@@ -24,7 +54,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 
 	.hardware_unsetup = vmx_hardware_unsetup,
 
-	.hardware_enable = vmx_hardware_enable,
+	.hardware_enable = vt_hardware_enable,
 	.hardware_disable = vmx_hardware_disable,
 	.has_emulated_msr = vmx_has_emulated_msr,

@@ -159,7 +189,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {
-	.hardware_setup = vmx_hardware_setup,
+	.hardware_setup = vt_hardware_setup,
 	.handle_intel_pt_intr = NULL,
 
 	.runtime_ops = &vt_x86_ops,

arch/x86/kvm/vmx/tdx.c new file +46 -0

46 lines not on this page · raw message

arch/x86/kvm/vmx/x86_ops.h +8 -0

index 051b5c4b5c2f..f59e5197836a 100644
@@ -20,6 +20,8 @@ bool kvm_is_vmx_supported(void);
 int __init vmx_init(void);
 void vmx_exit(void);
 
+__init int vmx_hardware_setup(void);
+
 extern struct kvm_x86_ops vt_x86_ops __initdata;
 extern struct kvm_x86_init_ops vt_init_ops __initdata;

@@ -133,4 +135,10 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu);
 #endif
 void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
+#ifdef CONFIG_INTEL_TDX_HOST
+int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
+#else
+static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
+#endif
+
 #endif /* __KVM_X86_VMX_X86_OPS_H */

arch/x86/kvm/x86.c +10 -0

index 2125fcaa3973..b264012a8478 100644
@@ -9435,6 +9435,16 @@ static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 
 	kvm_init_pmu_capability(ops->pmu_ops);
 
+	/*
+	 * TDX requires those methods to enable VMXON by
+	 * kvm_hardware_enable/disable_all()
+	 */
+	static_call_update(kvm_x86_check_processor_compatibility,
+			   ops->runtime_ops->check_processor_compatibility);
+	static_call_update(kvm_x86_hardware_enable,
+			   ops->runtime_ops->hardware_enable);
+	static_call_update(kvm_x86_hardware_disable,
+			   ops->runtime_ops->hardware_disable);
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;

Signature · 6 lines

2.25.1



-- 
Isaku Yamahata <[email protected]>

Huang, Kai

2023-03-24 10:42 UTC #14 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Sorry for late reply.

Quoted 71 lines

arch/x86/kvm/vmx/main.c +32 -2

+static bool enable_tdx __ro_after_init;
+module_param_named(tdx, enable_tdx, bool, 0444);
+
+static int vt_hardware_enable(void)
+{
+	int ret;
+
+	ret = vmx_hardware_enable();
+	if (ret || !enable_tdx)
+		return ret;
+
+	ret = tdx_cpu_enable();
+	if (ret)
+		vmx_hardware_disable();
+	return ret;
+}
+
+static __init int vt_hardware_setup(void)
+{
+	int ret;
+
+	ret = vmx_hardware_setup();
+	if (ret)
+		return ret;
+
+	enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops);
+
+	return 0;
+}
+
 #define VMX_REQUIRED_APICV_INHIBITS		       \
 (						       \
        BIT(APICV_INHIBIT_REASON_DISABLE)|	       \

@@ -24,7 +54,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 
 	.hardware_unsetup = vmx_hardware_unsetup,
 
-	.hardware_enable = vmx_hardware_enable,
+	.hardware_enable = vt_hardware_enable,
 	.hardware_disable = vmx_hardware_disable,
 	.has_emulated_msr = vmx_has_emulated_msr,

@@ -159,7 +189,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {
-	.hardware_setup = vmx_hardware_setup,
+	.hardware_setup = vt_hardware_setup,
 	.handle_intel_pt_intr = NULL,
 
 	.runtime_ops = &vt_x86_ops,

arch/x86/kvm/vmx/tdx.c new file +13 -0

new file mode 100644
index 000000000000..f13fdf71430b
@@ -0,0 +1,46 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/cpu.h>
+
+#include <asm/tdx.h>
+
+#include "capabilities.h"
+#include "x86_ops.h"
+#include "x86.h"
+
+#undef pr_fmt
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+static int __init tdx_module_setup(void)

arch/x86/kvm/vmx/tdx.c line 23

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c new file +10 -0

+{
+	int ret;
+
+	ret = tdx_enable();
+	if (ret) {
+		pr_info("Failed to initialize TDX module.\n");
+		return ret;
+	}
+
+	pr_info("TDX is supported.\n");

Both pr_info()s are not required, because tdx_enable() internally prints them.

Quoted 24 lines

arch/x86/kvm/vmx/tdx.c new file +23 -0

+	return 0;
+}
+
+int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
+{
+	int r = 0;
+
+	if (!enable_ept) {
+		pr_warn("Cannot enable TDX with EPT disabled\n");
+		return -EINVAL;
+	}
+
+	/* tdx_enable() in tdx_module_setup() requires cpus lock. */
+	cpus_read_lock();
+	r = kvm_hardware_enable_all();
+	if (!r) {
+		r = tdx_module_setup();
+		kvm_hardware_disable_all();
+	}
+	cpus_read_unlock();
+
+	return r;
+}

[...]

Quoted 12 lines

#endif /* __KVM_X86_VMX_X86_OPS_H */

arch/x86/kvm/x86.c +3 -0

index 2125fcaa3973..b264012a8478 100644
@@ -9435,6 +9435,16 @@ static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 
 	kvm_init_pmu_capability(ops->pmu_ops);
 
+	/*
+	 * TDX requires those methods to enable VMXON by
+	 * kvm_hardware_enable/disable_all()

arch/x86/kvm/x86.c static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) · line 9450

Quoted 10 lines

arch/x86/kvm/x86.c +7 -0

+	 */
+	static_call_update(kvm_x86_check_processor_compatibility,
+			   ops->runtime_ops->check_processor_compatibility);
+	static_call_update(kvm_x86_hardware_enable,
+			   ops->runtime_ops->hardware_enable);
+	static_call_update(kvm_x86_hardware_disable,
+			   ops->runtime_ops->hardware_disable);
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;

Hmm.. I think this is ugly. Perhaps we should never do any static_call(kvm_x86_xxx)() in hardware_setup(), because hardware_setup() is called before kvm_ops_update() and may update vendor's kvm_x86_ops.

So probably use hardware_enable_all() in hardware_setup() is a bad idea.

I think we have below options on how to handle:

1) Use VMX's kvm_x86_ops directly in tdx_hardware_setup(). For instance, something like below:

int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)

{
	...

	cpus_read_lock();
	r = on_each_cpu(vt_x86_ops.hardware_enable, ...);
	if (!r)
		r = tdx_module_setup();
	on_each_cpu(vt_x86_ops.hardware_disable, ...);
	cpus_read_unlock();

	...
}

But this doesn't clean up nicely when there's some particular cpus fail to do hardware_enable(). To clean up nicely, we do need additional things similar to the hardware_enable_all() code path: a per-cpu variable or a cpumask_t + a wrapper of vt_x86_ops->hardware_enable() to track which cpus have done hardware_enable() successfully.

2) Move those static_call_update() into tdx_hardware_setup() so they are TDX code self-contained. But this would require exposing kvm_x86_ops as symbol, which isn't nice either.

3) Introduce another kvm_x86_init_ops->hardware_post_setup(), which is called after kvm_ops_update().

Personally, I think 3) perhaps is the most elegant one, but not sure whether Sean/Paolo has any opinion.

Isaku Yamahata

2023-03-28 23:59 UTC #15 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Quoted 5 lines · Kai

Quoted 5 lines

(quoted diff) +5 -0

+static int __init tdx_module_setup(void)
+{
+	int ret;
+
+	ret = tdx_enable();

Quoted 8 lines

Review
Quoted 6 lines
(quoted diff) +6 -0
+	if (ret) {
+		pr_info("Failed to initialize TDX module.\n");
+		return ret;
+	}
+
+	pr_info("TDX is supported.\n");
Both pr_info()s are not required, because tdx_enable() internally prints them.

Ok, will drop this line.

Quoted 58 lines

Quoted 12 lines
#endif /* __KVM_X86_VMX_X86_OPS_H */
arch/x86/kvm/x86.c +3 -0
index 2125fcaa3973..b264012a8478 100644
@@ -9435,6 +9435,16 @@ static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 
 	kvm_init_pmu_capability(ops->pmu_ops);
 
+	/*
+	 * TDX requires those methods to enable VMXON by
+	 * kvm_hardware_enable/disable_all()
arch/x86/kvm/x86.c static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops) · line 9450
Quoted 10 lines
arch/x86/kvm/x86.c +7 -0
+	 */
+	static_call_update(kvm_x86_check_processor_compatibility,
+			   ops->runtime_ops->check_processor_compatibility);
+	static_call_update(kvm_x86_hardware_enable,
+			   ops->runtime_ops->hardware_enable);
+	static_call_update(kvm_x86_hardware_disable,
+			   ops->runtime_ops->hardware_disable);
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Hmm.. I think this is ugly. Perhaps we should never do any static_call(kvm_x86_xxx)() in hardware_setup(), because hardware_setup() is called before kvm_ops_update() and may update vendor's kvm_x86_ops.
So probably use hardware_enable_all() in hardware_setup() is a bad idea.
I think we have below options on how to handle:
1) Use VMX's kvm_x86_ops directly in tdx_hardware_setup(). For instance, something like below:
int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
{
	...

	cpus_read_lock();
	r = on_each_cpu(vt_x86_ops.hardware_enable, ...);
	if (!r)
		r = tdx_module_setup();
	on_each_cpu(vt_x86_ops.hardware_disable, ...);
	cpus_read_unlock();

	...
}
But this doesn't clean up nicely when there's some particular cpus fail to do hardware_enable(). To clean up nicely, we do need additional things similar to the hardware_enable_all() code path: a per-cpu variable or a cpumask_t + a wrapper of vt_x86_ops->hardware_enable() to track which cpus have done hardware_enable() successfully.
2) Move those static_call_update() into tdx_hardware_setup() so they are TDX code self-contained. But this would require exposing kvm_x86_ops as symbol, which isn't nice either.

Quoted 5 lines

3) Introduce another kvm_x86_init_ops->hardware_post_setup(), which is called after kvm_ops_update().
Personally, I think 3) perhaps is the most elegant one, but not sure whether Sean/Paolo has any opinion.

I think we can simply update the ops before calling hardware_enable() and clean up ops on failure.

arch/x86/kvm/x86.c +6 ~~-11~~

index 709134e7c12e..42c9b58fd1ef 100644
@@ -9436,20 +9436,15 @@ static int __kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 	kvm_init_pmu_capability(ops->pmu_ops);
 
 	/*
-	 * TDX requires those methods to enable VMXON by
-	 * kvm_hardware_enable/disable_all_nolock()
+	 * Because TDX hardware_setup uses x86_ops, update ops before calling
+	 * ops->hardware_setup().
 	 */
-	static_call_update(kvm_x86_check_processor_compatibility,
-			   ops->runtime_ops->check_processor_compatibility);
-	static_call_update(kvm_x86_hardware_enable,
-			   ops->runtime_ops->hardware_enable);
-	static_call_update(kvm_x86_hardware_disable,
-			   ops->runtime_ops->hardware_disable);
+	kvm_ops_update(ops);
 	r = ops->hardware_setup();
-	if (r != 0)
+	if (r != 0) {
+		kvm_x86_ops.hardware_enable = NULL;
 		goto out_mmu_exit;
-
-	kvm_ops_update(ops);
+	}
 
 	for_each_online_cpu(cpu) {
 		smp_call_function_single(cpu, kvm_x86_check_cpu_compat, &r, 1);

Signature · 1 line

Isaku Yamahata <[email protected]>

Huang, Kai

2023-03-29 01:13 UTC #16 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Quoted 51 lines

Quoted 51 lines
Quoted 5 lines
(quoted diff) +3 -0
+	/*
+	 * TDX requires those methods to enable VMXON by
+	 * kvm_hardware_enable/disable_all()
Review
Quoted 10 lines
(quoted diff) +7 -0
+	 */
+	static_call_update(kvm_x86_check_processor_compatibility,
+			   ops->runtime_ops->check_processor_compatibility);
+	static_call_update(kvm_x86_hardware_enable,
+			   ops->runtime_ops->hardware_enable);
+	static_call_update(kvm_x86_hardware_disable,
+			   ops->runtime_ops->hardware_disable);
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Hmm.. I think this is ugly. Perhaps we should never do any static_call(kvm_x86_xxx)() in hardware_setup(), because hardware_setup() is called before kvm_ops_update() and may update vendor's kvm_x86_ops.
So probably use hardware_enable_all() in hardware_setup() is a bad idea.
I think we have below options on how to handle:
1) Use VMX's kvm_x86_ops directly in tdx_hardware_setup(). For instance, something like below:
int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
{
	...

	cpus_read_lock();
	r = on_each_cpu(vt_x86_ops.hardware_enable, ...);
	if (!r)
		r = tdx_module_setup();
	on_each_cpu(vt_x86_ops.hardware_disable, ...);
	cpus_read_unlock();

	...
}
But this doesn't clean up nicely when there's some particular cpus fail to do hardware_enable(). To clean up nicely, we do need additional things similar to the hardware_enable_all() code path: a per-cpu variable or a cpumask_t + a wrapper of vt_x86_ops->hardware_enable() to track which cpus have done hardware_enable() successfully.
2) Move those static_call_update() into tdx_hardware_setup() so they are TDX code self-contained. But this would require exposing kvm_x86_ops as symbol, which isn't nice either.

Quoted 10 lines

Quoted 5 lines
3) Introduce another kvm_x86_init_ops->hardware_post_setup(), which is called after kvm_ops_update().
Personally, I think 3) perhaps is the most elegant one, but not sure whether Sean/Paolo has any opinion.
I think we can simply update the ops before calling hardware_enable() and clean up ops on failure.

This doesn't work because hardware_setup() may update vendor's kvm_x86_ops.

If you do kvm_ops_update() before hardware_setup(), you need to manually update those updated (in hardware_setup()) callbacks again after.

Isaku Yamahata

2023-03-29 21:56 UTC #17 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Quoted 62 lines · Kai

Quoted 51 lines
Quoted 51 lines
Quoted 5 lines
(quoted diff) +3 -0
+	/*
+	 * TDX requires those methods to enable VMXON by
+	 * kvm_hardware_enable/disable_all()
Review
Quoted 10 lines
(quoted diff) +7 -0
+	 */
+	static_call_update(kvm_x86_check_processor_compatibility,
+			   ops->runtime_ops->check_processor_compatibility);
+	static_call_update(kvm_x86_hardware_enable,
+			   ops->runtime_ops->hardware_enable);
+	static_call_update(kvm_x86_hardware_disable,
+			   ops->runtime_ops->hardware_disable);
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Hmm.. I think this is ugly. Perhaps we should never do any static_call(kvm_x86_xxx)() in hardware_setup(), because hardware_setup() is called before kvm_ops_update() and may update vendor's kvm_x86_ops.
So probably use hardware_enable_all() in hardware_setup() is a bad idea.
I think we have below options on how to handle:
1) Use VMX's kvm_x86_ops directly in tdx_hardware_setup(). For instance, something like below:
int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
{
	...

	cpus_read_lock();
	r = on_each_cpu(vt_x86_ops.hardware_enable, ...);
	if (!r)
		r = tdx_module_setup();
	on_each_cpu(vt_x86_ops.hardware_disable, ...);
	cpus_read_unlock();

	...
}
But this doesn't clean up nicely when there's some particular cpus fail to do hardware_enable(). To clean up nicely, we do need additional things similar to the hardware_enable_all() code path: a per-cpu variable or a cpumask_t + a wrapper of vt_x86_ops->hardware_enable() to track which cpus have done hardware_enable() successfully.
2) Move those static_call_update() into tdx_hardware_setup() so they are TDX code self-contained. But this would require exposing kvm_x86_ops as symbol, which isn't nice either.
Quoted 10 lines
Quoted 5 lines
3) Introduce another kvm_x86_init_ops->hardware_post_setup(), which is called after kvm_ops_update().
Personally, I think 3) perhaps is the most elegant one, but not sure whether Sean/Paolo has any opinion.
I think we can simply update the ops before calling hardware_enable() and clean up ops on failure.

Quoted 4 lines

This doesn't work because hardware_setup() may update vendor's kvm_x86_ops.
If you do kvm_ops_update() before hardware_setup(), you need to manually update those updated (in hardware_setup()) callbacks again after.

We can call kvm_ops_update() twice before and after hardware_setup().

Signature · 1 line

Isaku Yamahata <[email protected]>

Huang, Kai

2023-03-29 23:17 UTC #18 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Quoted 65 lines · Isaku Yamahata

Quoted 62 lines · Kai
Quoted 61 lines
Quoted 51 lines
Quoted 5 lines
(quoted diff) +3 -0
+	/*
+	 * TDX requires those methods to enable VMXON by
+	 * kvm_hardware_enable/disable_all()
Review
Quoted 10 lines
(quoted diff) +7 -0
+	 */
+	static_call_update(kvm_x86_check_processor_compatibility,
+			   ops->runtime_ops->check_processor_compatibility);
+	static_call_update(kvm_x86_hardware_enable,
+			   ops->runtime_ops->hardware_enable);
+	static_call_update(kvm_x86_hardware_disable,
+			   ops->runtime_ops->hardware_disable);
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Hmm.. I think this is ugly. Perhaps we should never do any static_call(kvm_x86_xxx)() in hardware_setup(), because hardware_setup() is called before kvm_ops_update() and may update vendor's kvm_x86_ops.
So probably use hardware_enable_all() in hardware_setup() is a bad idea.
I think we have below options on how to handle:
1) Use VMX's kvm_x86_ops directly in tdx_hardware_setup(). For instance, something like below:
int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
{
	...

	cpus_read_lock();
	r = on_each_cpu(vt_x86_ops.hardware_enable, ...);
	if (!r)
		r = tdx_module_setup();
	on_each_cpu(vt_x86_ops.hardware_disable, ...);
	cpus_read_unlock();

	...
}
But this doesn't clean up nicely when there's some particular cpus fail to do hardware_enable(). To clean up nicely, we do need additional things similar to the hardware_enable_all() code path: a per-cpu variable or a cpumask_t + a wrapper of vt_x86_ops->hardware_enable() to track which cpus have done hardware_enable() successfully.
2) Move those static_call_update() into tdx_hardware_setup() so they are TDX code self-contained. But this would require exposing kvm_x86_ops as symbol, which isn't nice either.
Quoted 5 lines
3) Introduce another kvm_x86_init_ops->hardware_post_setup(), which is called after kvm_ops_update().
Personally, I think 3) perhaps is the most elegant one, but not sure whether Sean/Paolo has any opinion.
I think we can simply update the ops before calling hardware_enable() and clean up ops on failure.

Quoted 7 lines

Quoted 4 lines
This doesn't work because hardware_setup() may update vendor's kvm_x86_ops.
If you do kvm_ops_update() before hardware_setup(), you need to manually update those updated (in hardware_setup()) callbacks again after.
We can call kvm_ops_update() twice before and after hardware_setup().

Personally I think it's too ugly.

Isaku Yamahata

2023-03-30 01:04 UTC #19 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Quoted 68 lines · Kai

Quoted 67 lines · Isaku Yamahata
Quoted 64 lines · Kai
Quoted 51 lines
Quoted 51 lines
Quoted 5 lines
(quoted diff) +3 -0
+	/*
+	 * TDX requires those methods to enable VMXON by
+	 * kvm_hardware_enable/disable_all()
Review
Quoted 10 lines
(quoted diff) +7 -0
+	 */
+	static_call_update(kvm_x86_check_processor_compatibility,
+			   ops->runtime_ops->check_processor_compatibility);
+	static_call_update(kvm_x86_hardware_enable,
+			   ops->runtime_ops->hardware_enable);
+	static_call_update(kvm_x86_hardware_disable,
+			   ops->runtime_ops->hardware_disable);
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Hmm.. I think this is ugly. Perhaps we should never do any static_call(kvm_x86_xxx)() in hardware_setup(), because hardware_setup() is called before kvm_ops_update() and may update vendor's kvm_x86_ops.
So probably use hardware_enable_all() in hardware_setup() is a bad idea.
I think we have below options on how to handle:
1) Use VMX's kvm_x86_ops directly in tdx_hardware_setup(). For instance, something like below:
int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
{
	...

	cpus_read_lock();
	r = on_each_cpu(vt_x86_ops.hardware_enable, ...);
	if (!r)
		r = tdx_module_setup();
	on_each_cpu(vt_x86_ops.hardware_disable, ...);
	cpus_read_unlock();

	...
}
But this doesn't clean up nicely when there's some particular cpus fail to do hardware_enable(). To clean up nicely, we do need additional things similar to the hardware_enable_all() code path: a per-cpu variable or a cpumask_t + a wrapper of vt_x86_ops->hardware_enable() to track which cpus have done hardware_enable() successfully.
2) Move those static_call_update() into tdx_hardware_setup() so they are TDX code self-contained. But this would require exposing kvm_x86_ops as symbol, which isn't nice either.
Quoted 10 lines
Quoted 5 lines
3) Introduce another kvm_x86_init_ops->hardware_post_setup(), which is called after kvm_ops_update().
Personally, I think 3) perhaps is the most elegant one, but not sure whether Sean/Paolo has any opinion.
I think we can simply update the ops before calling hardware_enable() and clean up ops on failure.
This doesn't work because hardware_setup() may update vendor's kvm_x86_ops.

Quoted 7 lines

Quoted 5 lines
Quoted 2 lines
If you do kvm_ops_update() before hardware_setup(), you need to manually update those updated (in hardware_setup()) callbacks again after.
We can call kvm_ops_update() twice before and after hardware_setup().
Personally I think it's too ugly.

So you prefer the option 3 to calling kvm_ops_update() twice. Okay, I'll update the patch.

Signature · 1 line

Isaku Yamahata <[email protected]>

Isaku Yamahata

2023-04-05 20:07 UTC #20 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Quoted 74 lines · Isaku Yamahata

Quoted 71 lines · Kai
Quoted 70 lines · Isaku Yamahata
Quoted 66 lines · Kai
Quoted 51 lines
Quoted 51 lines
Quoted 5 lines
(quoted diff) +3 -0
+	/*
+	 * TDX requires those methods to enable VMXON by
+	 * kvm_hardware_enable/disable_all()
Review
Quoted 10 lines
(quoted diff) +7 -0
+	 */
+	static_call_update(kvm_x86_check_processor_compatibility,
+			   ops->runtime_ops->check_processor_compatibility);
+	static_call_update(kvm_x86_hardware_enable,
+			   ops->runtime_ops->hardware_enable);
+	static_call_update(kvm_x86_hardware_disable,
+			   ops->runtime_ops->hardware_disable);
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Hmm.. I think this is ugly. Perhaps we should never do any static_call(kvm_x86_xxx)() in hardware_setup(), because hardware_setup() is called before kvm_ops_update() and may update vendor's kvm_x86_ops.
So probably use hardware_enable_all() in hardware_setup() is a bad idea.
I think we have below options on how to handle:
1) Use VMX's kvm_x86_ops directly in tdx_hardware_setup(). For instance, something like below:
int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
{
	...

	cpus_read_lock();
	r = on_each_cpu(vt_x86_ops.hardware_enable, ...);
	if (!r)
		r = tdx_module_setup();
	on_each_cpu(vt_x86_ops.hardware_disable, ...);
	cpus_read_unlock();

	...
}
But this doesn't clean up nicely when there's some particular cpus fail to do hardware_enable(). To clean up nicely, we do need additional things similar to the hardware_enable_all() code path: a per-cpu variable or a cpumask_t + a wrapper of vt_x86_ops->hardware_enable() to track which cpus have done hardware_enable() successfully.
2) Move those static_call_update() into tdx_hardware_setup() so they are TDX code self-contained. But this would require exposing kvm_x86_ops as symbol, which isn't nice either.
Quoted 10 lines
Quoted 5 lines
3) Introduce another kvm_x86_init_ops->hardware_post_setup(), which is called after kvm_ops_update().
Personally, I think 3) perhaps is the most elegant one, but not sure whether Sean/Paolo has any opinion.
I think we can simply update the ops before calling hardware_enable() and clean up ops on failure.
This doesn't work because hardware_setup() may update vendor's kvm_x86_ops.
If you do kvm_ops_update() before hardware_setup(), you need to manually update those updated (in hardware_setup()) callbacks again after.

Quoted 7 lines

Quoted 4 lines
Quoted 2 lines
We can call kvm_ops_update() twice before and after hardware_setup().
Personally I think it's too ugly.
So you prefer the option 3 to calling kvm_ops_update() twice. Okay, I'll update the patch.

After playing with hardware_post_setup(), it's inevitable to call kvm_ops_update() twice.
When VMX initialization succeeded with hardware_setup(), but TDX initialization with hardware_post_setup() failed, we'd like to support only VMX with warning message. In such case, we need to revert x86_ops to VMX only.
It doesn't make sense to introduce hardware_post_setup() to avoid calling kvm_update_ops twice.

Signature · 1 line

Isaku Yamahata <[email protected]>

Huang, Kai

2023-04-05 23:10 UTC #21 lore raw

Re: [PATCH v13 003/113] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module

Quoted 85 lines · Isaku Yamahata

Quoted 74 lines · Isaku Yamahata
Quoted 71 lines · Kai
Quoted 70 lines · Isaku Yamahata
Quoted 66 lines · Kai
Quoted 51 lines
Quoted 51 lines
Quoted 5 lines
(quoted diff) +3 -0
+	/*
+	 * TDX requires those methods to enable VMXON by
+	 * kvm_hardware_enable/disable_all()
Review
Quoted 10 lines
(quoted diff) +7 -0
+	 */
+	static_call_update(kvm_x86_check_processor_compatibility,
+			   ops->runtime_ops->check_processor_compatibility);
+	static_call_update(kvm_x86_hardware_enable,
+			   ops->runtime_ops->hardware_enable);
+	static_call_update(kvm_x86_hardware_disable,
+			   ops->runtime_ops->hardware_disable);
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Hmm.. I think this is ugly. Perhaps we should never do any static_call(kvm_x86_xxx)() in hardware_setup(), because hardware_setup() is called before kvm_ops_update() and may update vendor's kvm_x86_ops.
So probably use hardware_enable_all() in hardware_setup() is a bad idea.
I think we have below options on how to handle:
1) Use VMX's kvm_x86_ops directly in tdx_hardware_setup(). For instance, something like below:
int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
{
	...

	cpus_read_lock();
	r = on_each_cpu(vt_x86_ops.hardware_enable, ...);
	if (!r)
		r = tdx_module_setup();
	on_each_cpu(vt_x86_ops.hardware_disable, ...);
	cpus_read_unlock();

	...
}
But this doesn't clean up nicely when there's some particular cpus fail to do hardware_enable(). To clean up nicely, we do need additional things similar to the hardware_enable_all() code path: a per-cpu variable or a cpumask_t + a wrapper of vt_x86_ops->hardware_enable() to track which cpus have done hardware_enable() successfully.
2) Move those static_call_update() into tdx_hardware_setup() so they are TDX code self-contained. But this would require exposing kvm_x86_ops as symbol, which isn't nice either.
Quoted 10 lines
Quoted 5 lines
3) Introduce another kvm_x86_init_ops->hardware_post_setup(), which is called after kvm_ops_update().
Personally, I think 3) perhaps is the most elegant one, but not sure whether Sean/Paolo has any opinion.
I think we can simply update the ops before calling hardware_enable() and clean up ops on failure.
This doesn't work because hardware_setup() may update vendor's kvm_x86_ops.
If you do kvm_ops_update() before hardware_setup(), you need to manually update those updated (in hardware_setup()) callbacks again after.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
We can call kvm_ops_update() twice before and after hardware_setup().
Personally I think it's too ugly.
So you prefer the option 3 to calling kvm_ops_update() twice. Okay, I'll update the patch.

Quoted 8 lines

After playing with hardware_post_setup(), it's inevitable to call kvm_ops_update() twice.
When VMX initialization succeeded with hardware_setup(), but TDX initialization with hardware_post_setup() failed, we'd like to support only VMX with warning message. In such case, we need to revert x86_ops to VMX only.
It doesn't make sense to introduce hardware_post_setup() to avoid calling kvm_update_ops twice.

OK. Then how about option 1) ?

We just need another wrapper around vt_x86_ops.hardware_{enable|disable}() and use a VT's own per-cpu variable to track which has cpu has done the vmx_hardware_enable().

We can even put the per-cpu variable inside the vt_hardware_enable() itself w/o introducing the wrapper.

But again it's better if Sean can input here.

[email protected]

author

2023-03-12 17:57 UTC #22 lore raw

PATCH v13 4/113 KVM: TDX: Initialize logical processor when onlined

From: Isaku Yamahata <[email protected]>

TDX requires to call a TDX initialization function per logical processor (LP) before the LP uses TDX. When CPU is onlined, call the TDX LP initialization API when cpu is onlined. If it failed refuse onlininig of the cpu for simplicity instead of TDX avoiding the LP.

---
 arch/x86/kvm/vmx/main.c    | 15 ++++++++++++++-
 arch/x86/kvm/vmx/tdx.c     |  5 +++++
 arch/x86/kvm/vmx/x86_ops.h |  2 ++
 3 files changed, 21 insertions(+), 1 deletion(-)

arch/x86/kvm/vmx/main.c +14 -1

index 5c9f5e00b3c4..d1c9c7f55050 100644
@@ -9,6 +9,19 @@
 static bool enable_tdx __ro_after_init;
 module_param_named(tdx, enable_tdx, bool, 0444);
 
+static int vt_hardware_enable(void)
+{
+	int ret;
+
+	ret = vmx_hardware_enable();
+	if (!ret && enable_tdx) {
+		ret = tdx_hardware_enable();
+		if (ret)
+			vmx_hardware_disable();
+	}
+	return ret;
+}
+
 static __init int vt_hardware_setup(void)
 {
 	int ret;

@@ -40,7 +53,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 
 	.hardware_unsetup = vmx_hardware_unsetup,
 
-	.hardware_enable = vmx_hardware_enable,
+	.hardware_enable = vt_hardware_enable,
 	.hardware_disable = vmx_hardware_disable,
 	.has_emulated_msr = vmx_has_emulated_msr,

arch/x86/kvm/vmx/tdx.c +5 -0

index f3eb0138b60b..e51314bbb439 100644
@@ -10,6 +10,11 @@
 #undef pr_fmt
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+int tdx_hardware_enable(void)
+{
+	return tdx_cpu_enable();
+}
+
 static int __init tdx_module_setup(void)
 {
 	int ret;

arch/x86/kvm/vmx/x86_ops.h +2 -0

index 0f200aead411..010d02c86ba4 100644
@@ -139,8 +139,10 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
 #ifdef CONFIG_INTEL_TDX_HOST
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
+int tdx_hardware_enable(void);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
+static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

Signature · 1 line

2.25.1

Huang, Kai

2023-03-15 09:59 UTC #23 lore raw

Re: [PATCH v13 004/113] KVM: TDX: Initialize logical processor when onlined

Quoted 6 lines · On Sun, 2023-03-12 at 10:55 -0700, [email protected] wrote:

From: Isaku Yamahata <[email protected]>
TDX requires to call a TDX initialization function per logical processor (LP) before the LP uses TDX. When CPU is onlined, call the TDX LP initialization API when cpu is onlined.

Double "when CPU is onlined".

Quoted 2 lines

If it failed refuse onlininig of
the cpu for simplicity instead of TDX avoiding the LP.

Sorry cannot understand this sentence, particularly, the "instead ..." part.

Quoted 72 lines

---
 arch/x86/kvm/vmx/main.c    | 15 ++++++++++++++-
 arch/x86/kvm/vmx/tdx.c     |  5 +++++
 arch/x86/kvm/vmx/x86_ops.h |  2 ++
 3 files changed, 21 insertions(+), 1 deletion(-)

arch/x86/kvm/vmx/main.c +14 -1

index 5c9f5e00b3c4..d1c9c7f55050 100644
@@ -9,6 +9,19 @@
 static bool enable_tdx __ro_after_init;
 module_param_named(tdx, enable_tdx, bool, 0444);
 
+static int vt_hardware_enable(void)
+{
+	int ret;
+
+	ret = vmx_hardware_enable();
+	if (!ret && enable_tdx) {
+		ret = tdx_hardware_enable();
+		if (ret)
+			vmx_hardware_disable();
+	}
+	return ret;
+}
+
 static __init int vt_hardware_setup(void)
 {
 	int ret;

@@ -40,7 +53,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 
 	.hardware_unsetup = vmx_hardware_unsetup,
 
-	.hardware_enable = vmx_hardware_enable,
+	.hardware_enable = vt_hardware_enable,
 	.hardware_disable = vmx_hardware_disable,
 	.has_emulated_msr = vmx_has_emulated_msr,

arch/x86/kvm/vmx/tdx.c +5 -0

index f3eb0138b60b..e51314bbb439 100644
@@ -10,6 +10,11 @@
 #undef pr_fmt
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+int tdx_hardware_enable(void)
+{
+	return tdx_cpu_enable();
+}
+
 static int __init tdx_module_setup(void)
 {
 	int ret;

arch/x86/kvm/vmx/x86_ops.h +2 -0

index 0f200aead411..010d02c86ba4 100644
@@ -139,8 +139,10 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
 #ifdef CONFIG_INTEL_TDX_HOST
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
+int tdx_hardware_enable(void);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
+static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

[email protected]

author

2023-03-12 17:57 UTC #24 lore raw

PATCH v13 5/113 KVM: TDX: Add placeholders for TDX VM/vcpu structure

From: Isaku Yamahata <[email protected]>

Add placeholders TDX VM/vcpu structure that overlays with VMX VM/vcpu structures. Initialize VM structure size and vcpu size/align so that x86 KVM common code knows those size irrespective of VMX or TDX. Those structures will be populated as guest creation logic develops.

Add helper functions to check if the VM is guest TD and add conversion functions between KVM VM/VCPU and TDX VM/VCPU.

---
 arch/x86/kvm/vmx/main.c |  9 ++++++++
 arch/x86/kvm/vmx/tdx.c  |  1 +
 arch/x86/kvm/vmx/tdx.h  | 50 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 60 insertions(+)
 create mode 100644 arch/x86/kvm/vmx/tdx.h

arch/x86/kvm/vmx/main.c +9 -0

index d1c9c7f55050..5bc31c41bea9 100644
@@ -5,6 +5,7 @@
 #include "vmx.h"
 #include "nested.h"
 #include "pmu.h"
+#include "tdx.h"
 
 static bool enable_tdx __ro_after_init;
 module_param_named(tdx, enable_tdx, bool, 0444);

@@ -224,6 +225,14 @@ static int __init vt_init(void)
 	vt_x86_ops.vm_size = sizeof(struct kvm_vmx);
 	vcpu_size = sizeof(struct vcpu_vmx);
 	vcpu_align = __alignof__(struct vcpu_vmx);
+	if (enable_tdx) {
+		vt_x86_ops.vm_size = max_t(unsigned int, vt_x86_ops.vm_size,
+					   sizeof(struct kvm_tdx));
+		vcpu_size = max_t(unsigned int, vcpu_size,
+				  sizeof(struct vcpu_tdx));
+		vcpu_align = max_t(unsigned int, vcpu_align,
+				   __alignof__(struct vcpu_tdx));
+	}
 	r = kvm_init(vcpu_size, vcpu_align, THIS_MODULE);
 	if (r)
 		goto err_kvm_init;

arch/x86/kvm/vmx/tdx.c +1 -0

index e51314bbb439..2f9eaefee249 100644
@@ -6,6 +6,7 @@
 #include "capabilities.h"
 #include "x86_ops.h"
 #include "x86.h"
+#include "tdx.h"
 
 #undef pr_fmt
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

arch/x86/kvm/vmx/tdx.h new file +50 -0

50 lines not on this page · raw message

Signature · 1 line

2.25.1

Huang, Kai

2023-03-15 10:42 UTC #25 lore raw

Re: [PATCH v13 005/113] KVM: TDX: Add placeholders for TDX VM/vcpu structure

arch/x86/kvm/vmx/tdx.h line 20

Quoted 6 lines · On Sun, 2023-03-12 at 10:55 -0700, [email protected] wrote:

arch/x86/kvm/vmx/tdx.h new file +6 -0

+
+static inline bool is_td(struct kvm *kvm)
+{
+	return kvm->arch.vm_type == KVM_X86_PROTECTED_VM;
+}
+

KVM_X86_PROTECTED_VM is defined in the next patch, which means this patch cannot compile.

Also, why KVM_X86_PROTECTED_VM == TDX VM?

Isaku Yamahata

2023-03-16 15:42 UTC #26 lore raw

Re: [PATCH v13 005/113] KVM: TDX: Add placeholders for TDX VM/vcpu structure

Quoted 11 lines · Kai

arch/x86/kvm/vmx/tdx.h line 20
Quoted 6 lines · On Sun, 2023-03-12 at 10:55 -0700, [email protected] wrote:
arch/x86/kvm/vmx/tdx.h new file +6 -0
+
+static inline bool is_td(struct kvm *kvm)
+{
+	return kvm->arch.vm_type == KVM_X86_PROTECTED_VM;
+}
+
KVM_X86_PROTECTED_VM is defined in the next patch, which means this patch cannot compile.

Quoted 1 line

Also, why KVM_X86_PROTECTED_VM == TDX VM?

This part needs to be resolved.
Use KVM_X86_PROTECTED_VM as TDX VM or new type KVM_X86_TDX_VM or other way to specify VM type.
Given pKVM is coming, dedicated VM type seems better. I'll switch it to KVM_X86_TDX_VM next respin.

Signature · 1 line

Isaku Yamahata <[email protected]>

Huang, Kai

2023-03-16 21:42 UTC #27 lore raw

Re: [PATCH v13 005/113] KVM: TDX: Add placeholders for TDX VM/vcpu structure

Quoted 4 lines · Isaku Yamahata

Quoted 1 line · Kai
On Sun, 2023-03-12 at 10:55 -0700, [email protected] wrote:

Quoted 9 lines

Quoted 9 lines
Review
Quoted 6 lines
(quoted diff) +6 -0
+
+static inline bool is_td(struct kvm *kvm)
+{
+	return kvm->arch.vm_type == KVM_X86_PROTECTED_VM;
+}
+
KVM_X86_PROTECTED_VM is defined in the next patch, which means this patch cannot compile.

Oh I see your patch only added this type to tools/.../asm/kvm.h

tools/arch/x86/include/uapi/asm/kvm.h +3 -0

@@ -559,4 +559,7 @@ struct kvm_pmu_event_filter {
 #define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
 #define   KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
 
+#define KVM_X86_DEFAULT_VM	0
+#define KVM_X86_PROTECTED_VM	1
+

So this one has already been in UPM series.

But I thought Sean had a comment saying we should not add such definition to header file under tools/:

Even it needs to be added, it should be in the relevant patch in UPM series, but not here.

Quoted 10 lines

Quoted 2 lines
Also, why KVM_X86_PROTECTED_VM == TDX VM?
This part needs to be resolved.
Use KVM_X86_PROTECTED_VM as TDX VM or new type KVM_X86_TDX_VM or other way to specify VM type.
Given pKVM is coming, dedicated VM type seems better. I'll switch it to KVM_X86_TDX_VM next respin.
Signature · 1 line
Isaku Yamahata <[email protected]>

I'll leave to Sean and Paolo.

[email protected]

author

2023-03-12 17:58 UTC #28 lore raw

PATCH v13 6/113 KVM: x86: Introduce vm_type to differentiate default VMs from confidential VMs

From: Sean Christopherson <[email protected]>

Unlike default VMs, confidential VMs (Intel TDX and AMD SEV-ES) don't allow some operations (e.g., memory read/write, register state access, etc).

Introduce vm_type to track the type of the VM to x86 KVM. Other arch KVMs already use vm_type, KVM_INIT_VM accepts vm_type, and x86 KVM callback vm_init accepts vm_type. So follow them. Further, a different policy can be made based on vm_type. Define KVM_X86_DEFAULT_VM for default VM as default and define KVM_X86_TDX_VM for Intel TDX VM. The wrapper function will be defined as "bool is_td(kvm) { return vm_type == VM_TYPE_TDX; }"

Add a capability KVM_CAP_VM_TYPES to effectively allow device model, e.g. qemu, to query what VM types are supported by KVM. This (introduce a new capability and add vm_type) is chosen to align with other arch KVMs that have VM types already. Other arch KVMs use different names to query supported vm types and there is no common name for it, so new name was chosen.

Co-developed-by Xiaoyao Li <[email protected]>

Signed-off-by Xiaoyao Li <[email protected]>

Signed-off-by Sean Christopherson <[email protected]>

---
 Documentation/virt/kvm/api.rst        | 4 +++-
 arch/x86/include/asm/kvm-x86-ops.h    | 1 +
 arch/x86/include/asm/kvm_host.h       | 1 +
 arch/x86/kvm/svm/svm.c                | 7 +++++++
 arch/x86/kvm/vmx/main.c               | 1 +
 arch/x86/kvm/vmx/vmx.c                | 5 +++++
 arch/x86/kvm/vmx/x86_ops.h            | 1 +
 arch/x86/kvm/x86.c                    | 8 +++++++-
 arch/x86/kvm/x86.h                    | 2 ++
 tools/arch/x86/include/uapi/asm/kvm.h | 3 +++
 10 files changed, 31 insertions(+), 2 deletions(-)

Documentation/virt/kvm/api.rst +3 -1

index 15754ef8470f..7a34efd31dcc 100644
@@ -150,7 +150,9 @@ You probably want to use 0 as machine type.
 X86:
 ^^^^
 
-Supported X86 VM types can be queried via KVM_CAP_VM_TYPES.
+Supported X86 VM types can be queried via KVM_CAP_VM_TYPES, which returns the
+bitmap of supported vm types. The 1-setting of bit @n means vm type with value
+@n is supported.
 
 S390:
 ^^^^^

arch/x86/include/asm/kvm-x86-ops.h +1 -0

index 8dc345cc6318..eac4b65d1b01 100644
@@ -20,6 +20,7 @@ KVM_X86_OP(hardware_disable)
 KVM_X86_OP(hardware_unsetup)
 KVM_X86_OP(has_emulated_msr)
 KVM_X86_OP(vcpu_after_set_cpuid)
+KVM_X86_OP(is_vm_type_supported)
 KVM_X86_OP(vm_init)
 KVM_X86_OP_OPTIONAL(vm_destroy)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)

arch/x86/include/asm/kvm_host.h +1 -0

index d9fc5a1c7442..00c25f6ab871 100644
@@ -1561,6 +1561,7 @@ struct kvm_x86_ops {
 	bool (*has_emulated_msr)(struct kvm *kvm, u32 index);
 	void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu);
 
+	bool (*is_vm_type_supported)(unsigned long vm_type);
 	unsigned int vm_size;
 	int (*vm_init)(struct kvm *kvm);
 	void (*vm_destroy)(struct kvm *kvm);

arch/x86/kvm/svm/svm.c +7 -0

index 252e7f37e4e2..687df130fbad 100644
@@ -4682,6 +4682,12 @@ static void svm_vm_destroy(struct kvm *kvm)
 	sev_vm_destroy(kvm);
 }
 
+static bool svm_is_vm_type_supported(unsigned long type)
+{
+	/* FIXME: Check if CPU is capable of SEV. */
+	return __kvm_is_vm_type_supported(type);
+}
+
 static int svm_vm_init(struct kvm *kvm)
 {
 	if (!pause_filter_count || !pause_filter_thresh)

@@ -4710,6 +4716,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.vcpu_free = svm_vcpu_free,
 	.vcpu_reset = svm_vcpu_reset,
 
+	.is_vm_type_supported = svm_is_vm_type_supported,
 	.vm_size = sizeof(struct kvm_svm),
 	.vm_init = svm_vm_init,
 	.vm_destroy = svm_vm_destroy,

arch/x86/kvm/vmx/main.c +1 -0

index 5bc31c41bea9..e36d392c2d40 100644
@@ -58,6 +58,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.hardware_disable = vmx_hardware_disable,
 	.has_emulated_msr = vmx_has_emulated_msr,
 
+	.is_vm_type_supported = vmx_is_vm_type_supported,
 	.vm_size = sizeof(struct kvm_vmx),
 	.vm_init = vmx_vm_init,
 	.vm_destroy = vmx_vm_destroy,

arch/x86/kvm/vmx/vmx.c +5 -0

index d23830d92f61..8134c77860d4 100644
@@ -7486,6 +7486,11 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)
 	return err;
 }
 
+bool vmx_is_vm_type_supported(unsigned long type)
+{
+	return type == KVM_X86_DEFAULT_VM;
+}
+
 #define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
 #define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"

arch/x86/kvm/vmx/x86_ops.h +1 -0

index 010d02c86ba4..30961fdbce10 100644
@@ -32,6 +32,7 @@ void vmx_hardware_unsetup(void);
 int vmx_check_processor_compat(void);
 int vmx_hardware_enable(void);
 void vmx_hardware_disable(void);
+bool vmx_is_vm_type_supported(unsigned long type);
 int vmx_vm_init(struct kvm *kvm);
 void vmx_vm_destroy(struct kvm *kvm);
 int vmx_vcpu_precreate(struct kvm *kvm);

arch/x86/kvm/x86.c +7 -1

index 2125fcaa3973..27ab684f8374 100644
@@ -4356,12 +4356,18 @@ static int kvm_ioctl_get_supported_hv_cpuid(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
-static bool kvm_is_vm_type_supported(unsigned long type)
+bool __kvm_is_vm_type_supported(unsigned long type)
 {
 	return type == KVM_X86_DEFAULT_VM ||
 	       (type == KVM_X86_PROTECTED_VM &&
 	        IS_ENABLED(CONFIG_KVM_PROTECTED_VM) && tdp_enabled);
 }
+EXPORT_SYMBOL_GPL(__kvm_is_vm_type_supported);
+
+static bool kvm_is_vm_type_supported(unsigned long type)
+{
+	return static_call(kvm_x86_is_vm_type_supported)(type);
+}
 
 int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 {

arch/x86/kvm/x86.h +2 -0

index a8167b47b8c8..ef1282c7de8d 100644
@@ -8,6 +8,8 @@
 #include "kvm_cache_regs.h"
 #include "kvm_emulate.h"
 
+bool __kvm_is_vm_type_supported(unsigned long type);
+
 struct kvm_caps {
 	/* control of guest tsc rate supported? */
 	bool has_tsc_control;

tools/arch/x86/include/uapi/asm/kvm.h +3 -0

index 7f467fe05d42..6afbfbb32d56 100644
@@ -559,4 +559,7 @@ struct kvm_pmu_event_filter {
 #define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
 #define   KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
 
+#define KVM_X86_DEFAULT_VM	0
+#define KVM_X86_PROTECTED_VM	1
+
 #endif /* _ASM_X86_KVM_H */

Signature · 1 line

2.25.1

Huang, Kai

2023-03-15 10:46 UTC #29 lore raw

Re: [PATCH v13 006/113] KVM: x86: Introduce vm_type to differentiate default VMs from confidential VMs

Quoted 9 lines · On Sun, 2023-03-12 at 10:55 -0700, [email protected] wrote:

arch/x86/kvm/x86.c +1 -1
-static bool kvm_is_vm_type_supported(unsigned long type)
+bool __kvm_is_vm_type_supported(unsigned long type)
{
return type == KVM_X86_DEFAULT_VM ||
(type == KVM_X86_PROTECTED_VM && IS_ENABLED(CONFIG_KVM_PROTECTED_VM) && tdp_enabled); }
+EXPORT_SYMBOL_GPL(__kvm_is_vm_type_supported);

Where is CONFIG_KVM_PROTECTED_VM introduced?

Isaku Yamahata

2023-03-16 15:38 UTC #30 lore raw

Re: [PATCH v13 006/113] KVM: x86: Introduce vm_type to differentiate default VMs from confidential VMs

Quoted 12 lines · Kai

Quoted 9 lines · On Sun, 2023-03-12 at 10:55 -0700, [email protected] wrote:
(quoted diff) +1 -1
-static bool kvm_is_vm_type_supported(unsigned long type)
+bool __kvm_is_vm_type_supported(unsigned long type)
{
return type == KVM_X86_DEFAULT_VM ||
(type == KVM_X86_PROTECTED_VM && IS_ENABLED(CONFIG_KVM_PROTECTED_VM) && tdp_enabled); }
+EXPORT_SYMBOL_GPL(__kvm_is_vm_type_supported);
Where is CONFIG_KVM_PROTECTED_VM introduced?

UPM patch series.

KVM: x86: Add support for "protected VMs" that can utilize private memory

arch/x86/include/asm/kvm_host.h +7 -0

index fa195797461e..e8f334b0ff2c 100644
@@ -1245,6 +1245,7 @@ enum kvm_apicv_inhibit {
 };
 
 struct kvm_arch {
+       unsigned long vm_type;
        unsigned long n_used_mmu_pages;
        unsigned long n_requested_mmu_pages;
        unsigned long n_max_mmu_pages;

@@ -2049,6 +2050,12 @@ void kvm_mmu_new_pgd(struct kvm_vcpu *vcpu, gpa_t new_pgd);
 void kvm_configure_mmu(bool enable_tdp, int tdp_forced_root_level,
                       int tdp_max_root_level, int tdp_huge_page_level);
 
+#ifdef CONFIG_KVM_PRIVATE_MEM
+#define kvm_arch_has_private_mem(kvm) ((kvm)->arch.vm_type != KVM_X86_DEFAULT_VM)
+#else
+#define kvm_arch_has_private_mem(kvm) false
+#endif
+
 static inline u16 kvm_read_ldt(void)
 {
        u16 ldt;

...

arch/x86/include/uapi/asm/kvm.h +3 -0

index 7f467fe05d42..6afbfbb32d56 100644
@@ -559,4 +559,7 @@ struct kvm_pmu_event_filter {
 #define KVM_VCPU_TSC_CTRL 0 /* control group for the timestamp counter (TSC) */
 #define   KVM_VCPU_TSC_OFFSET 0 /* attribute for the TSC offset */
 
+#define KVM_X86_DEFAULT_VM     0
+#define KVM_X86_PROTECTED_VM   1
+
 #endif /* _ASM_X86_KVM_H */

...

arch/x86/kvm/Kconfig +11 -0

index ed84bb3cd82d..718010600956 100644
@@ -78,6 +78,17 @@ config KVM_WERROR
 
          If in doubt, say "N".
 
+config KVM_PROTECTED_VM
+       bool "Enable support for KVM-protected VMs"
+       depends on EXPERT
+       depends on X86_64
+       select KVM_GENERIC_PRIVATE_MEM
+       help
+         Enable support KVM-protected VMs.  Currently 'protected' means the VM
+         can be backed with restricted/private memory.
+
+         If unsure, say "N".
+
 config KVM_INTEL
        tristate "KVM for Intel (and compatible) processo

Signature · 1 line

Isaku Yamahata <[email protected]>

[email protected]

author

2023-03-12 17:58 UTC #31 lore raw

PATCH v13 7/113 KVM: TDX: Make TDX VM type supported

From: Isaku Yamahata <[email protected]>

NOTE: This patch is in position of the patch series for developers to be able to test codes during the middle of the patch series although this patch series doesn't provide functional features until the all the patches of this patch series. When merging this patch series, this patch can be moved to the end.

As first step TDX VM support, return that TDX VM type supported to device model, e.g. qemu. The callback to create guest TD is vm_init callback for KVM_CREATE_VM.

---
 arch/x86/kvm/vmx/main.c    | 18 ++++++++++++++++--
 arch/x86/kvm/vmx/tdx.c     |  6 ++++++
 arch/x86/kvm/vmx/vmx.c     |  5 -----
 arch/x86/kvm/vmx/x86_ops.h |  3 ++-
 4 files changed, 24 insertions(+), 8 deletions(-)

arch/x86/kvm/vmx/main.c +16 -2

index e36d392c2d40..c8548004802a 100644
@@ -23,6 +23,12 @@ static int vt_hardware_enable(void)
 	return ret;
 }
 
+static bool vt_is_vm_type_supported(unsigned long type)
+{
+	return type == KVM_X86_DEFAULT_VM ||
+		(enable_tdx && tdx_is_vm_type_supported(type));
+}
+
 static __init int vt_hardware_setup(void)
 {
 	int ret;

@@ -36,6 +42,14 @@ static __init int vt_hardware_setup(void)
 	return 0;
 }
 
+static int vt_vm_init(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		return -EOPNOTSUPP;	/* Not ready to create guest TD yet. */
+
+	return vmx_vm_init(kvm);
+}
+
 #define VMX_REQUIRED_APICV_INHIBITS		       \
 (						       \
        BIT(APICV_INHIBIT_REASON_DISABLE)|	       \

@@ -58,9 +72,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.hardware_disable = vmx_hardware_disable,
 	.has_emulated_msr = vmx_has_emulated_msr,
 
-	.is_vm_type_supported = vmx_is_vm_type_supported,
+	.is_vm_type_supported = vt_is_vm_type_supported,
 	.vm_size = sizeof(struct kvm_vmx),
-	.vm_init = vmx_vm_init,
+	.vm_init = vt_vm_init,
 	.vm_destroy = vmx_vm_destroy,
 
 	.vcpu_precreate = vmx_vcpu_precreate,

arch/x86/kvm/vmx/tdx.c +6 -0

index 2f9eaefee249..36738904c26a 100644
@@ -30,6 +30,12 @@ static int __init tdx_module_setup(void)
 	return 0;
 }
 
+bool tdx_is_vm_type_supported(unsigned long type)
+{
+	/* enable_tdx check is done by the caller. */
+	return type == KVM_X86_PROTECTED_VM;
+}
+
 static int __init tdx_cpu_enable_cpu(void *unused)
 {
 	return tdx_cpu_enable();

arch/x86/kvm/vmx/vmx.c +0 -5

index 8134c77860d4..d23830d92f61 100644
@@ -7486,11 +7486,6 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)
 	return err;
 }
 
-bool vmx_is_vm_type_supported(unsigned long type)
-{
-	return type == KVM_X86_DEFAULT_VM;
-}
-
 #define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
 #define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"

arch/x86/kvm/vmx/x86_ops.h +2 -1

index 30961fdbce10..b2c74c1b5bbd 100644
@@ -32,7 +32,6 @@ void vmx_hardware_unsetup(void);
 int vmx_check_processor_compat(void);
 int vmx_hardware_enable(void);
 void vmx_hardware_disable(void);
-bool vmx_is_vm_type_supported(unsigned long type);
 int vmx_vm_init(struct kvm *kvm);
 void vmx_vm_destroy(struct kvm *kvm);
 int vmx_vcpu_precreate(struct kvm *kvm);

@@ -141,9 +140,11 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 #ifdef CONFIG_INTEL_TDX_HOST
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
 int tdx_hardware_enable(void);
+bool tdx_is_vm_type_supported(unsigned long type);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
+static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 17:58 UTC #32 lore raw

PATCH v13 8/113 The start of TDX KVM patch series: TDX architectural definitions

From: Isaku Yamahata <[email protected]>

This empty commit is to mark the start of patch series of TDX architectural definitions.

---
 Documentation/virt/kvm/index.rst              |  2 ++
 .../virt/kvm/intel-tdx-layer-status.rst       | 29 +++++++++++++++++++
 2 files changed, 31 insertions(+)
 create mode 100644 Documentation/virt/kvm/intel-tdx-layer-status.rst

Documentation/virt/kvm/index.rst +2 -0

index ad13ec55ddfe..7f5f803fa87a 100644
@@ -19,3 +19,5 @@ KVM
    vcpu-requests
    halt-polling
    review-checklist
+
+   intel-tdx-layer-status.rst

Documentation/virt/kvm/intel-tdx-layer-status.rst new file +29 -0

new file mode 100644
index 000000000000..f11ea701dc19
@@ -0,0 +1,29 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+===================================
+Intel Trust Dodmain Extensions(TDX)
+===================================
+
+Layer status
+============
+What qemu can do
+----------------
+- TDX VM TYPE is exposed to Qemu.
+- Qemu can try to create VM of TDX VM type and then fails.
+
+Patch Layer status
+------------------
+  Patch layer                          Status
+
+* TDX, VMX coexistence:                 Applied
+* TDX architectural definitions:        Applying
+* TD VM creation/destruction:           Not yet
+* TD vcpu creation/destruction:         Not yet
+* TDX EPT violation:                    Not yet
+* TD finalization:                      Not yet
+* TD vcpu enter/exit:                   Not yet
+* TD vcpu interrupts/exit/hypercall:    Not yet
+
+* KVM MMU GPA shared bits:              Not yet
+* KVM TDP refactoring for TDX:          Not yet
+* KVM TDP MMU hooks:                    Not yet

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 17:58 UTC #33 lore raw

PATCH v13 9/113 KVM: TDX: Define TDX architectural definitions

From: Isaku Yamahata <[email protected]>

Define architectural definitions for KVM to issue the TDX SEAMCALLs.

Structures and values that are architecturally defined in the TDX module specifications the chapter of ABI Reference.

Co-developed-by Sean Christopherson <[email protected]>

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/kvm/vmx/tdx_arch.h | 168 ++++++++++++++++++++++++++++++++++++
 1 file changed, 168 insertions(+)
 create mode 100644 arch/x86/kvm/vmx/tdx_arch.h

arch/x86/kvm/vmx/tdx_arch.h new file +168 -0

new file mode 100644
index 000000000000..942a0e561a7b
@@ -0,0 +1,168 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* architectural constants/data definitions for TDX SEAMCALLs */
+
+#ifndef __KVM_X86_TDX_ARCH_H
+#define __KVM_X86_TDX_ARCH_H
+
+#include <linux/types.h>
+
+/*
+ * TDX SEAMCALL API function leaves
+ */
+#define TDH_VP_ENTER			0
+#define TDH_MNG_ADDCX			1
+#define TDH_MEM_PAGE_ADD		2
+#define TDH_MEM_SEPT_ADD		3
+#define TDH_VP_ADDCX			4
+#define TDH_MEM_PAGE_RELOCATE		5
+#define TDH_MEM_PAGE_AUG		6
+#define TDH_MEM_RANGE_BLOCK		7
+#define TDH_MNG_KEY_CONFIG		8
+#define TDH_MNG_CREATE			9
+#define TDH_VP_CREATE			10
+#define TDH_MNG_RD			11
+#define TDH_MR_EXTEND			16
+#define TDH_MR_FINALIZE			17
+#define TDH_VP_FLUSH			18
+#define TDH_MNG_VPFLUSHDONE		19
+#define TDH_MNG_KEY_FREEID		20
+#define TDH_MNG_INIT			21
+#define TDH_VP_INIT			22
+#define TDH_VP_RD			26
+#define TDH_MNG_KEY_RECLAIMID		27
+#define TDH_PHYMEM_PAGE_RECLAIM		28
+#define TDH_MEM_PAGE_REMOVE		29
+#define TDH_MEM_SEPT_REMOVE		30
+#define TDH_MEM_TRACK			38
+#define TDH_MEM_RANGE_UNBLOCK		39
+#define TDH_PHYMEM_CACHE_WB		40
+#define TDH_PHYMEM_PAGE_WBINVD		41
+#define TDH_VP_WR			43
+#define TDH_SYS_LP_SHUTDOWN		44
+
+#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO		0x10000
+#define TDG_VP_VMCALL_MAP_GPA				0x10001
+#define TDG_VP_VMCALL_GET_QUOTE				0x10002
+#define TDG_VP_VMCALL_REPORT_FATAL_ERROR		0x10003
+#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT	0x10004
+
+/* TDX control structure (TDR/TDCS/TDVPS) field access codes */
+#define TDX_NON_ARCH			BIT_ULL(63)
+#define TDX_CLASS_SHIFT			56
+#define TDX_FIELD_MASK			GENMASK_ULL(31, 0)
+
+#define __BUILD_TDX_FIELD(non_arch, class, field)	\
+	(((non_arch) ? TDX_NON_ARCH : 0) |		\
+	 ((u64)(class) << TDX_CLASS_SHIFT) |		\
+	 ((u64)(field) & TDX_FIELD_MASK))
+
+#define BUILD_TDX_FIELD(class, field)			\
+	__BUILD_TDX_FIELD(false, (class), (field))
+
+#define BUILD_TDX_FIELD_NON_ARCH(class, field)		\
+	__BUILD_TDX_FIELD(true, (class), (field))
+
+
+/* Class code for TD */
+#define TD_CLASS_EXECUTION_CONTROLS	17ULL
+
+/* Class code for TDVPS */
+#define TDVPS_CLASS_VMCS		0ULL
+#define TDVPS_CLASS_GUEST_GPR		16ULL
+#define TDVPS_CLASS_OTHER_GUEST		17ULL
+#define TDVPS_CLASS_MANAGEMENT		32ULL
+
+enum tdx_tdcs_execution_control {
+	TD_TDCS_EXEC_TSC_OFFSET = 10,
+};
+
+/* @field is any of enum tdx_tdcs_execution_control */
+#define TDCS_EXEC(field)		BUILD_TDX_FIELD(TD_CLASS_EXECUTION_CONTROLS, (field))
+
+/* @field is the VMCS field encoding */
+#define TDVPS_VMCS(field)		BUILD_TDX_FIELD(TDVPS_CLASS_VMCS, (field))
+
+enum tdx_vcpu_guest_other_state {
+	TD_VCPU_STATE_DETAILS_NON_ARCH = 0x100,
+};
+
+union tdx_vcpu_state_details {
+	struct {
+		u64 vmxip	: 1;
+		u64 reserved	: 63;
+	};
+	u64 full;
+};
+
+/* @field is any of enum tdx_guest_other_state */
+#define TDVPS_STATE(field)		BUILD_TDX_FIELD(TDVPS_CLASS_OTHER_GUEST, (field))
+#define TDVPS_STATE_NON_ARCH(field)	BUILD_TDX_FIELD_NON_ARCH(TDVPS_CLASS_OTHER_GUEST, (field))
+
+/* Management class fields */
+enum tdx_vcpu_guest_management {
+	TD_VCPU_PEND_NMI = 11,
+};
+
+/* @field is any of enum tdx_vcpu_guest_management */
+#define TDVPS_MANAGEMENT(field)		BUILD_TDX_FIELD(TDVPS_CLASS_MANAGEMENT, (field))
+
+#define TDX_EXTENDMR_CHUNKSIZE		256
+
+struct tdx_cpuid_value {
+	u32 eax;
+	u32 ebx;
+	u32 ecx;
+	u32 edx;
+} __packed;
+
+#define TDX_TD_ATTRIBUTE_DEBUG		BIT_ULL(0)
+#define TDX_TD_ATTRIBUTE_PKS		BIT_ULL(30)
+#define TDX_TD_ATTRIBUTE_KL		BIT_ULL(31)
+#define TDX_TD_ATTRIBUTE_PERFMON	BIT_ULL(63)
+
+/*
+ * TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B.
+ */
+#define TDX_MAX_VCPUS	(~(u16)0)
+
+struct td_params {
+	u64 attributes;
+	u64 xfam;
+	u16 max_vcpus;
+	u8 reserved0[6];
+
+	u64 eptp_controls;
+	u64 exec_controls;
+	u16 tsc_frequency;
+	u8  reserved1[38];
+
+	u64 mrconfigid[6];
+	u64 mrowner[6];
+	u64 mrownerconfig[6];
+	u64 reserved2[4];
+
+	union {
+		struct tdx_cpuid_value cpuid_values[0];
+		u8 reserved3[768];
+	};
+} __packed __aligned(1024);
+
+/*
+ * Guest uses MAX_PA for GPAW when set.
+ * 0: GPA.SHARED bit is GPA[47]
+ * 1: GPA.SHARED bit is GPA[51]
+ */
+#define TDX_EXEC_CONTROL_MAX_GPAW      BIT_ULL(0)
+
+/*
+ * TDX requires the frequency to be defined in units of 25MHz, which is the
+ * frequency of the core crystal clock on TDX-capable platforms, i.e. the TDX
+ * module can only program frequencies that are multiples of 25MHz.  The
+ * frequency must be between 100mhz and 10ghz (inclusive).
+ */
+#define TDX_TSC_KHZ_TO_25MHZ(tsc_in_khz)	((tsc_in_khz) / (25 * 1000))
+#define TDX_TSC_25MHZ_TO_KHZ(tsc_in_25mhz)	((tsc_in_25mhz) * (25 * 1000))
+#define TDX_MIN_TSC_FREQUENCY_KHZ		(100 * 1000)
+#define TDX_MAX_TSC_FREQUENCY_KHZ		(10 * 1000 * 1000)
+
+#endif /* __KVM_X86_TDX_ARCH_H */

Signature · 1 line

2.25.1

Xiaoyao Li

2023-03-13 11:37 UTC #34 lore raw

Re: [PATCH v13 009/113] KVM: TDX: Define TDX architectural definitions

Quoted 5 lines · On 3/13/2023 1:55 AM, [email protected] wrote:

arch/x86/kvm/vmx/tdx_arch.h new file +0 -0
new file mode 100644
index 000000000000..942a0e561a7b

...

arch/x86/kvm/vmx/tdx_arch.h line 47

Quoted 6 lines

arch/x86/kvm/vmx/tdx_arch.h new file +6 -0

+
+#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO		0x10000
+#define TDG_VP_VMCALL_MAP_GPA				0x10001
+#define TDG_VP_VMCALL_GET_QUOTE				0x10002
+#define TDG_VP_VMCALL_REPORT_FATAL_ERROR		0x10003
+#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT	0x10004

TDX guest side implementation defines

    /* TDX hypercall Leaf IDs */
    #define TDVMCALL_MAP_GPA			0x10001
    #define TDVMCALL_REPORT_FATAL_ERROR		0x10003

We need put these shared definitions between host and guest into some shared header file.

Isaku Yamahata

2023-03-13 19:17 UTC #35 lore raw

Re: [PATCH v13 009/113] KVM: TDX: Define TDX architectural definitions

Quoted 16 lines · Xiaoyao Li

Quoted 5 lines · On 3/13/2023 1:55 AM, [email protected] wrote:
arch/x86/kvm/vmx/tdx_arch.h new file +0 -0
new file mode 100644
index 000000000000..942a0e561a7b
...
arch/x86/kvm/vmx/tdx_arch.h line 47
Quoted 6 lines
arch/x86/kvm/vmx/tdx_arch.h new file +6 -0
+
+#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO		0x10000
+#define TDG_VP_VMCALL_MAP_GPA				0x10001
+#define TDG_VP_VMCALL_GET_QUOTE				0x10002
+#define TDG_VP_VMCALL_REPORT_FATAL_ERROR		0x10003
+#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT	0x10004
TDX guest side implementation defines

Quoted 6 lines

    /* TDX hypercall Leaf IDs */
    #define TDVMCALL_MAP_GPA			0x10001
    #define TDVMCALL_REPORT_FATAL_ERROR		0x10003
We need put these shared definitions between host and guest into some shared header file.

Good suggestion. Let me fix it.

Signature · 1 line

Isaku Yamahata <[email protected]>

[email protected]

author

2023-03-12 17:58 UTC #36 lore raw

PATCH v13 10/113 KVM: TDX: Add TDX "architectural" error codes

From: Sean Christopherson <[email protected]>

Add error codes for the TDX SEAMCALLs both for TDX VMM side for TDH SEAMCALL and TDX guest side for TDG.VP.VMCALL. KVM issues the TDX SEAMCALLs and checks its error code. KVM handles hypercall from the TDX guest and may return an error. So error code for the TDX guest is also needed.

TDX SEAMCALL uses bits 31:0 to return more information, so these error codes will only exactly match RAX[63:32]. Error codes for TDG.VP.VMCALL is defined by TDX Guest-Host-Communication interface spec.

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/kvm/vmx/tdx_errno.h | 38 ++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)
 create mode 100644 arch/x86/kvm/vmx/tdx_errno.h

arch/x86/kvm/vmx/tdx_errno.h new file +38 -0

new file mode 100644
index 000000000000..389b1b53da25
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* architectural status code for SEAMCALL */
+
+#ifndef __KVM_X86_TDX_ERRNO_H
+#define __KVM_X86_TDX_ERRNO_H
+
+#define TDX_SEAMCALL_STATUS_MASK		0xFFFFFFFF00000000ULL
+
+/*
+ * TDX SEAMCALL Status Codes (returned in RAX)
+ */
+#define TDX_SUCCESS				0x0000000000000000ULL
+#define TDX_NON_RECOVERABLE_VCPU		0x4000000100000000ULL
+#define TDX_INTERRUPTED_RESUMABLE		0x8000000300000000ULL
+#define TDX_OPERAND_BUSY			0x8000020000000000ULL
+#define TDX_VCPU_NOT_ASSOCIATED			0x8000070200000000ULL
+#define TDX_KEY_GENERATION_FAILED		0x8000080000000000ULL
+#define TDX_KEY_STATE_INCORRECT			0xC000081100000000ULL
+#define TDX_KEY_CONFIGURED			0x0000081500000000ULL
+#define TDX_NO_HKID_READY_TO_WBCACHE		0x0000082100000000ULL
+#define TDX_EPT_WALK_FAILED			0xC0000B0000000000ULL
+
+/*
+ * TDG.VP.VMCALL Status Codes (returned in R10)
+ */
+#define TDG_VP_VMCALL_SUCCESS			0x0000000000000000ULL
+#define TDG_VP_VMCALL_RETRY			0x0000000000000001ULL
+#define TDG_VP_VMCALL_INVALID_OPERAND		0x8000000000000000ULL
+#define TDG_VP_VMCALL_TDREPORT_FAILED		0x8000000000000001ULL
+
+/*
+ * TDX module operand ID, appears in 31:0 part of error code as
+ * detail information
+ */
+#define TDX_OPERAND_ID_RCX			0x01
+#define TDX_OPERAND_ID_SEPT			0x92
+
+#endif /* __KVM_X86_TDX_ERRNO_H */

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 17:58 UTC #37 lore raw

PATCH v13 11/113 KVM: TDX: Add C wrapper functions for SEAMCALLs to the TDX module

From: Isaku Yamahata <[email protected]>

A VMM interacts with the TDX module using a new instruction (SEAMCALL). A TDX VMM uses SEAMCALLs where a VMX VMM would have directly interacted with VMX instructions. For instance, a TDX VMM does not have full access to the VM control structure corresponding to VMX VMCS. Instead, a VMM induces the TDX module to act on behalf via SEAMCALLs.

Export __seamcall and define C wrapper functions for SEAMCALLs for readability.

Some SEAMCALL APIs donates host pages to TDX module or guest TD and the donated pages are encrypted. Some of such SEAMCALLs flush cache lines (typically by movdir64b instruction), some don't. Those that doesn't clear cache lines require the VMM to flush the cache lines to avoid cache line alias.

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/include/asm/tdx.h       |   4 +
 arch/x86/kvm/vmx/tdx_ops.h       | 202 +++++++++++++++++++++++++++++++
 arch/x86/virt/vmx/tdx/seamcall.S |   2 +
 arch/x86/virt/vmx/tdx/tdx.h      |   3 -
 4 files changed, 208 insertions(+), 3 deletions(-)
 create mode 100644 arch/x86/kvm/vmx/tdx_ops.h

arch/x86/include/asm/tdx.h +4 -0

index 112a5b9bd5cd..6c01ab572c1f 100644
@@ -104,10 +104,14 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,
 bool platform_tdx_enabled(void);
 int tdx_cpu_enable(void);
 int tdx_enable(void);
+u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
+	       struct tdx_module_output *out);
 #else	/* !CONFIG_INTEL_TDX_HOST */
 static inline bool platform_tdx_enabled(void) { return false; }
 static inline int tdx_cpu_enable(void) { return -EINVAL; }
 static inline int tdx_enable(void)  { return -EINVAL; }
+static inline u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
+			     struct tdx_module_output *out) { return TDX_SEAMCALL_UD; };
 #endif	/* CONFIG_INTEL_TDX_HOST */
 
 #endif /* !__ASSEMBLY__ */

arch/x86/kvm/vmx/tdx_ops.h new file +202 -0

202 lines not on this page · raw message

arch/x86/virt/vmx/tdx/seamcall.S +2 -0

index f81be6b9c133..b90a7fe05494 100644
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 #include <linux/linkage.h>
+#include <asm/export.h>
 #include <asm/frame.h>
 
 #include "tdxcall.S"

@@ -50,3 +51,4 @@ SYM_FUNC_START(__seamcall)
 	FRAME_END
 	RET
 SYM_FUNC_END(__seamcall)
+EXPORT_SYMBOL_GPL(__seamcall)

arch/x86/virt/vmx/tdx/tdx.h +0 -3

index 48f830087e7e..4e497f202586 100644
@@ -144,7 +144,4 @@ struct tdmr_info_list {
 	int max_tdmrs;	/* How many 'tdmr_info's are allocated */
 };
 
-struct tdx_module_output;
-u64 __seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
-	       struct tdx_module_output *out);
 #endif

Signature · 1 line

2.25.1

Zhi Wang

2023-03-24 15:20 UTC #38 lore raw

Re: [PATCH v13 011/113] KVM: TDX: Add C wrapper functions for SEAMCALLs to the TDX module

Quoted 4 lines · On Sun, 12 Mar 2023 10:55:35 -0700 [email protected] wrote:

From: Isaku Yamahata <[email protected]>
A VMM interacts with the TDX module using a new instruction (SEAMCALL). A TDX VMM uses SEAMCALLs where a VMX VMM would have directly interacted with

^ This sentence is little bit confusing, it can be removed. The next two sentences have stated the situation clearly.

Quoted 9 lines

VMX instructions. For instance, a TDX VMM does not have full access to the VM control structure corresponding to VMX VMCS. Instead, a VMM induces the TDX module to act on behalf via SEAMCALLs.
Export __seamcall and define C wrapper functions for SEAMCALLs for readability.
Some SEAMCALL APIs donates host pages to TDX module or guest TD and the donated pages are encrypted. Some of such SEAMCALLs flush cache lines

^ "some of" can be removed.

Quoted 262 lines

(typically by movdir64b instruction), some don't. Those that doesn't clear cache lines require the VMM to flush the cache lines to avoid cache line alias.

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/include/asm/tdx.h       |   4 +
 arch/x86/kvm/vmx/tdx_ops.h       | 202 +++++++++++++++++++++++++++++++
 arch/x86/virt/vmx/tdx/seamcall.S |   2 +
 arch/x86/virt/vmx/tdx/tdx.h      |   3 -
 4 files changed, 208 insertions(+), 3 deletions(-)
 create mode 100644 arch/x86/kvm/vmx/tdx_ops.h

arch/x86/include/asm/tdx.h +4 -0

index 112a5b9bd5cd..6c01ab572c1f 100644
@@ -104,10 +104,14 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,
 bool platform_tdx_enabled(void);
 int tdx_cpu_enable(void);
 int tdx_enable(void);
+u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
+	       struct tdx_module_output *out);
 #else	/* !CONFIG_INTEL_TDX_HOST */
 static inline bool platform_tdx_enabled(void) { return false; }
 static inline int tdx_cpu_enable(void) { return -EINVAL; }
 static inline int tdx_enable(void)  { return -EINVAL; }
+static inline u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
+			     struct tdx_module_output *out) { return TDX_SEAMCALL_UD; };
 #endif	/* CONFIG_INTEL_TDX_HOST */
 
 #endif /* !__ASSEMBLY__ */

arch/x86/kvm/vmx/tdx_ops.h new file +202 -0

new file mode 100644
index 000000000000..70e569838e1c
@@ -0,0 +1,202 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* constants/data definitions for TDX SEAMCALLs */
+
+#ifndef __KVM_X86_TDX_OPS_H
+#define __KVM_X86_TDX_OPS_H
+
+#include <linux/compiler.h>
+
+#include <asm/cacheflush.h>
+#include <asm/asm.h>
+#include <asm/kvm_host.h>
+
+#include "tdx_errno.h"
+#include "tdx_arch.h"
+#include "x86.h"
+
+static inline u64 kvm_seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
+			       struct tdx_module_output *out)
+{
+	u64 ret;
+
+	ret = __seamcall(op, rcx, rdx, r8, r9, out);
+	if (ret == TDX_SEAMCALL_UD) {
+		/*
+		 * TDX requires VMXON or #UD.  In the case of reboot or kexec,
+		 * VMX is made off (VMXOFF) by kvm reboot notifier,
+		 * kvm_reboot(), while TDs are still running.  The callers check
+		 * the returned error and complain.  Suppress it by returning 0.
+		 */
+		kvm_spurious_fault();
+		return 0;
+	}
+	return ret;
+}
+
+static inline u64 tdh_mng_addcx(hpa_t tdr, hpa_t addr)
+{
+	clflush_cache_range(__va(addr), PAGE_SIZE);
+	return kvm_seamcall(TDH_MNG_ADDCX, addr, tdr, 0, 0, NULL);
+}
+
+static inline u64 tdh_mem_page_add(hpa_t tdr, gpa_t gpa, hpa_t hpa, hpa_t source,
+				   struct tdx_module_output *out)
+{
+	clflush_cache_range(__va(hpa), PAGE_SIZE);
+	return kvm_seamcall(TDH_MEM_PAGE_ADD, gpa, tdr, hpa, source, out);
+}
+
+static inline u64 tdh_mem_sept_add(hpa_t tdr, gpa_t gpa, int level, hpa_t page,
+				   struct tdx_module_output *out)
+{
+	clflush_cache_range(__va(page), PAGE_SIZE);
+	return kvm_seamcall(TDH_MEM_SEPT_ADD, gpa | level, tdr, page, 0, out);
+}
+
+static inline u64 tdh_mem_sept_remove(hpa_t tdr, gpa_t gpa, int level,
+				      struct tdx_module_output *out)
+{
+	return kvm_seamcall(TDH_MEM_SEPT_REMOVE, gpa | level, tdr, 0, 0, out);
+}
+
+static inline u64 tdh_vp_addcx(hpa_t tdvpr, hpa_t addr)
+{
+	clflush_cache_range(__va(addr), PAGE_SIZE);
+	return kvm_seamcall(TDH_VP_ADDCX, addr, tdvpr, 0, 0, NULL);
+}
+
+static inline u64 tdh_mem_page_relocate(hpa_t tdr, gpa_t gpa, hpa_t hpa,
+					struct tdx_module_output *out)
+{
+	clflush_cache_range(__va(hpa), PAGE_SIZE);
+	return kvm_seamcall(TDH_MEM_PAGE_RELOCATE, gpa, tdr, hpa, 0, out);
+}
+
+static inline u64 tdh_mem_page_aug(hpa_t tdr, gpa_t gpa, hpa_t hpa,
+				   struct tdx_module_output *out)
+{
+	clflush_cache_range(__va(hpa), PAGE_SIZE);
+	return kvm_seamcall(TDH_MEM_PAGE_AUG, gpa, tdr, hpa, 0, out);
+}
+
+static inline u64 tdh_mem_range_block(hpa_t tdr, gpa_t gpa, int level,
+				      struct tdx_module_output *out)
+{
+	return kvm_seamcall(TDH_MEM_RANGE_BLOCK, gpa | level, tdr, 0, 0, out);
+}
+
+static inline u64 tdh_mng_key_config(hpa_t tdr)
+{
+	return kvm_seamcall(TDH_MNG_KEY_CONFIG, tdr, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mng_create(hpa_t tdr, int hkid)
+{
+	clflush_cache_range(__va(tdr), PAGE_SIZE);
+	return kvm_seamcall(TDH_MNG_CREATE, tdr, hkid, 0, 0, NULL);
+}
+
+static inline u64 tdh_vp_create(hpa_t tdr, hpa_t tdvpr)
+{
+	clflush_cache_range(__va(tdvpr), PAGE_SIZE);
+	return kvm_seamcall(TDH_VP_CREATE, tdvpr, tdr, 0, 0, NULL);
+}
+
+static inline u64 tdh_mng_rd(hpa_t tdr, u64 field, struct tdx_module_output *out)
+{
+	return kvm_seamcall(TDH_MNG_RD, tdr, field, 0, 0, out);
+}
+
+static inline u64 tdh_mr_extend(hpa_t tdr, gpa_t gpa,
+				struct tdx_module_output *out)
+{
+	return kvm_seamcall(TDH_MR_EXTEND, gpa, tdr, 0, 0, out);
+}
+
+static inline u64 tdh_mr_finalize(hpa_t tdr)
+{
+	return kvm_seamcall(TDH_MR_FINALIZE, tdr, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_vp_flush(hpa_t tdvpr)
+{
+	return kvm_seamcall(TDH_VP_FLUSH, tdvpr, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mng_vpflushdone(hpa_t tdr)
+{
+	return kvm_seamcall(TDH_MNG_VPFLUSHDONE, tdr, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mng_key_freeid(hpa_t tdr)
+{
+	return kvm_seamcall(TDH_MNG_KEY_FREEID, tdr, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mng_init(hpa_t tdr, hpa_t td_params,
+			       struct tdx_module_output *out)
+{
+	return kvm_seamcall(TDH_MNG_INIT, tdr, td_params, 0, 0, out);
+}
+
+static inline u64 tdh_vp_init(hpa_t tdvpr, u64 rcx)
+{
+	return kvm_seamcall(TDH_VP_INIT, tdvpr, rcx, 0, 0, NULL);
+}
+
+static inline u64 tdh_vp_rd(hpa_t tdvpr, u64 field,
+			    struct tdx_module_output *out)
+{
+	return kvm_seamcall(TDH_VP_RD, tdvpr, field, 0, 0, out);
+}
+
+static inline u64 tdh_mng_key_reclaimid(hpa_t tdr)
+{
+	return kvm_seamcall(TDH_MNG_KEY_RECLAIMID, tdr, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_phymem_page_reclaim(hpa_t page,
+					  struct tdx_module_output *out)
+{
+	return kvm_seamcall(TDH_PHYMEM_PAGE_RECLAIM, page, 0, 0, 0, out);
+}
+
+static inline u64 tdh_mem_page_remove(hpa_t tdr, gpa_t gpa, int level,
+				      struct tdx_module_output *out)
+{
+	return kvm_seamcall(TDH_MEM_PAGE_REMOVE, gpa | level, tdr, 0, 0, out);
+}
+
+static inline u64 tdh_sys_lp_shutdown(void)
+{
+	return kvm_seamcall(TDH_SYS_LP_SHUTDOWN, 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mem_track(hpa_t tdr)
+{
+	return kvm_seamcall(TDH_MEM_TRACK, tdr, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_mem_range_unblock(hpa_t tdr, gpa_t gpa, int level,
+					struct tdx_module_output *out)
+{
+	return kvm_seamcall(TDH_MEM_RANGE_UNBLOCK, gpa | level, tdr, 0, 0, out);
+}
+
+static inline u64 tdh_phymem_cache_wb(bool resume)
+{
+	return kvm_seamcall(TDH_PHYMEM_CACHE_WB, resume ? 1 : 0, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_phymem_page_wbinvd(hpa_t page)
+{
+	return kvm_seamcall(TDH_PHYMEM_PAGE_WBINVD, page, 0, 0, 0, NULL);
+}
+
+static inline u64 tdh_vp_wr(hpa_t tdvpr, u64 field, u64 val, u64 mask,
+			    struct tdx_module_output *out)
+{
+	return kvm_seamcall(TDH_VP_WR, tdvpr, field, val, mask, out);
+}
+
+#endif /* __KVM_X86_TDX_OPS_H */

arch/x86/virt/vmx/tdx/seamcall.S +2 -0

index f81be6b9c133..b90a7fe05494 100644
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 #include <linux/linkage.h>
+#include <asm/export.h>
 #include <asm/frame.h>
 
 #include "tdxcall.S"

@@ -50,3 +51,4 @@ SYM_FUNC_START(__seamcall)
 	FRAME_END
 	RET
 SYM_FUNC_END(__seamcall)
+EXPORT_SYMBOL_GPL(__seamcall)

arch/x86/virt/vmx/tdx/tdx.h +0 -0

index 48f830087e7e..4e497f202586 100644

arch/x86/virt/vmx/tdx/tdx.h struct tdmr_info_list { · line 150

Quoted 7 lines

arch/x86/virt/vmx/tdx/tdx.h +0 -3

@@ -144,7 +144,4 @@ struct tdmr_info_list {
 	int max_tdmrs;	/* How many 'tdmr_info's are allocated */
 };
 
 
-struct tdx_module_output;
-u64 __seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
-	       struct tdx_module_output *out);

Better move this part to a correct place.

Quoted 1 line

#endif

[email protected]

author

2023-03-12 17:58 UTC #39 lore raw

PATCH v13 12/113 KVM: TDX: Add helper functions to print TDX SEAMCALL error

From: Isaku Yamahata <[email protected]>

Add helper functions to print out errors from the TDX module in a uniform manner.

---
 arch/x86/kvm/Makefile        |  2 +-
 arch/x86/kvm/vmx/tdx_error.c | 21 +++++++++++++++++++++
 arch/x86/kvm/vmx/tdx_ops.h   |  5 +++++
 3 files changed, 27 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kvm/vmx/tdx_error.c

arch/x86/kvm/Makefile +1 -1

index 4b01ab842ab7..e3354b784e10 100644
@@ -25,7 +25,7 @@ kvm-$(CONFIG_KVM_SMM)	+= smm.o
 kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \
 			   vmx/hyperv.o vmx/nested.o vmx/posted_intr.o vmx/main.o
 kvm-intel-$(CONFIG_X86_SGX_KVM)	+= vmx/sgx.o
-kvm-intel-$(CONFIG_INTEL_TDX_HOST)	+= vmx/tdx.o
+kvm-intel-$(CONFIG_INTEL_TDX_HOST)	+= vmx/tdx.o vmx/tdx_error.o
 
 kvm-amd-y		+= svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \
 			   svm/sev.o svm/hyperv.o

arch/x86/kvm/vmx/tdx_error.c new file +21 -0

new file mode 100644
index 000000000000..574b72d34e1e
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+/* functions to record TDX SEAMCALL error */
+
+#include <linux/kernel.h>
+#include <linux/bug.h>
+
+#include "tdx_ops.h"
+
+void pr_tdx_error(u64 op, u64 error_code, const struct tdx_module_output *out)
+{
+	if (!out) {
+		pr_err_ratelimited("SEAMCALL[%lld] failed: 0x%llx\n",
+				   op, error_code);
+		return;
+	}
+
+	pr_err_ratelimited("SEAMCALL[%lld] failed: 0x%llx RCX 0x%llx, RDX 0x%llx,"
+			   " R8 0x%llx, R9 0x%llx, R10 0x%llx, R11 0x%llx\n",
+			   op, error_code,
+			   out->rcx, out->rdx, out->r8, out->r9, out->r10, out->r11);
+}

arch/x86/kvm/vmx/tdx_ops.h +5 -0

index 70e569838e1c..177f444ebbb3 100644
@@ -9,6 +9,7 @@
 #include <asm/cacheflush.h>
 #include <asm/asm.h>
 #include <asm/kvm_host.h>
+#include <asm/tdx.h>
 
 #include "tdx_errno.h"
 #include "tdx_arch.h"

@@ -33,6 +34,10 @@ static inline u64 kvm_seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
 	return ret;
 }
 
+#ifdef CONFIG_INTEL_TDX_HOST
+void pr_tdx_error(u64 op, u64 error_code, const struct tdx_module_output *out);
+#endif
+
 static inline u64 tdh_mng_addcx(hpa_t tdr, hpa_t addr)
 {
 	clflush_cache_range(__va(addr), PAGE_SIZE);

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 17:58 UTC #40 lore raw

PATCH v13 13/113 The start of TDX KVM patch series: TD VM creation/destruction

From: Isaku Yamahata <[email protected]>

This empty commit is to mark the start of patch series of TD VM creation/destruction.

---
 Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Documentation/virt/kvm/intel-tdx-layer-status.rst +2 -2

index f11ea701dc19..098150da6ea2 100644
@@ -16,8 +16,8 @@ Patch Layer status
   Patch layer                          Status
 
 * TDX, VMX coexistence:                 Applied
-* TDX architectural definitions:        Applying
-* TD VM creation/destruction:           Not yet
+* TDX architectural definitions:        Applied
+* TD VM creation/destruction:           Applying
 * TD vcpu creation/destruction:         Not yet
 * TDX EPT violation:                    Not yet
 * TD finalization:                      Not yet

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 17:58 UTC #41 lore raw

PATCH v13 14/113 x86/cpu: Add helper functions to allocate/free TDX private host key id

From: Isaku Yamahata <[email protected]>

TDX private host key id (HKID) is assigned to guest TD. The memory controller encrypts guest TD memory with the assigned TDX HKID. Add helper functions to allocate/free TDX private HKID so that TDX KVM can manage it.

Also export the global TDX private HKID that is used to encrypt TDX module, its memory and some dynamic data (TDR). When VMM releasing encrypted page to reuse it, the page needs to be flushed with the used HKID. VMM needs the global TDX private HKID to flush such pages.

---
 arch/x86/include/asm/tdx.h  | 12 ++++++++++++
 arch/x86/virt/vmx/tdx/tdx.c | 34 +++++++++++++++++++++++++++++++++-
 2 files changed, 45 insertions(+), 1 deletion(-)

arch/x86/include/asm/tdx.h +12 -0

index 6c01ab572c1f..7d99a48a98cc 100644
@@ -104,6 +104,16 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,
 bool platform_tdx_enabled(void);
 int tdx_cpu_enable(void);
 int tdx_enable(void);
+/*
+ * Key id globally used by TDX module: TDX module maps TDR with this TDX global
+ * key id.  TDR includes key id assigned to the TD.  Then TDX module maps other
+ * TD-related pages with the assigned key id.  TDR requires this TDX global key
+ * id for cache flush unlike other TD-related pages.
+ */
+extern u32 tdx_global_keyid __ro_after_init;
+int tdx_guest_keyid_alloc(void);
+void tdx_guest_keyid_free(int keyid);
+
 u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
 	       struct tdx_module_output *out);
 #else	/* !CONFIG_INTEL_TDX_HOST */

@@ -112,6 +122,8 @@ static inline int tdx_cpu_enable(void) { return -EINVAL; }
 static inline int tdx_enable(void)  { return -EINVAL; }
 static inline u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
 			     struct tdx_module_output *out) { return TDX_SEAMCALL_UD; };
+static inline int tdx_guest_keyid_alloc(void) { return -EOPNOTSUPP; }
+static inline void tdx_guest_keyid_free(int keyid) { }
 #endif	/* CONFIG_INTEL_TDX_HOST */
 
 #endif /* !__ASSEMBLY__ */

arch/x86/virt/vmx/tdx/tdx.c +33 -1

index ee94a7327d93..3251fdf74667 100644
@@ -33,7 +33,8 @@
 #include <asm/tdx.h>
 #include "tdx.h"
 
-static u32 tdx_global_keyid __ro_after_init;
+u32 tdx_global_keyid __ro_after_init;
+EXPORT_SYMBOL_GPL(tdx_global_keyid);
 static u32 tdx_guest_keyid_start __ro_after_init;
 static u32 tdx_nr_guest_keyids __ro_after_init;

@@ -138,6 +139,31 @@ static struct notifier_block tdx_memory_nb = {
 	.notifier_call = tdx_memory_notifier,
 };
 
+/* TDX KeyID pool */
+static DEFINE_IDA(tdx_guest_keyid_pool);
+
+int tdx_guest_keyid_alloc(void)
+{
+	if (WARN_ON_ONCE(!tdx_guest_keyid_start || !tdx_nr_guest_keyids))
+		return -EINVAL;
+
+	/* The first keyID is reserved for the global key. */
+	return ida_alloc_range(&tdx_guest_keyid_pool, tdx_guest_keyid_start + 1,
+			       tdx_guest_keyid_start + tdx_nr_guest_keyids - 1,
+			       GFP_KERNEL);
+}
+EXPORT_SYMBOL_GPL(tdx_guest_keyid_alloc);
+
+void tdx_guest_keyid_free(int keyid)
+{
+	/* keyid = 0 is reserved. */
+	if (WARN_ON_ONCE(keyid <= 0))
+		return;
+
+	ida_free(&tdx_guest_keyid_pool, keyid);
+}
+EXPORT_SYMBOL_GPL(tdx_guest_keyid_free);
+
 static int __init tdx_init(void)
 {
 	u32 tdx_keyid_start, nr_tdx_keyids;

@@ -1218,6 +1244,12 @@ static int init_tdx_module(void)
 	if (ret)
 		goto out_free_pamts;
 
+	/*
+	 * Reserve the first TDX KeyID as global KeyID to protect
+	 * TDX module metadata.
+	 */
+	tdx_global_keyid = tdx_keyid_start;
+
 	/* Initialize TDMRs to complete the TDX module initialization */
 	ret = init_tdmrs(&tdmr_list);

Signature · 1 line

2.25.1

Zhi Wang

2023-03-24 16:35 UTC #42 lore raw

Re: [PATCH v13 014/113] x86/cpu: Add helper functions to allocate/free TDX private host key id

Quoted 6 lines · On Sun, 12 Mar 2023 10:55:38 -0700 [email protected] wrote:

From: Isaku Yamahata <[email protected]>
TDX private host key id (HKID) is assigned to guest TD. The memory controller encrypts guest TD memory with the assigned TDX HKID. Add helper functions to allocate/free TDX private HKID so that TDX KVM can manage it.

^ This sentence should go to the below paragraph, which is "problem/way to go" paragraph. For "background" paragraph, you can talk more about the usage of HKID, like global/private HKID. Then you don't need to introduce them again in the below paragraph and just talk about the solution.

Quoted 100 lines

---
 arch/x86/include/asm/tdx.h  | 12 ++++++++++++
 arch/x86/virt/vmx/tdx/tdx.c | 34 +++++++++++++++++++++++++++++++++-
 2 files changed, 45 insertions(+), 1 deletion(-)

arch/x86/include/asm/tdx.h +12 -0

index 6c01ab572c1f..7d99a48a98cc 100644
@@ -104,6 +104,16 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,
 bool platform_tdx_enabled(void);
 int tdx_cpu_enable(void);
 int tdx_enable(void);
+/*
+ * Key id globally used by TDX module: TDX module maps TDR with this TDX global
+ * key id.  TDR includes key id assigned to the TD.  Then TDX module maps other
+ * TD-related pages with the assigned key id.  TDR requires this TDX global key
+ * id for cache flush unlike other TD-related pages.
+ */
+extern u32 tdx_global_keyid __ro_after_init;
+int tdx_guest_keyid_alloc(void);
+void tdx_guest_keyid_free(int keyid);
+
 u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
 	       struct tdx_module_output *out);
 #else	/* !CONFIG_INTEL_TDX_HOST */

@@ -112,6 +122,8 @@ static inline int tdx_cpu_enable(void) { return -EINVAL; }
 static inline int tdx_enable(void)  { return -EINVAL; }
 static inline u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
 			     struct tdx_module_output *out) { return TDX_SEAMCALL_UD; };
+static inline int tdx_guest_keyid_alloc(void) { return -EOPNOTSUPP; }
+static inline void tdx_guest_keyid_free(int keyid) { }
 #endif	/* CONFIG_INTEL_TDX_HOST */
 
 #endif /* !__ASSEMBLY__ */

arch/x86/virt/vmx/tdx/tdx.c +33 -1

index ee94a7327d93..3251fdf74667 100644
@@ -33,7 +33,8 @@
 #include <asm/tdx.h>
 #include "tdx.h"
 
-static u32 tdx_global_keyid __ro_after_init;
+u32 tdx_global_keyid __ro_after_init;
+EXPORT_SYMBOL_GPL(tdx_global_keyid);
 static u32 tdx_guest_keyid_start __ro_after_init;
 static u32 tdx_nr_guest_keyids __ro_after_init;

@@ -138,6 +139,31 @@ static struct notifier_block tdx_memory_nb = {
 	.notifier_call = tdx_memory_notifier,
 };
 
+/* TDX KeyID pool */
+static DEFINE_IDA(tdx_guest_keyid_pool);
+
+int tdx_guest_keyid_alloc(void)
+{
+	if (WARN_ON_ONCE(!tdx_guest_keyid_start || !tdx_nr_guest_keyids))
+		return -EINVAL;
+
+	/* The first keyID is reserved for the global key. */
+	return ida_alloc_range(&tdx_guest_keyid_pool, tdx_guest_keyid_start + 1,
+			       tdx_guest_keyid_start + tdx_nr_guest_keyids - 1,
+			       GFP_KERNEL);
+}
+EXPORT_SYMBOL_GPL(tdx_guest_keyid_alloc);
+
+void tdx_guest_keyid_free(int keyid)
+{
+	/* keyid = 0 is reserved. */
+	if (WARN_ON_ONCE(keyid <= 0))
+		return;
+
+	ida_free(&tdx_guest_keyid_pool, keyid);
+}
+EXPORT_SYMBOL_GPL(tdx_guest_keyid_free);
+
 static int __init tdx_init(void)
 {
 	u32 tdx_keyid_start, nr_tdx_keyids;

@@ -1218,6 +1244,12 @@ static int init_tdx_module(void)
 	if (ret)
 		goto out_free_pamts;
 
+	/*
+	 * Reserve the first TDX KeyID as global KeyID to protect
+	 * TDX module metadata.
+	 */
+	tdx_global_keyid = tdx_keyid_start;
+
 	/* Initialize TDMRs to complete the TDX module initialization */
 	ret = init_tdmrs(&tdmr_list);

[email protected]

author

2023-03-12 17:59 UTC #43 lore raw

PATCH v13 15/113 x86/virt/tdx: Add a helper function to return system wide info about TDX module

From: Isaku Yamahata <[email protected]>

TDX KVM needs system-wide information about the TDX module, struct tdsysinfo_struct. Add a helper function tdx_get_sysinfo() to return it instead of KVM getting it with various error checks. Make KVM call the function and stash the info. Move out the struct definition about it to common place arch/x86/include/asm/tdx.h.

---
 arch/x86/include/asm/tdx.h  | 54 +++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.c      | 15 ++++++++++-
 arch/x86/virt/vmx/tdx/tdx.c | 21 ++++++++++++---
 arch/x86/virt/vmx/tdx/tdx.h | 51 -----------------------------------
 4 files changed, 85 insertions(+), 56 deletions(-)

arch/x86/include/asm/tdx.h +54 -0

index 7d99a48a98cc..05870e5ed131 100644
@@ -101,6 +101,58 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,
 #endif /* CONFIG_INTEL_TDX_GUEST && CONFIG_KVM_GUEST */
 
 #ifdef CONFIG_INTEL_TDX_HOST
+struct tdx_cpuid_config {
+	u32	leaf;
+	u32	sub_leaf;
+	u32	eax;
+	u32	ebx;
+	u32	ecx;
+	u32	edx;
+} __packed;
+
+#define TDSYSINFO_STRUCT_SIZE		1024
+#define TDSYSINFO_STRUCT_ALIGNMENT	1024
+
+/*
+ * The size of this structure itself is flexible.  The actual structure
+ * passed to TDH.SYS.INFO must be padded to TDSYSINFO_STRUCT_SIZE and be
+ * aligned to TDSYSINFO_STRUCT_ALIGNMENT using DECLARE_PADDED_STRUCT().
+ */
+struct tdsysinfo_struct {
+	/* TDX-SEAM Module Info */
+	u32	attributes;
+	u32	vendor_id;
+	u32	build_date;
+	u16	build_num;
+	u16	minor_version;
+	u16	major_version;
+	u8	reserved0[14];
+	/* Memory Info */
+	u16	max_tdmrs;
+	u16	max_reserved_per_tdmr;
+	u16	pamt_entry_size;
+	u8	reserved1[10];
+	/* Control Struct Info */
+	u16	tdcs_base_size;
+	u8	reserved2[2];
+	u16	tdvps_base_size;
+	u8	tdvps_xfam_dependent_size;
+	u8	reserved3[9];
+	/* TD Capabilities */
+	u64	attributes_fixed0;
+	u64	attributes_fixed1;
+	u64	xfam_fixed0;
+	u64	xfam_fixed1;
+	u8	reserved4[32];
+	u32	num_cpuid_config;
+	/*
+	 * The actual number of CPUID_CONFIG depends on above
+	 * 'num_cpuid_config'.
+	 */
+	DECLARE_FLEX_ARRAY(struct tdx_cpuid_config, cpuid_configs);
+} __packed;
+
+const struct tdsysinfo_struct *tdx_get_sysinfo(void);
 bool platform_tdx_enabled(void);
 int tdx_cpu_enable(void);
 int tdx_enable(void);

@@ -117,6 +169,8 @@ void tdx_guest_keyid_free(int keyid);
 u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9,
 	       struct tdx_module_output *out);
 #else	/* !CONFIG_INTEL_TDX_HOST */
+struct tdsysinfo_struct;
+static inline const struct tdsysinfo_struct *tdx_get_sysinfo(void) { return NULL; }
 static inline bool platform_tdx_enabled(void) { return false; }
 static inline int tdx_cpu_enable(void) { return -EINVAL; }
 static inline int tdx_enable(void)  { return -EINVAL; }

arch/x86/kvm/vmx/tdx.c +14 -1

index 36738904c26a..e9b7aa5654e9 100644
@@ -11,6 +11,11 @@
 #undef pr_fmt
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
+#define TDX_MAX_NR_CPUID_CONFIGS					\
+	((TDSYSINFO_STRUCT_SIZE -					\
+		offsetof(struct tdsysinfo_struct, cpuid_configs))	\
+		/ sizeof(struct tdx_cpuid_config))
+
 int tdx_hardware_enable(void)
 {
 	return tdx_cpu_enable();

@@ -18,7 +23,11 @@ int tdx_hardware_enable(void)
 
 static int __init tdx_module_setup(void)
 {
-	int ret;
+	const struct tdsysinfo_struct *tdsysinfo;
+	int ret = 0;
+
+	BUILD_BUG_ON(sizeof(*tdsysinfo) > TDSYSINFO_STRUCT_SIZE);
+	BUILD_BUG_ON(TDX_MAX_NR_CPUID_CONFIGS != 37);
 
 	ret = tdx_enable();
 	if (ret) {

@@ -26,6 +35,10 @@ static int __init tdx_module_setup(void)
 		return ret;
 	}
 
+	/* Sanitary check just in case. */
+	tdsysinfo = tdx_get_sysinfo();
+	WARN_ON(tdsysinfo->num_cpuid_config > TDX_MAX_NR_CPUID_CONFIGS);
+
 	pr_info("TDX is supported.\n");
 	return 0;
 }

arch/x86/virt/vmx/tdx/tdx.c +17 -4

index 3251fdf74667..f9f9c1b76501 100644
@@ -385,7 +385,7 @@ static void print_cmrs(struct cmr_info *cmr_array, int nr_cmrs)
  * CMRs, and save them to @sysinfo and @cmr_array.  @sysinfo must have
  * been padded to have enough room to save the TDSYSINFO_STRUCT.
  */
-static int tdx_get_sysinfo(struct tdsysinfo_struct *sysinfo,
+static int __tdx_get_sysinfo(struct tdsysinfo_struct *sysinfo,
 			   struct cmr_info *cmr_array)
 {
 	struct tdx_module_output out;

@@ -410,6 +410,21 @@ static int tdx_get_sysinfo(struct tdsysinfo_struct *sysinfo,
 	return 0;
 }
 
+static DECLARE_PADDED_STRUCT(tdsysinfo_struct, tdsysinfo,
+			     TDSYSINFO_STRUCT_SIZE, TDSYSINFO_STRUCT_ALIGNMENT);
+
+const struct tdsysinfo_struct *tdx_get_sysinfo(void)
+{
+	const struct tdsysinfo_struct *r = NULL;
+
+	mutex_lock(&tdx_module_lock);
+	if (tdx_module_status == TDX_MODULE_INITIALIZED)
+		r = &PADDED_STRUCT(tdsysinfo);
+	mutex_unlock(&tdx_module_lock);
+	return r;
+}
+EXPORT_SYMBOL_GPL(tdx_get_sysinfo);
+
 /*
  * Add a memory region as a TDX memory block.  The caller must make sure
  * all memory regions are added in address ascending order and don't

@@ -1186,15 +1201,13 @@ static int init_tdmrs(struct tdmr_info_list *tdmr_list)
 
 static int init_tdx_module(void)
 {
-	static DECLARE_PADDED_STRUCT(tdsysinfo_struct, tdsysinfo,
-			TDSYSINFO_STRUCT_SIZE, TDSYSINFO_STRUCT_ALIGNMENT);
 	static struct cmr_info cmr_array[MAX_CMRS]
 			__aligned(CMR_INFO_ARRAY_ALIGNMENT);
 	struct tdsysinfo_struct *sysinfo = &PADDED_STRUCT(tdsysinfo);
 	struct tdmr_info_list tdmr_list;
 	int ret;
 
-	ret = tdx_get_sysinfo(sysinfo, cmr_array);
+	ret = __tdx_get_sysinfo(sysinfo, cmr_array);
 	if (ret)
 		return ret;

arch/x86/virt/vmx/tdx/tdx.h +0 ~~-51~~

63 lines not on this page · raw message

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 17:59 UTC #44 lore raw

PATCH v13 16/113 KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

From: Sean Christopherson <[email protected]>

Implement a system-scoped ioctl to get system-wide parameters for TDX.

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/include/asm/kvm-x86-ops.h    |  1 +
 arch/x86/include/asm/kvm_host.h       |  1 +
 arch/x86/include/uapi/asm/kvm.h       | 48 +++++++++++++++++++++++++
 arch/x86/kvm/vmx/main.c               |  2 ++
 arch/x86/kvm/vmx/tdx.c                | 51 +++++++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h            |  2 ++
 arch/x86/kvm/x86.c                    |  6 ++++
 tools/arch/x86/include/uapi/asm/kvm.h | 48 +++++++++++++++++++++++++
 8 files changed, 159 insertions(+)

arch/x86/include/asm/kvm-x86-ops.h +1 -0

index eac4b65d1b01..b46dcac078b2 100644
@@ -117,6 +117,7 @@ KVM_X86_OP(enter_smm)
 KVM_X86_OP(leave_smm)
 KVM_X86_OP(enable_smi_window)
 #endif
+KVM_X86_OP_OPTIONAL(dev_mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_register_region)
 KVM_X86_OP_OPTIONAL(mem_enc_unregister_region)

arch/x86/include/asm/kvm_host.h +1 -0

index 00c25f6ab871..49e3ca89aced 100644
@@ -1719,6 +1719,7 @@ struct kvm_x86_ops {
 	void (*enable_smi_window)(struct kvm_vcpu *vcpu);
 #endif
 
+	int (*dev_mem_enc_ioctl)(void __user *argp);
 	int (*mem_enc_ioctl)(struct kvm *kvm, void __user *argp);
 	int (*mem_enc_register_region)(struct kvm *kvm, struct kvm_enc_region *argp);
 	int (*mem_enc_unregister_region)(struct kvm *kvm, struct kvm_enc_region *argp);

arch/x86/include/uapi/asm/kvm.h +48 -0

52 lines not on this page · raw message

arch/x86/kvm/vmx/main.c +2 -0

index c8548004802a..6a5d0c7a2950 100644
@@ -201,6 +201,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.complete_emulated_msr = kvm_complete_insn_gp,
 
 	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
+
+	.dev_mem_enc_ioctl = tdx_dev_ioctl,
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {

arch/x86/kvm/vmx/tdx.c +51 -0

57 lines not on this page · raw message

arch/x86/kvm/vmx/x86_ops.h +2 -0

index b2c74c1b5bbd..78c5537e23a1 100644
@@ -141,10 +141,12 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
+int tdx_dev_ioctl(void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
+static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

arch/x86/kvm/x86.c +6 -0

index 27ab684f8374..a3dc32e33aca 100644
@@ -4718,6 +4718,12 @@ long kvm_arch_dev_ioctl(struct file *filp,
 		r = kvm_x86_dev_has_attr(&attr);
 		break;
 	}
+	case KVM_MEMORY_ENCRYPT_OP:
+		r = -EINVAL;
+		if (!kvm_x86_ops.dev_mem_enc_ioctl)
+			goto out;
+		r = static_call(kvm_x86_dev_mem_enc_ioctl)(argp);
+		break;
 	default:
 		r = -EINVAL;
 		break;

tools/arch/x86/include/uapi/asm/kvm.h +48 -0

52 lines not on this page · raw message

Signature · 1 line

2.25.1

Zhi Wang

2023-03-25 08:43 UTC #45 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:

Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.

Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?

Quoted 264 lines

From: Sean Christopherson <[email protected]>

Implement a system-scoped ioctl to get system-wide parameters for TDX.

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/include/asm/kvm-x86-ops.h    |  1 +
 arch/x86/include/asm/kvm_host.h       |  1 +
 arch/x86/include/uapi/asm/kvm.h       | 48 +++++++++++++++++++++++++
 arch/x86/kvm/vmx/main.c               |  2 ++
 arch/x86/kvm/vmx/tdx.c                | 51 +++++++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h            |  2 ++
 arch/x86/kvm/x86.c                    |  6 ++++
 tools/arch/x86/include/uapi/asm/kvm.h | 48 +++++++++++++++++++++++++
 8 files changed, 159 insertions(+)

arch/x86/include/asm/kvm-x86-ops.h +1 -0

index eac4b65d1b01..b46dcac078b2 100644
@@ -117,6 +117,7 @@ KVM_X86_OP(enter_smm)
 KVM_X86_OP(leave_smm)
 KVM_X86_OP(enable_smi_window)
 #endif
+KVM_X86_OP_OPTIONAL(dev_mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_register_region)
 KVM_X86_OP_OPTIONAL(mem_enc_unregister_region)

arch/x86/include/asm/kvm_host.h +1 -0

index 00c25f6ab871..49e3ca89aced 100644
@@ -1719,6 +1719,7 @@ struct kvm_x86_ops {
 	void (*enable_smi_window)(struct kvm_vcpu *vcpu);
 #endif
 
+	int (*dev_mem_enc_ioctl)(void __user *argp);
 	int (*mem_enc_ioctl)(struct kvm *kvm, void __user *argp);
 	int (*mem_enc_register_region)(struct kvm *kvm, struct kvm_enc_region *argp);
 	int (*mem_enc_unregister_region)(struct kvm *kvm, struct kvm_enc_region *argp);

arch/x86/include/uapi/asm/kvm.h +48 -0

index 6afbfbb32d56..af4c5bd0af1c 100644
@@ -562,4 +562,52 @@ struct kvm_pmu_event_filter {
 #define KVM_X86_DEFAULT_VM	0
 #define KVM_X86_PROTECTED_VM	1
 
+/* Trust Domain eXtension sub-ioctl() commands. */
+enum kvm_tdx_cmd_id {
+	KVM_TDX_CAPABILITIES = 0,
+
+	KVM_TDX_CMD_NR_MAX,
+};
+
+struct kvm_tdx_cmd {
+	/* enum kvm_tdx_cmd_id */
+	__u32 id;
+	/* flags for sub-commend. If sub-command doesn't use this, set zero. */
+	__u32 flags;
+	/*
+	 * data for each sub-command. An immediate or a pointer to the actual
+	 * data in process virtual address.  If sub-command doesn't use it,
+	 * set zero.
+	 */
+	__u64 data;
+	/*
+	 * Auxiliary error code.  The sub-command may return TDX SEAMCALL
+	 * status code in addition to -Exxx.
+	 * Defined for consistency with struct kvm_sev_cmd.
+	 */
+	__u64 error;
+	/* Reserved: Defined for consistency with struct kvm_sev_cmd. */
+	__u64 unused;
+};
+
+struct kvm_tdx_cpuid_config {
+	__u32 leaf;
+	__u32 sub_leaf;
+	__u32 eax;
+	__u32 ebx;
+	__u32 ecx;
+	__u32 edx;
+};
+
+struct kvm_tdx_capabilities {
+	__u64 attrs_fixed0;
+	__u64 attrs_fixed1;
+	__u64 xfam_fixed0;
+	__u64 xfam_fixed1;
+
+	__u32 nr_cpuid_configs;
+	__u32 padding;
+	struct kvm_tdx_cpuid_config cpuid_configs[0];
+};
+
 #endif /* _ASM_X86_KVM_H */

arch/x86/kvm/vmx/main.c +2 -0

index c8548004802a..6a5d0c7a2950 100644
@@ -201,6 +201,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.complete_emulated_msr = kvm_complete_insn_gp,
 
 	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
+
+	.dev_mem_enc_ioctl = tdx_dev_ioctl,
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {

arch/x86/kvm/vmx/tdx.c +51 -0

index e9b7aa5654e9..b59d3081d061 100644
@@ -21,6 +21,57 @@ int tdx_hardware_enable(void)
 	return tdx_cpu_enable();
 }
 
+int tdx_dev_ioctl(void __user *argp)
+{
+	struct kvm_tdx_capabilities __user *user_caps;
+	const struct tdsysinfo_struct *tdsysinfo;
+	struct kvm_tdx_capabilities caps;
+	struct kvm_tdx_cmd cmd;
+
+	BUILD_BUG_ON(sizeof(struct kvm_tdx_cpuid_config) !=
+		     sizeof(struct tdx_cpuid_config));
+
+	if (copy_from_user(&cmd, argp, sizeof(cmd)))
+		return -EFAULT;
+	if (cmd.flags || cmd.error || cmd.unused)
+		return -EINVAL;
+	/*
+	 * Currently only KVM_TDX_CAPABILITIES is defined for system-scoped
+	 * mem_enc_ioctl().
+	 */
+	if (cmd.id != KVM_TDX_CAPABILITIES)
+		return -EINVAL;
+
+	tdsysinfo = tdx_get_sysinfo();
+	if (!tdsysinfo)
+		return -ENOTSUPP;
+
+	user_caps = (void __user *)cmd.data;
+	if (copy_from_user(&caps, user_caps, sizeof(caps)))
+		return -EFAULT;
+
+	if (caps.nr_cpuid_configs < tdsysinfo->num_cpuid_config)
+		return -E2BIG;
+
+	caps = (struct kvm_tdx_capabilities) {
+		.attrs_fixed0 = tdsysinfo->attributes_fixed0,
+		.attrs_fixed1 = tdsysinfo->attributes_fixed1,
+		.xfam_fixed0 = tdsysinfo->xfam_fixed0,
+		.xfam_fixed1 = tdsysinfo->xfam_fixed1,
+		.nr_cpuid_configs = tdsysinfo->num_cpuid_config,
+		.padding = 0,
+	};
+
+	if (copy_to_user(user_caps, &caps, sizeof(caps)))
+		return -EFAULT;
+	if (copy_to_user(user_caps->cpuid_configs, &tdsysinfo->cpuid_configs,
+			 tdsysinfo->num_cpuid_config *
+			 sizeof(struct tdx_cpuid_config)))
+		return -EFAULT;
+
+	return 0;
+}
+
 static int __init tdx_module_setup(void)
 {
 	const struct tdsysinfo_struct *tdsysinfo;

arch/x86/kvm/vmx/x86_ops.h +2 -0

index b2c74c1b5bbd..78c5537e23a1 100644
@@ -141,10 +141,12 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
+int tdx_dev_ioctl(void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
+static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

arch/x86/kvm/x86.c +6 -0

index 27ab684f8374..a3dc32e33aca 100644
@@ -4718,6 +4718,12 @@ long kvm_arch_dev_ioctl(struct file *filp,
 		r = kvm_x86_dev_has_attr(&attr);
 		break;
 	}
+	case KVM_MEMORY_ENCRYPT_OP:
+		r = -EINVAL;
+		if (!kvm_x86_ops.dev_mem_enc_ioctl)
+			goto out;
+		r = static_call(kvm_x86_dev_mem_enc_ioctl)(argp);
+		break;
 	default:
 		r = -EINVAL;
 		break;

tools/arch/x86/include/uapi/asm/kvm.h +48 -0

index 6afbfbb32d56..af4c5bd0af1c 100644
@@ -562,4 +562,52 @@ struct kvm_pmu_event_filter {
 #define KVM_X86_DEFAULT_VM	0
 #define KVM_X86_PROTECTED_VM	1
 
+/* Trust Domain eXtension sub-ioctl() commands. */
+enum kvm_tdx_cmd_id {
+	KVM_TDX_CAPABILITIES = 0,
+
+	KVM_TDX_CMD_NR_MAX,
+};
+
+struct kvm_tdx_cmd {
+	/* enum kvm_tdx_cmd_id */
+	__u32 id;
+	/* flags for sub-commend. If sub-command doesn't use this, set zero. */
+	__u32 flags;
+	/*
+	 * data for each sub-command. An immediate or a pointer to the actual
+	 * data in process virtual address.  If sub-command doesn't use it,
+	 * set zero.
+	 */
+	__u64 data;
+	/*
+	 * Auxiliary error code.  The sub-command may return TDX SEAMCALL
+	 * status code in addition to -Exxx.
+	 * Defined for consistency with struct kvm_sev_cmd.
+	 */
+	__u64 error;
+	/* Reserved: Defined for consistency with struct kvm_sev_cmd. */
+	__u64 unused;
+};
+
+struct kvm_tdx_cpuid_config {
+	__u32 leaf;
+	__u32 sub_leaf;
+	__u32 eax;
+	__u32 ebx;
+	__u32 ecx;
+	__u32 edx;
+};
+
+struct kvm_tdx_capabilities {
+	__u64 attrs_fixed0;
+	__u64 attrs_fixed1;
+	__u64 xfam_fixed0;
+	__u64 xfam_fixed1;
+
+	__u32 nr_cpuid_configs;
+	__u32 padding;
+	struct kvm_tdx_cpuid_config cpuid_configs[0];
+};
+
 #endif /* _ASM_X86_KVM_H */

Isaku Yamahata

2023-03-29 23:17 UTC #46 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

Quoted 11 lines · Zhi Wang

On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:
Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.
Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?

We only need global parameters of the TDX module, and we don't interact with TDX module at this point. One alternative is to export those parameters via sysfs. Also the existence of the sysfs node indicates that the TDX module is loaded(initialized?) or not in addition to boot log. Thus we can drop system scope one.
What do you think?

Regarding to other TDX KVM specific ioctls (KVM_TDX_INIT_VM, KVM_TDX_INIT_VCPU, KVM_TDX_INIT_MEM_REGION, and KVM_TDX_FINALIZE_VM), they are specific to KVM. So I don't think it can be split out to independent driver.

Thanks,

Quoted 265 lines

Quoted 264 lines

From: Sean Christopherson <[email protected]>

Implement a system-scoped ioctl to get system-wide parameters for TDX.

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/include/asm/kvm-x86-ops.h    |  1 +
 arch/x86/include/asm/kvm_host.h       |  1 +
 arch/x86/include/uapi/asm/kvm.h       | 48 +++++++++++++++++++++++++
 arch/x86/kvm/vmx/main.c               |  2 ++
 arch/x86/kvm/vmx/tdx.c                | 51 +++++++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h            |  2 ++
 arch/x86/kvm/x86.c                    |  6 ++++
 tools/arch/x86/include/uapi/asm/kvm.h | 48 +++++++++++++++++++++++++
 8 files changed, 159 insertions(+)

arch/x86/include/asm/kvm-x86-ops.h +1 -0

index eac4b65d1b01..b46dcac078b2 100644
@@ -117,6 +117,7 @@ KVM_X86_OP(enter_smm)
 KVM_X86_OP(leave_smm)
 KVM_X86_OP(enable_smi_window)
 #endif
+KVM_X86_OP_OPTIONAL(dev_mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_register_region)
 KVM_X86_OP_OPTIONAL(mem_enc_unregister_region)

arch/x86/include/asm/kvm_host.h +1 -0

index 00c25f6ab871..49e3ca89aced 100644
@@ -1719,6 +1719,7 @@ struct kvm_x86_ops {
 	void (*enable_smi_window)(struct kvm_vcpu *vcpu);
 #endif
 
+	int (*dev_mem_enc_ioctl)(void __user *argp);
 	int (*mem_enc_ioctl)(struct kvm *kvm, void __user *argp);
 	int (*mem_enc_register_region)(struct kvm *kvm, struct kvm_enc_region *argp);
 	int (*mem_enc_unregister_region)(struct kvm *kvm, struct kvm_enc_region *argp);

arch/x86/include/uapi/asm/kvm.h +48 -0

index 6afbfbb32d56..af4c5bd0af1c 100644
@@ -562,4 +562,52 @@ struct kvm_pmu_event_filter {
 #define KVM_X86_DEFAULT_VM	0
 #define KVM_X86_PROTECTED_VM	1
 
+/* Trust Domain eXtension sub-ioctl() commands. */
+enum kvm_tdx_cmd_id {
+	KVM_TDX_CAPABILITIES = 0,
+
+	KVM_TDX_CMD_NR_MAX,
+};
+
+struct kvm_tdx_cmd {
+	/* enum kvm_tdx_cmd_id */
+	__u32 id;
+	/* flags for sub-commend. If sub-command doesn't use this, set zero. */
+	__u32 flags;
+	/*
+	 * data for each sub-command. An immediate or a pointer to the actual
+	 * data in process virtual address.  If sub-command doesn't use it,
+	 * set zero.
+	 */
+	__u64 data;
+	/*
+	 * Auxiliary error code.  The sub-command may return TDX SEAMCALL
+	 * status code in addition to -Exxx.
+	 * Defined for consistency with struct kvm_sev_cmd.
+	 */
+	__u64 error;
+	/* Reserved: Defined for consistency with struct kvm_sev_cmd. */
+	__u64 unused;
+};
+
+struct kvm_tdx_cpuid_config {
+	__u32 leaf;
+	__u32 sub_leaf;
+	__u32 eax;
+	__u32 ebx;
+	__u32 ecx;
+	__u32 edx;
+};
+
+struct kvm_tdx_capabilities {
+	__u64 attrs_fixed0;
+	__u64 attrs_fixed1;
+	__u64 xfam_fixed0;
+	__u64 xfam_fixed1;
+
+	__u32 nr_cpuid_configs;
+	__u32 padding;
+	struct kvm_tdx_cpuid_config cpuid_configs[0];
+};
+
 #endif /* _ASM_X86_KVM_H */

arch/x86/kvm/vmx/main.c +2 -0

index c8548004802a..6a5d0c7a2950 100644
@@ -201,6 +201,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.complete_emulated_msr = kvm_complete_insn_gp,
 
 	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
+
+	.dev_mem_enc_ioctl = tdx_dev_ioctl,
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {

arch/x86/kvm/vmx/tdx.c +51 -0

index e9b7aa5654e9..b59d3081d061 100644
@@ -21,6 +21,57 @@ int tdx_hardware_enable(void)
 	return tdx_cpu_enable();
 }
 
+int tdx_dev_ioctl(void __user *argp)
+{
+	struct kvm_tdx_capabilities __user *user_caps;
+	const struct tdsysinfo_struct *tdsysinfo;
+	struct kvm_tdx_capabilities caps;
+	struct kvm_tdx_cmd cmd;
+
+	BUILD_BUG_ON(sizeof(struct kvm_tdx_cpuid_config) !=
+		     sizeof(struct tdx_cpuid_config));
+
+	if (copy_from_user(&cmd, argp, sizeof(cmd)))
+		return -EFAULT;
+	if (cmd.flags || cmd.error || cmd.unused)
+		return -EINVAL;
+	/*
+	 * Currently only KVM_TDX_CAPABILITIES is defined for system-scoped
+	 * mem_enc_ioctl().
+	 */
+	if (cmd.id != KVM_TDX_CAPABILITIES)
+		return -EINVAL;
+
+	tdsysinfo = tdx_get_sysinfo();
+	if (!tdsysinfo)
+		return -ENOTSUPP;
+
+	user_caps = (void __user *)cmd.data;
+	if (copy_from_user(&caps, user_caps, sizeof(caps)))
+		return -EFAULT;
+
+	if (caps.nr_cpuid_configs < tdsysinfo->num_cpuid_config)
+		return -E2BIG;
+
+	caps = (struct kvm_tdx_capabilities) {
+		.attrs_fixed0 = tdsysinfo->attributes_fixed0,
+		.attrs_fixed1 = tdsysinfo->attributes_fixed1,
+		.xfam_fixed0 = tdsysinfo->xfam_fixed0,
+		.xfam_fixed1 = tdsysinfo->xfam_fixed1,
+		.nr_cpuid_configs = tdsysinfo->num_cpuid_config,
+		.padding = 0,
+	};
+
+	if (copy_to_user(user_caps, &caps, sizeof(caps)))
+		return -EFAULT;
+	if (copy_to_user(user_caps->cpuid_configs, &tdsysinfo->cpuid_configs,
+			 tdsysinfo->num_cpuid_config *
+			 sizeof(struct tdx_cpuid_config)))
+		return -EFAULT;
+
+	return 0;
+}
+
 static int __init tdx_module_setup(void)
 {
 	const struct tdsysinfo_struct *tdsysinfo;

arch/x86/kvm/vmx/x86_ops.h +2 -0

index b2c74c1b5bbd..78c5537e23a1 100644
@@ -141,10 +141,12 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
+int tdx_dev_ioctl(void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
+static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

arch/x86/kvm/x86.c +6 -0

index 27ab684f8374..a3dc32e33aca 100644
@@ -4718,6 +4718,12 @@ long kvm_arch_dev_ioctl(struct file *filp,
 		r = kvm_x86_dev_has_attr(&attr);
 		break;
 	}
+	case KVM_MEMORY_ENCRYPT_OP:
+		r = -EINVAL;
+		if (!kvm_x86_ops.dev_mem_enc_ioctl)
+			goto out;
+		r = static_call(kvm_x86_dev_mem_enc_ioctl)(argp);
+		break;
 	default:
 		r = -EINVAL;
 		break;

tools/arch/x86/include/uapi/asm/kvm.h +48 -0

index 6afbfbb32d56..af4c5bd0af1c 100644
@@ -562,4 +562,52 @@ struct kvm_pmu_event_filter {
 #define KVM_X86_DEFAULT_VM	0
 #define KVM_X86_PROTECTED_VM	1
 
+/* Trust Domain eXtension sub-ioctl() commands. */
+enum kvm_tdx_cmd_id {
+	KVM_TDX_CAPABILITIES = 0,
+
+	KVM_TDX_CMD_NR_MAX,
+};
+
+struct kvm_tdx_cmd {
+	/* enum kvm_tdx_cmd_id */
+	__u32 id;
+	/* flags for sub-commend. If sub-command doesn't use this, set zero. */
+	__u32 flags;
+	/*
+	 * data for each sub-command. An immediate or a pointer to the actual
+	 * data in process virtual address.  If sub-command doesn't use it,
+	 * set zero.
+	 */
+	__u64 data;
+	/*
+	 * Auxiliary error code.  The sub-command may return TDX SEAMCALL
+	 * status code in addition to -Exxx.
+	 * Defined for consistency with struct kvm_sev_cmd.
+	 */
+	__u64 error;
+	/* Reserved: Defined for consistency with struct kvm_sev_cmd. */
+	__u64 unused;
+};
+
+struct kvm_tdx_cpuid_config {
+	__u32 leaf;
+	__u32 sub_leaf;
+	__u32 eax;
+	__u32 ebx;
+	__u32 ecx;
+	__u32 edx;
+};
+
+struct kvm_tdx_capabilities {
+	__u64 attrs_fixed0;
+	__u64 attrs_fixed1;
+	__u64 xfam_fixed0;
+	__u64 xfam_fixed1;
+
+	__u32 nr_cpuid_configs;
+	__u32 padding;
+	struct kvm_tdx_cpuid_config cpuid_configs[0];
+};
+
 #endif /* _ASM_X86_KVM_H */

Signature · 1 line

Isaku Yamahata <[email protected]>

Isaku Yamahata

2023-03-31 00:20 UTC #47 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

Quoted 22 lines · Isaku Yamahata

Quoted 11 lines · Zhi Wang
On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:
Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.
Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?
We only need global parameters of the TDX module, and we don't interact with TDX module at this point. One alternative is to export those parameters via sysfs. Also the existence of the sysfs node indicates that the TDX module is loaded(initialized?) or not in addition to boot log. Thus we can drop system scope one.
What do you think?

Quoted 3 lines

Regarding to other TDX KVM specific ioctls (KVM_TDX_INIT_VM, KVM_TDX_INIT_VCPU, KVM_TDX_INIT_MEM_REGION, and KVM_TDX_FINALIZE_VM), they are specific to KVM. So I don't think it can be split out to independent driver.

Here is the patch to export those info via sysfs.

From e0744e506eb92e47d8317e489945a3ba804edfa7 Mon Sep 17 00:00:00 2001
Message-Id: <e0744e506eb92e47d8317e489945a3ba804edfa7.1680221730.git.isaku.yamahata@intel.com>
In-Reply-To: <8e0bc0e8e5d435f54f10c7642a862629ef2acb89.1680221729.git.isaku.yamahata@intel.com>
References: <8e0bc0e8e5d435f54f10c7642a862629ef2acb89.1680221729.git.isaku.yamahata@intel.com>
From: Isaku Yamahata <[email protected]>
Date: Thu, 30 Mar 2023 00:05:03 -0700
Subject: [PATCH] x86/virt/tdx: Export TD config params of TDX module via sysfs

TDX module has parameters for VMM to configure TD. User space VMM, e.g. qemu, needs to know it. Export them to user space via sysfs.

TDX 1.0 provides TDH.SYS.INFO to provide system information in TDSYSINFO_STRUCT. Its future extensibility is limited because of its struct. From TDX 1.5, TDH.SYS.RD(metadata field_id) to read the info specified by field id. So instead of exporting TDSYSINFO_STRUCT, adapt metadata way to export those system information.

---
 Documentation/ABI/testing/sysfs-firmware-tdx |  23 +++
 arch/x86/include/asm/tdx.h                   |  33 ++++
 arch/x86/virt/vmx/tdx/tdx.c                  | 164 +++++++++++++++++++
 arch/x86/virt/vmx/tdx/tdx.h                  |  18 ++
 4 files changed, 238 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-firmware-tdx

Documentation/ABI/testing/sysfs-firmware-tdx new file +23 -0

new file mode 100644
index 000000000000..1f26fb178144
@@ -0,0 +1,23 @@
+What:           /sys/firmware/tdx/tdx_module/metadata
+Date:           March 2023
+KernelVersion:  6.3
+Contact:        Isaku Yamahata <[email protected]>, [email protected]
+Users:          qemu, libvirt
+Description:
+                The TDX feature requires a firmware that is known as the TDX
+                module.  The TDX module exposes its metadata in the following
+                read-only files.  The information corresponds to the TDX global
+                metadata specified by 64bit field id.  The file name is hex
+                string in lower case.  The value is binary.
+                User space VMM like qemu needs refer to them to determine what
+                parameters are needed or allowed to configure guest TDs.
+
+                ================== ============================================
+                1900000300000000   ATTRIBUTES_FIXED0
+                1900000300000001   ATTRIBUTES_FIXED1
+                1900000300000002   XFAM_FIXED0
+                1900000300000003   XFAM_FIXED1
+                9900000100000004   NUM_CPUID_CONFIG
+                9900000300000400   CPUID_LEAVES
+                9900000300000500   CPUID_VALUES
+                ================== ============================================
\ No newline at end of file

arch/x86/include/asm/tdx.h +33 -0

index 05870e5ed131..c650ac22a916 100644
@@ -110,6 +110,39 @@ struct tdx_cpuid_config {
 	u32	edx;
 } __packed;
 
+struct tdx_cpuid_config_leaf {
+	u32	leaf;
+	u32	sub_leaf;
+} __packed;
+static_assert(offsetof(struct tdx_cpuid_config, leaf) ==
+	      offsetof(struct tdx_cpuid_config_leaf, leaf));
+static_assert(offsetof(struct tdx_cpuid_config, sub_leaf) ==
+	      offsetof(struct tdx_cpuid_config_leaf, sub_leaf));
+static_assert(offsetofend(struct tdx_cpuid_config, sub_leaf) ==
+	      sizeof(struct tdx_cpuid_config_leaf));
+
+struct tdx_cpuid_config_value {
+	u32	eax;
+	u32	ebx;
+	u32	ecx;
+	u32	edx;
+} __packed;
+static_assert(offsetof(struct tdx_cpuid_config, eax) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, eax));
+static_assert(offsetof(struct tdx_cpuid_config, ebx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, ebx));
+static_assert(offsetof(struct tdx_cpuid_config, ecx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, ecx));
+static_assert(offsetof(struct tdx_cpuid_config, edx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, edx));
+static_assert(offsetofend(struct tdx_cpuid_config, edx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      sizeof(struct tdx_cpuid_config_value));
+
 #define TDSYSINFO_STRUCT_SIZE		1024
 #define TDSYSINFO_STRUCT_ALIGNMENT	1024

arch/x86/virt/vmx/tdx/tdx.c +164 -0

179 lines not on this page · raw message

arch/x86/virt/vmx/tdx/tdx.h +18 -0

index db0cbcceb5b3..a48f38fe6cc4 100644
@@ -93,4 +93,22 @@ struct tdmr_info_list {
 	int max_tdmrs;	/* How many 'tdmr_info's are allocated */
 };
 
+/* TDX metadata base field id. */
+#define TDX_METADATA_ATTRIBUTES_FIXED0		0x1900000300000000ULL
+#define TDX_METADATA_ATTRIBUTES_FIXED1		0x1900000300000001ULL
+#define TDX_METADATA_XFAM_FIXED0		0x1900000300000002ULL
+#define TDX_METADATA_XFAM_FIXED1		0x1900000300000003ULL
+#define TDX_METADATA_NUM_CPUID_CONFIG		0x9900000100000004ULL
+#define TDX_METADATA_CPUID_LEAVES		0x9900000300000400ULL
+#define TDX_METADATA_CPUID_VALUES		0x9900000300000500ULL
+
+/* File name for sysfs: hex with lower case. */
+#define TDX_METADATA_ATTRIBUTES_FIXED0_NAME	"1900000300000000"
+#define TDX_METADATA_ATTRIBUTES_FIXED1_NAME	"1900000300000001"
+#define TDX_METADATA_XFAM_FIXED0_NAME		"1900000300000002"
+#define TDX_METADATA_XFAM_FIXED1_NAME		"1900000300000003"
+#define TDX_METADATA_NUM_CPUID_CONFIG_NAME	"9900000100000004"
+#define TDX_METADATA_CPUID_LEAVES_NAME		"9900000300000400"
+#define TDX_METADATA_CPUID_VALUES_NAME		"9900000300000500"
+
 #endif

Signature · 6 lines

2.25.1



-- 
Isaku Yamahata <[email protected]>

Zhi Wang

2023-03-31 12:47 UTC #48 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

Quoted 18 lines · On Thu, 30 Mar 2023 17:18:03 -0700 Isaku Yamahata <[email protected]>…

Quoted 15 lines · Isaku Yamahata
Quoted 11 lines · Zhi Wang
On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:
Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.
Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?

Quoted 7 lines

Quoted 7 lines
We only need global parameters of the TDX module, and we don't interact with TDX module at this point. One alternative is to export those parameters via sysfs. Also the existence of the sysfs node indicates that the TDX module is loaded(initialized?) or not in addition to boot log. Thus we can drop system scope one.
What do you think?

I like this idea and the patch below, it feels right for me now. It would be nice if more folks can chime in and comment.

Quoted 4 lines

Quoted 3 lines
Regarding to other TDX KVM specific ioctls (KVM_TDX_INIT_VM, KVM_TDX_INIT_VCPU, KVM_TDX_INIT_MEM_REGION, and KVM_TDX_FINALIZE_VM), they are specific to KVM. So I don't think it can be split out to independent driver.

They can stay in KVM as they are KVM-specific. SNP also has KVM-specific ioctls which wraps the SEV driver calls. At this level, both TDX and SNP go their specific implementation without more abstraction other than KVM_ENCRYPT_MEMORY_OP. Their strategies are aligned.

The problem of the previous approach was the abstraction that no other implementation is using it. It is like, TDX wants a higher abstraction to cover both TDX and SNP, but SNP is not using it, which makes the abstraction looks strange.

Quoted 315 lines

Here is the patch to export those info via sysfs.

TDX module has parameters for VMM to configure TD. User space VMM, e.g. qemu, needs to know it. Export them to user space via sysfs.

In reply to Isaku Yamahata

---
 Documentation/ABI/testing/sysfs-firmware-tdx |  23 +++
 arch/x86/include/asm/tdx.h                   |  33 ++++
 arch/x86/virt/vmx/tdx/tdx.c                  | 164 +++++++++++++++++++
 arch/x86/virt/vmx/tdx/tdx.h                  |  18 ++
 4 files changed, 238 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-firmware-tdx

Documentation/ABI/testing/sysfs-firmware-tdx new file +23 -0

new file mode 100644
index 000000000000..1f26fb178144
@@ -0,0 +1,23 @@
+What:           /sys/firmware/tdx/tdx_module/metadata
+Date:           March 2023
+KernelVersion:  6.3
+Contact:        Isaku Yamahata <[email protected]>, [email protected]
+Users:          qemu, libvirt
+Description:
+                The TDX feature requires a firmware that is known as the TDX
+                module.  The TDX module exposes its metadata in the following
+                read-only files.  The information corresponds to the TDX global
+                metadata specified by 64bit field id.  The file name is hex
+                string in lower case.  The value is binary.
+                User space VMM like qemu needs refer to them to determine what
+                parameters are needed or allowed to configure guest TDs.
+
+                ================== ============================================
+                1900000300000000   ATTRIBUTES_FIXED0
+                1900000300000001   ATTRIBUTES_FIXED1
+                1900000300000002   XFAM_FIXED0
+                1900000300000003   XFAM_FIXED1
+                9900000100000004   NUM_CPUID_CONFIG
+                9900000300000400   CPUID_LEAVES
+                9900000300000500   CPUID_VALUES
+                ================== ============================================
\ No newline at end of file

arch/x86/include/asm/tdx.h +33 -0

index 05870e5ed131..c650ac22a916 100644
@@ -110,6 +110,39 @@ struct tdx_cpuid_config {
 	u32	edx;
 } __packed;
 
+struct tdx_cpuid_config_leaf {
+	u32	leaf;
+	u32	sub_leaf;
+} __packed;
+static_assert(offsetof(struct tdx_cpuid_config, leaf) ==
+	      offsetof(struct tdx_cpuid_config_leaf, leaf));
+static_assert(offsetof(struct tdx_cpuid_config, sub_leaf) ==
+	      offsetof(struct tdx_cpuid_config_leaf, sub_leaf));
+static_assert(offsetofend(struct tdx_cpuid_config, sub_leaf) ==
+	      sizeof(struct tdx_cpuid_config_leaf));
+
+struct tdx_cpuid_config_value {
+	u32	eax;
+	u32	ebx;
+	u32	ecx;
+	u32	edx;
+} __packed;
+static_assert(offsetof(struct tdx_cpuid_config, eax) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, eax));
+static_assert(offsetof(struct tdx_cpuid_config, ebx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, ebx));
+static_assert(offsetof(struct tdx_cpuid_config, ecx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, ecx));
+static_assert(offsetof(struct tdx_cpuid_config, edx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, edx));
+static_assert(offsetofend(struct tdx_cpuid_config, edx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      sizeof(struct tdx_cpuid_config_value));
+
 #define TDSYSINFO_STRUCT_SIZE		1024
 #define TDSYSINFO_STRUCT_ALIGNMENT	1024

arch/x86/virt/vmx/tdx/tdx.c +164 -0

index f9f9c1b76501..56ca520d67d6 100644
@@ -33,6 +33,12 @@
 #include <asm/tdx.h>
 #include "tdx.h"
 
+#ifdef CONFIG_SYSFS
+static int tdx_sysfs_init(void);
+#else
+static inline int tdx_sysfs_init(void) { return 0;}
+#endif
+
 u32 tdx_global_keyid __ro_after_init;
 EXPORT_SYMBOL_GPL(tdx_global_keyid);
 static u32 tdx_guest_keyid_start __ro_after_init;

@@ -399,6 +405,10 @@ static int __tdx_get_sysinfo(struct tdsysinfo_struct *sysinfo,
 	if (ret)
 		return ret;
 
+	ret = tdx_sysfs_init();
+	if (ret)
+		return ret;
+
 	pr_info("TDX module: atributes 0x%x, vendor_id 0x%x, major_version %u, minor_version %u, build_date %u, build_num %u",
 		sysinfo->attributes,	sysinfo->vendor_id,
 		sysinfo->major_version, sysinfo->minor_version,

@@ -1367,3 +1377,157 @@ int tdx_enable(void)
 	return ret;
 }
 EXPORT_SYMBOL_GPL(tdx_enable);
+
+#ifdef CONFIG_SYSFS
+
+static struct kobject *tdx_kobj;
+static struct kobject *tdx_module_kobj;
+static struct kobject *tdx_metadata_kobj;
+
+#define TDX_METADATA_ATTR(_name, field_id_name, _size)		\
+static struct bin_attribute tdx_metadata_ ## _name = {		\
+	.attr = {						\
+		.name = field_id_name,				\
+		.mode = 0444,					\
+	},							\
+	.size = _size,						\
+	.read = tdx_metadata_ ## _name ## _show,		\
+}
+
+#define TDX_METADATA_ATTR_SHOW(_name, field_id_name)					\
+static ssize_t tdx_metadata_ ## _name ## _show(struct file *filp, struct kobject *kobj,	\
+					       struct bin_attribute *bin_attr,		\
+					       char *buf, loff_t offset, size_t count)	\
+{											\
+	struct tdsysinfo_struct *sysinfo = &PADDED_STRUCT(tdsysinfo);			\
+											\
+	return memory_read_from_buffer(buf, count, &offset,				\
+				       &sysinfo->_name,					\
+				       sizeof(sysinfo->_name));				\
+}											\
+TDX_METADATA_ATTR(_name, field_id_name, sizeof_field(struct tdsysinfo_struct, _name))
+
+TDX_METADATA_ATTR_SHOW(attributes_fixed0, TDX_METADATA_ATTRIBUTES_FIXED0_NAME);
+TDX_METADATA_ATTR_SHOW(attributes_fixed1, TDX_METADATA_ATTRIBUTES_FIXED1_NAME);
+TDX_METADATA_ATTR_SHOW(xfam_fixed0, TDX_METADATA_XFAM_FIXED0_NAME);
+TDX_METADATA_ATTR_SHOW(xfam_fixed1, TDX_METADATA_XFAM_FIXED1_NAME);
+
+static ssize_t tdx_metadata_num_cpuid_config_show(struct file *filp, struct kobject *kobj,
+						  struct bin_attribute *bin_attr,
+						  char *buf, loff_t offset, size_t count)
+{
+	struct tdsysinfo_struct *sysinfo = &PADDED_STRUCT(tdsysinfo);
+	/*
+	 * Although tdsysinfo_struct.num_cpuid_config is defined as u32 for
+	 * alignment, TDX 1.5 defines metadata NUM_CONFIG_CPUID as u16.
+	 */
+	u16 tmp = (u16)sysinfo->num_cpuid_config;
+
+	WARN_ON_ONCE(tmp != sysinfo->num_cpuid_config);
+	return memory_read_from_buffer(buf, count, &offset, &tmp, sizeof(tmp));
+}
+TDX_METADATA_ATTR(num_cpuid_config, TDX_METADATA_NUM_CPUID_CONFIG_NAME, sizeof(u16));
+
+static ssize_t tdx_metadata_cpuid_leaves_show(struct file *filp, struct kobject *kobj,
+					      struct bin_attribute *bin_attr, char *buf,
+					      loff_t offset, size_t count)
+{
+	struct tdsysinfo_struct *sysinfo = &PADDED_STRUCT(tdsysinfo);
+	ssize_t r;
+	struct tdx_cpuid_config_leaf *tmp;
+	u32 i;
+
+	tmp = kmalloc(bin_attr->size, GFP_KERNEL);
+	if (!tmp)
+		return -ENOMEM;
+
+	for (i = 0; i < sysinfo->num_cpuid_config; i++) {
+		struct tdx_cpuid_config *c = &sysinfo->cpuid_configs[i];
+		struct tdx_cpuid_config_leaf *leaf = (struct tdx_cpuid_config_leaf *)c;
+
+		memcpy(tmp + i, leaf, sizeof(*leaf));
+	}
+
+	r = memory_read_from_buffer(buf, count, &offset, tmp, bin_attr->size);
+	kfree(tmp);
+	return r;
+}
+
+TDX_METADATA_ATTR(cpuid_leaves, TDX_METADATA_CPUID_LEAVES_NAME, 0);
+
+static ssize_t tdx_metadata_cpuid_values_show(struct file *filp, struct kobject *kobj,
+					      struct bin_attribute *bin_attr, char *buf,
+					      loff_t offset, size_t count)
+{
+	struct tdsysinfo_struct *sysinfo = &PADDED_STRUCT(tdsysinfo);
+	struct tdx_cpuid_config_value *tmp;
+	ssize_t r;
+	u32 i;
+
+	tmp = kmalloc(bin_attr->size, GFP_KERNEL);
+	if (!tmp)
+		return -ENOMEM;
+
+	for (i = 0; i < sysinfo->num_cpuid_config; i++) {
+		struct tdx_cpuid_config *c = &sysinfo->cpuid_configs[i];
+		struct tdx_cpuid_config_value *value = (struct tdx_cpuid_config_value *)&c->eax;
+
+		memcpy(tmp + i, value, sizeof(*value));
+	}
+
+	r = memory_read_from_buffer(buf, count, &offset, tmp, bin_attr->size);
+	kfree(tmp);
+	return r;
+}
+
+TDX_METADATA_ATTR(cpuid_values, TDX_METADATA_CPUID_VALUES_NAME, 0);
+
+static struct bin_attribute *tdx_metadata_attrs[] = {
+	&tdx_metadata_attributes_fixed0,
+	&tdx_metadata_attributes_fixed1,
+	&tdx_metadata_xfam_fixed0,
+	&tdx_metadata_xfam_fixed1,
+	&tdx_metadata_num_cpuid_config,
+	&tdx_metadata_cpuid_leaves,
+	&tdx_metadata_cpuid_values,
+	NULL,
+};
+
+static const struct attribute_group tdx_metadata_attr_group = {
+	.bin_attrs = tdx_metadata_attrs,
+};
+
+static int tdx_sysfs_init(void)
+{
+	struct tdsysinfo_struct *sysinfo;
+	int ret;
+
+	tdx_kobj = kobject_create_and_add("tdx", firmware_kobj);
+	if (!tdx_kobj) {
+		pr_err("kobject_create_and_add tdx failed\n");
+		return -EINVAL;
+	}
+
+	tdx_module_kobj = kobject_create_and_add("tdx_module", tdx_kobj);
+	if (!tdx_module_kobj) {
+		pr_err("kobject_create_and_add tdx_module failed\n");
+		return -EINVAL;
+	}
+	tdx_metadata_kobj = kobject_create_and_add("metadata", tdx_module_kobj);
+	if (!tdx_metadata_kobj) {
+		pr_err("Sysfs exporting tdx global metadata failed %d\n", ret);
+		return -EINVAL;
+	}
+
+	sysinfo = &PADDED_STRUCT(tdsysinfo);
+	tdx_metadata_cpuid_leaves.size = sysinfo->num_cpuid_config *
+		sizeof(struct tdx_cpuid_config_leaf);
+	tdx_metadata_cpuid_values.size = sysinfo->num_cpuid_config *
+		sizeof(struct tdx_cpuid_config_value);
+	ret = sysfs_create_group(tdx_metadata_kobj, &tdx_metadata_attr_group);
+	if (ret)
+		pr_err("Sysfs exporting tdx module attributes failed %d\n", ret);
+
+	return ret;
+}
+#endif

arch/x86/virt/vmx/tdx/tdx.h +18 -0

index db0cbcceb5b3..a48f38fe6cc4 100644
@@ -93,4 +93,22 @@ struct tdmr_info_list {
 	int max_tdmrs;	/* How many 'tdmr_info's are allocated */
 };
 
+/* TDX metadata base field id. */
+#define TDX_METADATA_ATTRIBUTES_FIXED0		0x1900000300000000ULL
+#define TDX_METADATA_ATTRIBUTES_FIXED1		0x1900000300000001ULL
+#define TDX_METADATA_XFAM_FIXED0		0x1900000300000002ULL
+#define TDX_METADATA_XFAM_FIXED1		0x1900000300000003ULL
+#define TDX_METADATA_NUM_CPUID_CONFIG		0x9900000100000004ULL
+#define TDX_METADATA_CPUID_LEAVES		0x9900000300000400ULL
+#define TDX_METADATA_CPUID_VALUES		0x9900000300000500ULL
+
+/* File name for sysfs: hex with lower case. */
+#define TDX_METADATA_ATTRIBUTES_FIXED0_NAME	"1900000300000000"
+#define TDX_METADATA_ATTRIBUTES_FIXED1_NAME	"1900000300000001"
+#define TDX_METADATA_XFAM_FIXED0_NAME		"1900000300000002"
+#define TDX_METADATA_XFAM_FIXED1_NAME		"1900000300000003"
+#define TDX_METADATA_NUM_CPUID_CONFIG_NAME	"9900000100000004"
+#define TDX_METADATA_CPUID_LEAVES_NAME		"9900000300000400"
+#define TDX_METADATA_CPUID_VALUES_NAME		"9900000300000500"
+
 #endif

Xiaoyao Li

2023-04-03 03:46 UTC #49 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

Quoted 29 lines · Zhi Wang

Quoted 18 lines · On Thu, 30 Mar 2023 17:18:03 -0700 Isaku Yamahata <[email protected]>…
Quoted 15 lines · Isaku Yamahata
Quoted 11 lines · Zhi Wang
On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:
Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.
Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?
Quoted 7 lines
Quoted 7 lines
We only need global parameters of the TDX module, and we don't interact with TDX module at this point. One alternative is to export those parameters via sysfs. Also the existence of the sysfs node indicates that the TDX module is loaded(initialized?) or not in addition to boot log. Thus we can drop system scope one.
What do you think?

Quoted 2 lines

I like this idea and the patch below, it feels right for me now. It would be nice if more folks can chime in and comment.

SYSFS option requires CONFIG_SYSFS, which reqiures CONFIG_KVM_TDX to select CONFIG_SYSFS.

Quoted 5 lines

Quoted 4 lines
Quoted 3 lines
Regarding to other TDX KVM specific ioctls (KVM_TDX_INIT_VM, KVM_TDX_INIT_VCPU, KVM_TDX_INIT_MEM_REGION, and KVM_TDX_FINALIZE_VM), they are specific to KVM. So I don't think it can be split out to independent driver.

Quoted 8 lines

They can stay in KVM as they are KVM-specific. SNP also has KVM-specific ioctls which wraps the SEV driver calls. At this level, both TDX and SNP go their specific implementation without more abstraction other than KVM_ENCRYPT_MEMORY_OP. Their strategies are aligned.
The problem of the previous approach was the abstraction that no other implementation is using it. It is like, TDX wants a higher abstraction to cover both TDX and SNP, but SNP is not using it, which makes the abstraction looks strange.

Note, before this TDX enabling series, KVM_MEMORY_ENCRYPT_OP is a VM scope ioctl, that only serves for SEV and no other implementation uses it. I see no reason why cannot introduce a new IOCTL in x86 KVM that serves only one vendor.

We choose KVM_MEMORY_ENCRYPT_OP for TDX platform scope, just because we reuse KVM_MEMORY_ENCRYPT_OP for TDX VM-scope and extend it to TDX vcpu scope. It's just to avoid defining a new IOCTL number.

We can rename it to KVM_GET_CC_CAPABILITIES, and even return different capabilities based on VM type. And even, if SNP wants to use it, I think it can wrap SNP driver calls inside this IOCTL?

kvm.ko is special that it needs to serve two vendors. Sometime it's unaviodable that an interface is only used by one vendor.

Zhi Wang

2023-04-03 14:29 UTC #50 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

Quoted 46 lines · On Mon, 3 Apr 2023 11:46:15 +0800 Xiaoyao Li <[email protected]> wrote:

Quoted 29 lines · Zhi Wang
Quoted 18 lines · On Thu, 30 Mar 2023 17:18:03 -0700 Isaku Yamahata <[email protected]>…
Quoted 15 lines · Isaku Yamahata
Quoted 11 lines · Zhi Wang
On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:
Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.
Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?
Quoted 7 lines
Quoted 7 lines
We only need global parameters of the TDX module, and we don't interact with TDX module at this point. One alternative is to export those parameters via sysfs. Also the existence of the sysfs node indicates that the TDX module is loaded(initialized?) or not in addition to boot log. Thus we can drop system scope one.
What do you think?
Quoted 2 lines
I like this idea and the patch below, it feels right for me now. It would be nice if more folks can chime in and comment.
SYSFS option requires CONFIG_SYSFS, which reqiures CONFIG_KVM_TDX to select CONFIG_SYSFS.
Quoted 10 lines
Quoted 4 lines
Quoted 3 lines
Regarding to other TDX KVM specific ioctls (KVM_TDX_INIT_VM, KVM_TDX_INIT_VCPU, KVM_TDX_INIT_MEM_REGION, and KVM_TDX_FINALIZE_VM), they are specific to KVM. So I don't think it can be split out to independent driver.
They can stay in KVM as they are KVM-specific. SNP also has KVM-specific ioctls which wraps the SEV driver calls. At this level, both TDX and SNP go their specific implementation without more abstraction other than KVM_ENCRYPT_MEMORY_OP. Their strategies are aligned.

Quoted 9 lines

Quoted 3 lines
The problem of the previous approach was the abstraction that no other implementation is using it. It is like, TDX wants a higher abstraction to cover both TDX and SNP, but SNP is not using it, which makes the abstraction looks strange.
Note, before this TDX enabling series, KVM_MEMORY_ENCRYPT_OP is a VM scope ioctl, that only serves for SEV and no other implementation uses it. I see no reason why cannot introduce a new IOCTL in x86 KVM that serves only one vendor.

My point is: time is different. When KVM_MEMORY_ENCRYPT_OP is there, there was *only* one vendor and SEV/SNP didn't know how the future vendor is going to use the ioctl. That is a reasonable case an generic ioctl can have one vendor to back up.

The background here is: now another vendor is coming and there are going to be two vendors. The two vendors' flows are much clearer than early stage. Like, they know which flow is going to be used by each other.

With these kept in mind, IMHO, it is not appropriate to introduce an generic ioctl that only one vendor is going to use, meanwhile we have already known another vendor is not going to use it.

Defining a new userspace ABI is a serious thing and it is not an early stage anymore. Actually I think it is the best time to see how the code infrastructure should be re-purposed at this time.

Quoted 8 lines

We choose KVM_MEMORY_ENCRYPT_OP for TDX platform scope, just because we reuse KVM_MEMORY_ENCRYPT_OP for TDX VM-scope and extend it to TDX vcpu scope. It's just to avoid defining a new IOCTL number.
We can rename it to KVM_GET_CC_CAPABILITIES, and even return different capabilities based on VM type. And even, if SNP wants to use it, I think it can wrap SNP driver calls inside this IOCTL?

I am not opposed to this option as it shows effort to improve it and it is constructive. But this needs to be figured out with AMD folks and maintainers. E.g. what should be the best CC ioctl scheme for KVM? vendor-specific or generic, which brings better benefit for the userspace, and less maintenance burden.

Back to the reason why I think a vendor-specific sysinfo interface for TDX is necessary:

1) SEV driver has been there for quite some time. Unless people thinks an generic CC ioctl scheme is a way to go, then there will be motivation and efforts putting on it. The efforts is not only about wrapping SEV ioctls, it needs a systematic spec of generic CC ioctl scheme.

2) TDX doesn't have a driver like SEV and possibly not going to have one in the future. For those non-KVM related control flow of TDX in future, they can re-use this and stay away from KVM interface. (If vendor-specific scheme is the future direction.)

Quoted 2 lines

kvm.ko is special that it needs to serve two vendors. Sometime it's unaviodable that an interface is only used by one vendor.

I am afraid that in this case it is avoidable right?

Isaku Yamahata

2023-04-05 18:07 UTC #51 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

Quoted 104 lines · Zhi Wang

Quoted 46 lines · On Mon, 3 Apr 2023 11:46:15 +0800 Xiaoyao Li <[email protected]> wrote:
Quoted 29 lines · Zhi Wang
Quoted 18 lines · On Thu, 30 Mar 2023 17:18:03 -0700 Isaku Yamahata <[email protected]>…
Quoted 15 lines · Isaku Yamahata
Quoted 11 lines · Zhi Wang
On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:
Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.
Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?
Quoted 7 lines
Quoted 7 lines
We only need global parameters of the TDX module, and we don't interact with TDX module at this point. One alternative is to export those parameters via sysfs. Also the existence of the sysfs node indicates that the TDX module is loaded(initialized?) or not in addition to boot log. Thus we can drop system scope one.
What do you think?
Quoted 2 lines
I like this idea and the patch below, it feels right for me now. It would be nice if more folks can chime in and comment.
SYSFS option requires CONFIG_SYSFS, which reqiures CONFIG_KVM_TDX to select CONFIG_SYSFS.
Quoted 10 lines
Quoted 4 lines
Quoted 3 lines
Regarding to other TDX KVM specific ioctls (KVM_TDX_INIT_VM, KVM_TDX_INIT_VCPU, KVM_TDX_INIT_MEM_REGION, and KVM_TDX_FINALIZE_VM), they are specific to KVM. So I don't think it can be split out to independent driver.
They can stay in KVM as they are KVM-specific. SNP also has KVM-specific ioctls which wraps the SEV driver calls. At this level, both TDX and SNP go their specific implementation without more abstraction other than KVM_ENCRYPT_MEMORY_OP. Their strategies are aligned.
Quoted 9 lines
Quoted 3 lines
The problem of the previous approach was the abstraction that no other implementation is using it. It is like, TDX wants a higher abstraction to cover both TDX and SNP, but SNP is not using it, which makes the abstraction looks strange.
Note, before this TDX enabling series, KVM_MEMORY_ENCRYPT_OP is a VM scope ioctl, that only serves for SEV and no other implementation uses it. I see no reason why cannot introduce a new IOCTL in x86 KVM that serves only one vendor.
My point is: time is different. When KVM_MEMORY_ENCRYPT_OP is there, there was *only* one vendor and SEV/SNP didn't know how the future vendor is going to use the ioctl. That is a reasonable case an generic ioctl can have one vendor to back up.
The background here is: now another vendor is coming and there are going to be two vendors. The two vendors' flows are much clearer than early stage. Like, they know which flow is going to be used by each other.
With these kept in mind, IMHO, it is not appropriate to introduce an generic ioctl that only one vendor is going to use, meanwhile we have already known another vendor is not going to use it.
Defining a new userspace ABI is a serious thing and it is not an early stage anymore. Actually I think it is the best time to see how the code infrastructure should be re-purposed at this time.
Quoted 8 lines
We choose KVM_MEMORY_ENCRYPT_OP for TDX platform scope, just because we reuse KVM_MEMORY_ENCRYPT_OP for TDX VM-scope and extend it to TDX vcpu scope. It's just to avoid defining a new IOCTL number.
We can rename it to KVM_GET_CC_CAPABILITIES, and even return different capabilities based on VM type. And even, if SNP wants to use it, I think it can wrap SNP driver calls inside this IOCTL?
I am not opposed to this option as it shows effort to improve it and it is constructive. But this needs to be figured out with AMD folks and maintainers. E.g. what should be the best CC ioctl scheme for KVM? vendor-specific or generic, which brings better benefit for the userspace, and less maintenance burden.
Back to the reason why I think a vendor-specific sysinfo interface for TDX is necessary:
1) SEV driver has been there for quite some time. Unless people thinks an generic CC ioctl scheme is a way to go, then there will be motivation and efforts putting on it. The efforts is not only about wrapping SEV ioctls, it needs a systematic spec of generic CC ioctl scheme.
2) TDX doesn't have a driver like SEV and possibly not going to have one in the future. For those non-KVM related control flow of TDX in future, they can re-use this and stay away from KVM interface. (If vendor-specific scheme is the future direction.)

Quoted 4 lines

Quoted 2 lines
kvm.ko is special that it needs to serve two vendors. Sometime it's unaviodable that an interface is only used by one vendor.
I am afraid that in this case it is avoidable right?

We can make KVM_TDX_CAPABILITIES vm-scoped one so that devoce-scoped KVM_EMORY_ENCRYPT_OP isn't needed. At least qemu is fine.

Do you think vm-scoped KVM_TDX_CAPABILITIES is fine?

Signature · 1 line

Isaku Yamahata <[email protected]>

Zhi Wang

2023-04-06 16:25 UTC #52 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

Quoted 110 lines · On Wed, 5 Apr 2023 11:07:20 -0700 Isaku Yamahata <[email protected]>…

Quoted 107 lines · Zhi Wang
Quoted 46 lines · On Mon, 3 Apr 2023 11:46:15 +0800 Xiaoyao Li <[email protected]> wrote:
Quoted 29 lines · Zhi Wang
Quoted 18 lines · On Thu, 30 Mar 2023 17:18:03 -0700 Isaku Yamahata <[email protected]>…
Quoted 15 lines · Isaku Yamahata
Quoted 11 lines · Zhi Wang
On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:
Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.
Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?
Quoted 7 lines
Quoted 7 lines
We only need global parameters of the TDX module, and we don't interact with TDX module at this point. One alternative is to export those parameters via sysfs. Also the existence of the sysfs node indicates that the TDX module is loaded(initialized?) or not in addition to boot log. Thus we can drop system scope one.
What do you think?
Quoted 2 lines
I like this idea and the patch below, it feels right for me now. It would be nice if more folks can chime in and comment.
SYSFS option requires CONFIG_SYSFS, which reqiures CONFIG_KVM_TDX to select CONFIG_SYSFS.
Quoted 10 lines
Quoted 4 lines
Quoted 3 lines
Regarding to other TDX KVM specific ioctls (KVM_TDX_INIT_VM, KVM_TDX_INIT_VCPU, KVM_TDX_INIT_MEM_REGION, and KVM_TDX_FINALIZE_VM), they are specific to KVM. So I don't think it can be split out to independent driver.
They can stay in KVM as they are KVM-specific. SNP also has KVM-specific ioctls which wraps the SEV driver calls. At this level, both TDX and SNP go their specific implementation without more abstraction other than KVM_ENCRYPT_MEMORY_OP. Their strategies are aligned.
Quoted 9 lines
Quoted 3 lines
The problem of the previous approach was the abstraction that no other implementation is using it. It is like, TDX wants a higher abstraction to cover both TDX and SNP, but SNP is not using it, which makes the abstraction looks strange.
Note, before this TDX enabling series, KVM_MEMORY_ENCRYPT_OP is a VM scope ioctl, that only serves for SEV and no other implementation uses it. I see no reason why cannot introduce a new IOCTL in x86 KVM that serves only one vendor.
My point is: time is different. When KVM_MEMORY_ENCRYPT_OP is there, there was *only* one vendor and SEV/SNP didn't know how the future vendor is going to use the ioctl. That is a reasonable case an generic ioctl can have one vendor to back up.
The background here is: now another vendor is coming and there are going to be two vendors. The two vendors' flows are much clearer than early stage. Like, they know which flow is going to be used by each other.
With these kept in mind, IMHO, it is not appropriate to introduce an generic ioctl that only one vendor is going to use, meanwhile we have already known another vendor is not going to use it.
Defining a new userspace ABI is a serious thing and it is not an early stage anymore. Actually I think it is the best time to see how the code infrastructure should be re-purposed at this time.
Quoted 8 lines
We choose KVM_MEMORY_ENCRYPT_OP for TDX platform scope, just because we reuse KVM_MEMORY_ENCRYPT_OP for TDX VM-scope and extend it to TDX vcpu scope. It's just to avoid defining a new IOCTL number.
We can rename it to KVM_GET_CC_CAPABILITIES, and even return different capabilities based on VM type. And even, if SNP wants to use it, I think it can wrap SNP driver calls inside this IOCTL?
I am not opposed to this option as it shows effort to improve it and it is constructive. But this needs to be figured out with AMD folks and maintainers. E.g. what should be the best CC ioctl scheme for KVM? vendor-specific or generic, which brings better benefit for the userspace, and less maintenance burden.
Back to the reason why I think a vendor-specific sysinfo interface for TDX is necessary:
1) SEV driver has been there for quite some time. Unless people thinks an generic CC ioctl scheme is a way to go, then there will be motivation and efforts putting on it. The efforts is not only about wrapping SEV ioctls, it needs a systematic spec of generic CC ioctl scheme.
2) TDX doesn't have a driver like SEV and possibly not going to have one in the future. For those non-KVM related control flow of TDX in future, they can re-use this and stay away from KVM interface. (If vendor-specific scheme is the future direction.)
Quoted 2 lines
kvm.ko is special that it needs to serve two vendors. Sometime it's unaviodable that an interface is only used by one vendor.

Quoted 6 lines

Quoted 1 line
I am afraid that in this case it is avoidable right?
We can make KVM_TDX_CAPABILITIES vm-scoped one so that devoce-scoped KVM_EMORY_ENCRYPT_OP isn't needed. At least qemu is fine.
Do you think vm-scoped KVM_TDX_CAPABILITIES is fine?

That looks better to me as it will stay with other KVM_TDX* ioctls then.

Dan Williams

2024-01-31 06:25 UTC #53 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

Quoted 39 lines · Isaku Yamahata

Quoted 22 lines · Isaku Yamahata
Quoted 11 lines · Zhi Wang
On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:
Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.
Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?
We only need global parameters of the TDX module, and we don't interact with TDX module at this point. One alternative is to export those parameters via sysfs. Also the existence of the sysfs node indicates that the TDX module is loaded(initialized?) or not in addition to boot log. Thus we can drop system scope one.
What do you think?
Quoted 3 lines
Regarding to other TDX KVM specific ioctls (KVM_TDX_INIT_VM, KVM_TDX_INIT_VCPU, KVM_TDX_INIT_MEM_REGION, and KVM_TDX_FINALIZE_VM), they are specific to KVM. So I don't think it can be split out to independent driver.
Here is the patch to export those info via sysfs.
From e0744e506eb92e47d8317e489945a3ba804edfa7 Mon Sep 17 00:00:00 2001
Message-Id: <e0744e506eb92e47d8317e489945a3ba804edfa7.1680221730.git.isaku.yamahata@intel.com>
In-Reply-To: <8e0bc0e8e5d435f54f10c7642a862629ef2acb89.1680221729.git.isaku.yamahata@intel.com>
References: <8e0bc0e8e5d435f54f10c7642a862629ef2acb89.1680221729.git.isaku.yamahata@intel.com>
From: Isaku Yamahata <[email protected]>
Date: Thu, 30 Mar 2023 00:05:03 -0700
Subject: [PATCH] x86/virt/tdx: Export TD config params of TDX module via sysfs

Quoted 8 lines

TDX module has parameters for VMM to configure TD. User space VMM, e.g. qemu, needs to know it. Export them to user space via sysfs.
TDX 1.0 provides TDH.SYS.INFO to provide system information in TDSYSINFO_STRUCT. Its future extensibility is limited because of its struct. From TDX 1.5, TDH.SYS.RD(metadata field_id) to read the info specified by field id. So instead of exporting TDSYSINFO_STRUCT, adapt metadata way to export those system information.

Hi, I came across tdx_sysfs_init() recently and had some comments if this proposal is going to move forward:

Quoted 16 lines

[1]: http://lore.kernel.org/r/170660662589.224441.11503798303914595072.stgit@dwillia2-xfh.jf.intel.com

---
 Documentation/ABI/testing/sysfs-firmware-tdx |  23 +++
 arch/x86/include/asm/tdx.h                   |  33 ++++
 arch/x86/virt/vmx/tdx/tdx.c                  | 164 +++++++++++++++++++
 arch/x86/virt/vmx/tdx/tdx.h                  |  18 ++
 4 files changed, 238 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-firmware-tdx

Documentation/ABI/testing/sysfs-firmware-tdx new file +0 -0

new file mode 100644
index 000000000000..1f26fb178144

Documentation/ABI/testing/sysfs-firmware-tdx line 1

Quoted 1 line

Documentation/ABI/testing/sysfs-firmware-tdx new file +1 -0
@@ -0,0 +1,23 @@
+What:           /sys/firmware/tdx/tdx_module/metadata

The TDX module is not "platform firmware" in comparison to the other EFI and ACPI inhabitants in /sys/firmware. It is especially not static platform firmware given it needs to be dynamically activated via KVM module initialization.

Instead, sysfs already has a location for pure software construct objects to host a sysfs ABI and that is /sys/bus/virtual. I propose a common "TSM" class device here [1] and TDX can simply publish a named attribute group, "host", to extend that class device with TDX specifics.

For cross-vendor consistency "host" is a symlink to the CCP device on AMD.

Quoted 12 lines

Documentation/ABI/testing/sysfs-firmware-tdx new file +12 -0

+Date:           March 2023
+KernelVersion:  6.3
+Contact:        Isaku Yamahata <[email protected]>, [email protected]
+Users:          qemu, libvirt
+Description:
+                The TDX feature requires a firmware that is known as the TDX
+                module.  The TDX module exposes its metadata in the following
+                read-only files.  The information corresponds to the TDX global
+                metadata specified by 64bit field id.
+                string in lower case.  The value is binary.
+                User space VMM like qemu needs refer to them to determine what
+                parameters are needed or allowed to configure guest TDs.

Documentation/ABI/testing/sysfs-firmware-tdx line 23

Quoted 10 lines

Documentation/ABI/testing/sysfs-firmware-tdx new file +10 -0

+
+                ================== ============================================
+                1900000300000000   ATTRIBUTES_FIXED0
+                1900000300000001   ATTRIBUTES_FIXED1
+                1900000300000002   XFAM_FIXED0
+                1900000300000003   XFAM_FIXED1
+                9900000100000004   NUM_CPUID_CONFIG
+                9900000300000400   CPUID_LEAVES
+                9900000300000500   CPUID_VALUES
+                ================== ============================================

This documentation needs to be per file. With an explanation of how each file is expected to be used. Someone should reasonably be able to read this documentation and go write a tool, I don't get that from this documentation.

Quoted 114 lines

\ No newline at end of file

arch/x86/include/asm/tdx.h +33 -0

index 05870e5ed131..c650ac22a916 100644
@@ -110,6 +110,39 @@ struct tdx_cpuid_config {
 	u32	edx;
 } __packed;
 
+struct tdx_cpuid_config_leaf {
+	u32	leaf;
+	u32	sub_leaf;
+} __packed;
+static_assert(offsetof(struct tdx_cpuid_config, leaf) ==
+	      offsetof(struct tdx_cpuid_config_leaf, leaf));
+static_assert(offsetof(struct tdx_cpuid_config, sub_leaf) ==
+	      offsetof(struct tdx_cpuid_config_leaf, sub_leaf));
+static_assert(offsetofend(struct tdx_cpuid_config, sub_leaf) ==
+	      sizeof(struct tdx_cpuid_config_leaf));
+
+struct tdx_cpuid_config_value {
+	u32	eax;
+	u32	ebx;
+	u32	ecx;
+	u32	edx;
+} __packed;
+static_assert(offsetof(struct tdx_cpuid_config, eax) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, eax));
+static_assert(offsetof(struct tdx_cpuid_config, ebx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, ebx));
+static_assert(offsetof(struct tdx_cpuid_config, ecx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, ecx));
+static_assert(offsetof(struct tdx_cpuid_config, edx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      offsetof(struct tdx_cpuid_config_value, edx));
+static_assert(offsetofend(struct tdx_cpuid_config, edx) -
+	      offsetof(struct tdx_cpuid_config, eax) ==
+	      sizeof(struct tdx_cpuid_config_value));
+
 #define TDSYSINFO_STRUCT_SIZE		1024
 #define TDSYSINFO_STRUCT_ALIGNMENT	1024

arch/x86/virt/vmx/tdx/tdx.c +47 -0

index f9f9c1b76501..56ca520d67d6 100644
@@ -33,6 +33,12 @@
 #include <asm/tdx.h>
 #include "tdx.h"
 
+#ifdef CONFIG_SYSFS
+static int tdx_sysfs_init(void);
+#else
+static inline int tdx_sysfs_init(void) { return 0;}
+#endif
+
 u32 tdx_global_keyid __ro_after_init;
 EXPORT_SYMBOL_GPL(tdx_global_keyid);
 static u32 tdx_guest_keyid_start __ro_after_init;

@@ -399,6 +405,10 @@ static int __tdx_get_sysinfo(struct tdsysinfo_struct *sysinfo,
 	if (ret)
 		return ret;
 
+	ret = tdx_sysfs_init();
+	if (ret)
+		return ret;
+
 	pr_info("TDX module: atributes 0x%x, vendor_id 0x%x, major_version %u, minor_version %u, build_date %u, build_num %u",
 		sysinfo->attributes,	sysinfo->vendor_id,
 		sysinfo->major_version, sysinfo->minor_version,

@@ -1367,3 +1377,157 @@ int tdx_enable(void)
 	return ret;
 }
 EXPORT_SYMBOL_GPL(tdx_enable);
+
+#ifdef CONFIG_SYSFS
+
+static struct kobject *tdx_kobj;
+static struct kobject *tdx_module_kobj;
+static struct kobject *tdx_metadata_kobj;
+
+#define TDX_METADATA_ATTR(_name, field_id_name, _size)		\
+static struct bin_attribute tdx_metadata_ ## _name = {		\
+	.attr = {						\
+		.name = field_id_name,				\
+		.mode = 0444,					\
+	},							\
+	.size = _size,						\
+	.read = tdx_metadata_ ## _name ## _show,		\
+}
+
+#define TDX_METADATA_ATTR_SHOW(_name, field_id_name)					\
+static ssize_t tdx_metadata_ ## _name ## _show(struct file *filp, struct kobject *kobj,	\
+					       struct bin_attribute *bin_attr,		\
+					       char *buf, loff_t offset, size_t count)	\
+{											\
+	struct tdsysinfo_struct *sysinfo = &PADDED_STRUCT(tdsysinfo);			\
+											\
+	return memory_read_from_buffer(buf, count, &offset,				\
+				       &sysinfo->_name,					\
+				       sizeof(sysinfo->_name));				\
+}											\
+TDX_METADATA_ATTR(_name, field_id_name, sizeof_field(struct tdsysinfo_struct, _name))
+
+TDX_METADATA_ATTR_SHOW(attributes_fixed0, TDX_METADATA_ATTRIBUTES_FIXED0_NAME);
+TDX_METADATA_ATTR_SHOW(attributes_fixed1, TDX_METADATA_ATTRIBUTES_FIXED1_NAME);
+TDX_METADATA_ATTR_SHOW(xfam_fixed0, TDX_METADATA_XFAM_FIXED0_NAME);
+TDX_METADATA_ATTR_SHOW(xfam_fixed1, TDX_METADATA_XFAM_FIXED1_NAME);
+
+static ssize_t tdx_metadata_num_cpuid_config_show(struct file *filp, struct kobject *kobj,
+						  struct bin_attribute *bin_attr,

arch/x86/virt/vmx/tdx/tdx.c int tdx_enable(void) · line 1426

Quoted 10 lines

arch/x86/virt/vmx/tdx/tdx.c +10 -0

+						  char *buf, loff_t offset, size_t count)
+{
+	struct tdsysinfo_struct *sysinfo = &PADDED_STRUCT(tdsysinfo);
+	/*
+	 * Although tdsysinfo_struct.num_cpuid_config is defined as u32 for
+	 * alignment, TDX 1.5 defines metadata NUM_CONFIG_CPUID as u16.
+	 */
+	u16 tmp = (u16)sysinfo->num_cpuid_config;
+
+	WARN_ON_ONCE(tmp != sysinfo->num_cpuid_config);

Why crash the kernel here?

Quoted 6 lines

arch/x86/virt/vmx/tdx/tdx.c +6 -0

+	return memory_read_from_buffer(buf, count, &offset, &tmp, sizeof(tmp));
+}
+TDX_METADATA_ATTR(num_cpuid_config, TDX_METADATA_NUM_CPUID_CONFIG_NAME, sizeof(u16));
+
+static ssize_t tdx_metadata_cpuid_leaves_show(struct file *filp, struct kobject *kobj,
+					      struct bin_attribute *bin_attr, char *buf,

arch/x86/virt/vmx/tdx/tdx.c int tdx_enable(void) · line 1442

Quoted 10 lines

arch/x86/virt/vmx/tdx/tdx.c +10 -0

+					      loff_t offset, size_t count)
+{
+	struct tdsysinfo_struct *sysinfo = &PADDED_STRUCT(tdsysinfo);
+	ssize_t r;
+	struct tdx_cpuid_config_leaf *tmp;
+	u32 i;
+
+	tmp = kmalloc(bin_attr->size, GFP_KERNEL);
+	if (!tmp)
+		return -ENOMEM;

Why is this allocating and then blindly copying bin_attr->size into @buf? It it either knows that @buf is big enough, no need to allocate, or if it does not know if @buf is big enough then the copy into @tmp offers no protection.

Quoted 57 lines

arch/x86/virt/vmx/tdx/tdx.c +57 -0

+
+	for (i = 0; i < sysinfo->num_cpuid_config; i++) {
+		struct tdx_cpuid_config *c = &sysinfo->cpuid_configs[i];
+		struct tdx_cpuid_config_leaf *leaf = (struct tdx_cpuid_config_leaf *)c;
+
+		memcpy(tmp + i, leaf, sizeof(*leaf));
+	}
+
+	r = memory_read_from_buffer(buf, count, &offset, tmp, bin_attr->size);
+	kfree(tmp);
+	return r;
+}
+
+TDX_METADATA_ATTR(cpuid_leaves, TDX_METADATA_CPUID_LEAVES_NAME, 0);
+
+static ssize_t tdx_metadata_cpuid_values_show(struct file *filp, struct kobject *kobj,
+					      struct bin_attribute *bin_attr, char *buf,
+					      loff_t offset, size_t count)
+{
+	struct tdsysinfo_struct *sysinfo = &PADDED_STRUCT(tdsysinfo);
+	struct tdx_cpuid_config_value *tmp;
+	ssize_t r;
+	u32 i;
+
+	tmp = kmalloc(bin_attr->size, GFP_KERNEL);
+	if (!tmp)
+		return -ENOMEM;
+
+	for (i = 0; i < sysinfo->num_cpuid_config; i++) {
+		struct tdx_cpuid_config *c = &sysinfo->cpuid_configs[i];
+		struct tdx_cpuid_config_value *value = (struct tdx_cpuid_config_value *)&c->eax;
+
+		memcpy(tmp + i, value, sizeof(*value));
+	}
+
+	r = memory_read_from_buffer(buf, count, &offset, tmp, bin_attr->size);
+	kfree(tmp);
+	return r;
+}
+
+TDX_METADATA_ATTR(cpuid_values, TDX_METADATA_CPUID_VALUES_NAME, 0);
+
+static struct bin_attribute *tdx_metadata_attrs[] = {
+	&tdx_metadata_attributes_fixed0,
+	&tdx_metadata_attributes_fixed1,
+	&tdx_metadata_xfam_fixed0,
+	&tdx_metadata_xfam_fixed1,
+	&tdx_metadata_num_cpuid_config,
+	&tdx_metadata_cpuid_leaves,
+	&tdx_metadata_cpuid_values,
+	NULL,
+};
+
+static const struct attribute_group tdx_metadata_attr_group = {
+	.bin_attrs = tdx_metadata_attrs,
+};
+

arch/x86/virt/vmx/tdx/tdx.c int tdx_enable(void) · line 1509

Quoted 10 lines

arch/x86/virt/vmx/tdx/tdx.c +10 -0

+static int tdx_sysfs_init(void)
+{
+	struct tdsysinfo_struct *sysinfo;
+	int ret;
+
+	tdx_kobj = kobject_create_and_add("tdx", firmware_kobj);
+	if (!tdx_kobj) {
+		pr_err("kobject_create_and_add tdx failed\n");
+		return -EINVAL;
+	}

Subsystems, PCI for example [2], are slowly unwinding their usage of dynamic sysfs_create_*() APIs in favor of static attribute groups. Dynamic kobject_create_*() usage is even more of an anti-pattern for new code.

This goes away with static attribute group registration.

[2]: https://lore.kernel.org/linux-pci/20231019200110.GA1410324@bhelgaas/

Xiaoyao Li

2023-03-31 06:59 UTC #54 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

In reply to Zhi Wang

Quoted 11 lines · Zhi Wang

On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:
Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.
Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?

What's the real different of it? For me, it's just renaming KVM_MEMORY_ENCRYPT_OP to KVM_TDX_DEV_OP and maybe add some error message if the IOCTL is issued for AMD plaform.

Zhi Wang

2023-03-31 12:32 UTC #55 lore raw

Re: [PATCH v13 016/113] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters

Quoted 9 lines · On Fri, 31 Mar 2023 14:59:18 +0800 Xiaoyao Li <[email protected]> wrote:

Quoted 8 lines · Zhi Wang
On Sun, 12 Mar 2023 10:55:40 -0700
[email protected] wrote:
Does this have to be a new generic ioctl with a dedicated new x86_ops? SNP does not use it at all and all the system-scoped ioctl of SNP going through the CCP driver. So getting system-scope information of TDX/SNP will end up differently.

Quoted 8 lines

Quoted 3 lines
Any thought, Sean? Moving getting SNP system-wide information to KVM dev ioctl seems not ideal and TDX does not have a dedicated driver like CCP. Maybe make this ioctl TDX-specific? KVM_TDX_DEV_OP?
What's the real different of it? For me, it's just renaming KVM_MEMORY_ENCRYPT_OP to KVM_TDX_DEV_OP and maybe add some error message if the IOCTL is issued for AMD plaform.

Hi:

The ioctl is the API for the userspace. The purpose is to be orthogonal, avoid confusion and reflect its nature. A "generic" name with only one implementation is fine in the early design. But if the other implementation at the same level is pretty sure not going to use it, then the abstraction, which is only abstracted for one implementation, is just confusing.

The possible strategies are:

1) Re-factor the other implementation to fit the current abstraction.
2) Give up the abstraction. Go "specific".

For 1), it seems not realistic due to the efforts of re-factoring the SEV driver.

For 2), there can be several ways: a. renaming it, let the name reflect its nature. IMO, KVM_TDX_DEV_OP is not ideal as well, but I don't have a better one. b. moving it to a proper layer of the implementation. But it is also not realistic to have a "TDX" driver because of it. That's why I am torn here.

[email protected]

author

2023-03-12 17:59 UTC #56 lore raw

PATCH v13 17/113 KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl

From: Isaku Yamahata <[email protected]>

Add a place holder function for TDX specific VM-scoped ioctl as mem_enc_op. TDX specific sub-commands will be added to retrieve/pass TDX specific parameters. Make mem_enc_ioctl non-optional as it's not optional now.

KVM_MEMORY_ENCRYPT_OP was introduced for VM-scoped operations specific for guest state-protected VM. It defined subcommands for technology-specific operations under KVM_MEMORY_ENCRYPT_OP. Despite its name, the subcommands are not limited to memory encryption, but various technology-specific operations are defined. It's natural to repurpose KVM_MEMORY_ENCRYPT_OP for TDX specific operations and define subcommands.

TDX requires VM-scoped TDX-specific operations for device model, for example, qemu. Getting system-wide parameters, TDX-specific VM initialization.

---
 arch/x86/include/asm/kvm-x86-ops.h |  2 +-
 arch/x86/kvm/vmx/main.c            |  9 +++++++++
 arch/x86/kvm/vmx/tdx.c             | 26 ++++++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h         |  4 ++++
 arch/x86/kvm/x86.c                 |  4 ----
 5 files changed, 40 insertions(+), 5 deletions(-)

arch/x86/include/asm/kvm-x86-ops.h +1 -1

index b46dcac078b2..58fbaa05fc8c 100644
@@ -117,7 +117,7 @@ KVM_X86_OP(enter_smm)
 KVM_X86_OP(leave_smm)
 KVM_X86_OP(enable_smi_window)
 #endif
-KVM_X86_OP_OPTIONAL(dev_mem_enc_ioctl)
+KVM_X86_OP(dev_mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_register_region)
 KVM_X86_OP_OPTIONAL(mem_enc_unregister_region)

arch/x86/kvm/vmx/main.c +9 -0

index 6a5d0c7a2950..8ddd263eeabc 100644
@@ -50,6 +50,14 @@ static int vt_vm_init(struct kvm *kvm)
 	return vmx_vm_init(kvm);
 }
 
+static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
+{
+	if (!is_td(kvm))
+		return -ENOTTY;
+
+	return tdx_vm_ioctl(kvm, argp);
+}
+
 #define VMX_REQUIRED_APICV_INHIBITS		       \
 (						       \
        BIT(APICV_INHIBIT_REASON_DISABLE)|	       \

@@ -203,6 +211,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
 
 	.dev_mem_enc_ioctl = tdx_dev_ioctl,
+	.mem_enc_ioctl = vt_mem_enc_ioctl,
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {

arch/x86/kvm/vmx/tdx.c +26 -0

index b59d3081d061..d759028a698e 100644
@@ -72,6 +72,32 @@ int tdx_dev_ioctl(void __user *argp)
 	return 0;
 }
 
+int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)
+{
+	struct kvm_tdx_cmd tdx_cmd;
+	int r;
+
+	if (copy_from_user(&tdx_cmd, argp, sizeof(struct kvm_tdx_cmd)))
+		return -EFAULT;
+	if (tdx_cmd.error || tdx_cmd.unused)
+		return -EINVAL;
+
+	mutex_lock(&kvm->lock);
+
+	switch (tdx_cmd.id) {
+	default:
+		r = -EINVAL;
+		goto out;
+	}
+
+	if (copy_to_user(argp, &tdx_cmd, sizeof(struct kvm_tdx_cmd)))
+		r = -EFAULT;
+
+out:
+	mutex_unlock(&kvm->lock);
+	return r;
+}
+
 static int __init tdx_module_setup(void)
 {
 	const struct tdsysinfo_struct *tdsysinfo;

arch/x86/kvm/vmx/x86_ops.h +4 -0

index 78c5537e23a1..c70749114e9e 100644
@@ -142,11 +142,15 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
 int tdx_dev_ioctl(void __user *argp);
+
+int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
+
+static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

arch/x86/kvm/x86.c +0 -4

index a3dc32e33aca..8687623929c3 100644
@@ -6978,10 +6978,6 @@ long kvm_arch_vm_ioctl(struct file *filp,
 		goto out;
 	}
 	case KVM_MEMORY_ENCRYPT_OP: {
-		r = -ENOTTY;
-		if (!kvm_x86_ops.mem_enc_ioctl)
-			goto out;
-
 		r = static_call(kvm_x86_mem_enc_ioctl)(kvm, argp);
 		break;
 	}

Signature · 1 line

2.25.1

Zhi Wang

2023-03-25 09:05 UTC #57 lore raw

Re: [PATCH v13 017/113] KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl

Quoted 5 lines · On Sun, 12 Mar 2023 10:55:41 -0700 [email protected] wrote:

From: Isaku Yamahata <[email protected]>
Add a place holder function for TDX specific VM-scoped ioctl as mem_enc_op. TDX specific sub-commands will be added to retrieve/pass TDX specific parameters. Make mem_enc_ioctl non-optional as it's not optional now.

^ it is optional now.

Quoted 11 lines

KVM_MEMORY_ENCRYPT_OP was introduced for VM-scoped operations specific for guest state-protected VM. It defined subcommands for technology-specific operations under KVM_MEMORY_ENCRYPT_OP. Despite its name, the subcommands are not limited to memory encryption, but various technology-specific operations are defined. It's natural to repurpose KVM_MEMORY_ENCRYPT_OP for TDX specific operations and define subcommands.
TDX requires VM-scoped TDX-specific operations for device model, for example, qemu. Getting system-wide parameters, TDX-specific VM initialization.

^ Above two paragraphs are the "background" paragraphs, they should be in the beginning of the patch comments.

Quoted 15 lines

---
 arch/x86/include/asm/kvm-x86-ops.h |  2 +-
 arch/x86/kvm/vmx/main.c            |  9 +++++++++
 arch/x86/kvm/vmx/tdx.c             | 26 ++++++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h         |  4 ++++
 arch/x86/kvm/x86.c                 |  4 ----
 5 files changed, 40 insertions(+), 5 deletions(-)

arch/x86/include/asm/kvm-x86-ops.h +0 -0

index b46dcac078b2..58fbaa05fc8c 100644

arch/x86/include/asm/kvm-x86-ops.h KVM_X86_OP(enter_smm) · line 120

Quoted 5 lines

arch/x86/include/asm/kvm-x86-ops.h +1 -1

@@ -117,7 +117,7 @@ KVM_X86_OP(enter_smm)
 KVM_X86_OP(leave_smm)
 KVM_X86_OP(enable_smi_window)
 #endif
-KVM_X86_OP_OPTIONAL(dev_mem_enc_ioctl)
+KVM_X86_OP(dev_mem_enc_ioctl)

I guess it should be KVM_X86_OP(mem_enc_ioctl) here.

Quoted 102 lines

arch/x86/include/asm/kvm-x86-ops.h +0 -0

 KVM_X86_OP_OPTIONAL(mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_register_region)
 KVM_X86_OP_OPTIONAL(mem_enc_unregister_region)

arch/x86/kvm/vmx/main.c +9 -0

index 6a5d0c7a2950..8ddd263eeabc 100644
@@ -50,6 +50,14 @@ static int vt_vm_init(struct kvm *kvm)
 	return vmx_vm_init(kvm);
 }
 
+static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
+{
+	if (!is_td(kvm))
+		return -ENOTTY;
+
+	return tdx_vm_ioctl(kvm, argp);
+}
+
 #define VMX_REQUIRED_APICV_INHIBITS		       \
 (						       \
        BIT(APICV_INHIBIT_REASON_DISABLE)|	       \

@@ -203,6 +211,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
 
 	.dev_mem_enc_ioctl = tdx_dev_ioctl,
+	.mem_enc_ioctl = vt_mem_enc_ioctl,
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {

arch/x86/kvm/vmx/tdx.c +26 -0

index b59d3081d061..d759028a698e 100644
@@ -72,6 +72,32 @@ int tdx_dev_ioctl(void __user *argp)
 	return 0;
 }
 
+int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)
+{
+	struct kvm_tdx_cmd tdx_cmd;
+	int r;
+
+	if (copy_from_user(&tdx_cmd, argp, sizeof(struct kvm_tdx_cmd)))
+		return -EFAULT;
+	if (tdx_cmd.error || tdx_cmd.unused)
+		return -EINVAL;
+
+	mutex_lock(&kvm->lock);
+
+	switch (tdx_cmd.id) {
+	default:
+		r = -EINVAL;
+		goto out;
+	}
+
+	if (copy_to_user(argp, &tdx_cmd, sizeof(struct kvm_tdx_cmd)))
+		r = -EFAULT;
+
+out:
+	mutex_unlock(&kvm->lock);
+	return r;
+}
+
 static int __init tdx_module_setup(void)
 {
 	const struct tdsysinfo_struct *tdsysinfo;

arch/x86/kvm/vmx/x86_ops.h +4 -0

index 78c5537e23a1..c70749114e9e 100644
@@ -142,11 +142,15 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
 int tdx_dev_ioctl(void __user *argp);
+
+int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
+
+static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

arch/x86/kvm/x86.c +0 -4

index a3dc32e33aca..8687623929c3 100644
@@ -6978,10 +6978,6 @@ long kvm_arch_vm_ioctl(struct file *filp,
 		goto out;
 	}
 	case KVM_MEMORY_ENCRYPT_OP: {
-		r = -ENOTTY;
-		if (!kvm_x86_ops.mem_enc_ioctl)
-			goto out;
-
 		r = static_call(kvm_x86_mem_enc_ioctl)(kvm, argp);
 		break;
 	}

Isaku Yamahata

2023-03-29 23:23 UTC #58 lore raw

Re: [PATCH v13 017/113] KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl

Quoted 4 lines · Zhi Wang

Quoted 4 lines
arch/x86/include/asm/kvm-x86-ops.h +0 -0
index b46dcac078b2..58fbaa05fc8c 100644

Quoted 8 lines

arch/x86/include/asm/kvm-x86-ops.h KVM_X86_OP(enter_smm) · line 120
Quoted 6 lines
arch/x86/include/asm/kvm-x86-ops.h +1 -1
@@ -117,7 +117,7 @@ KVM_X86_OP(enter_smm)
 KVM_X86_OP(leave_smm)
 KVM_X86_OP(enable_smi_window)
 #endif
-KVM_X86_OP_OPTIONAL(dev_mem_enc_ioctl)
+KVM_X86_OP(dev_mem_enc_ioctl)
I guess it should be KVM_X86_OP(mem_enc_ioctl) here.

Yes, thanks for catching it.

Signature · 1 line

Isaku Yamahata <[email protected]>

[email protected]

author

2023-03-12 17:59 UTC #59 lore raw

PATCH v13 18/113 KVM: x86, tdx: Make KVM_CAP_MAX_VCPUS backend specific

From: Isaku Yamahata <[email protected]>

TDX has its own limitation on the maximum number of vcpus that the guest can accommodate. Allow x86 kvm backend to implement its own KVM_ENABLE_CAP handler and implement TDX backend for KVM_CAP_MAX_VCPUS. user space VMM, e.g. qemu, can specify its value instead of KVM_MAX_VCPUS.

---
 arch/x86/include/asm/kvm-x86-ops.h |  2 ++
 arch/x86/include/asm/kvm_host.h    |  2 ++
 arch/x86/kvm/vmx/main.c            | 22 ++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.c             | 30 ++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.h             |  3 +++
 arch/x86/kvm/vmx/x86_ops.h         |  2 ++
 arch/x86/kvm/x86.c                 |  4 ++++
 7 files changed, 65 insertions(+)

arch/x86/include/asm/kvm-x86-ops.h +2 -0

index 58fbaa05fc8c..7522c193f2b4 100644
@@ -21,6 +21,8 @@ KVM_X86_OP(hardware_unsetup)
 KVM_X86_OP(has_emulated_msr)
 KVM_X86_OP(vcpu_after_set_cpuid)
 KVM_X86_OP(is_vm_type_supported)
+KVM_X86_OP_OPTIONAL(max_vcpus);
+KVM_X86_OP_OPTIONAL(vm_enable_cap)
 KVM_X86_OP(vm_init)
 KVM_X86_OP_OPTIONAL(vm_destroy)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)

arch/x86/include/asm/kvm_host.h +2 -0

index 49e3ca89aced..d98d61e5213d 100644
@@ -1562,7 +1562,9 @@ struct kvm_x86_ops {
 	void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu);
 
 	bool (*is_vm_type_supported)(unsigned long vm_type);
+	int (*max_vcpus)(struct kvm *kvm);
 	unsigned int vm_size;
+	int (*vm_enable_cap)(struct kvm *kvm, struct kvm_enable_cap *cap);
 	int (*vm_init)(struct kvm *kvm);
 	void (*vm_destroy)(struct kvm *kvm);

arch/x86/kvm/vmx/main.c +22 -0

index 8ddd263eeabc..68bb320d0b6d 100644
@@ -6,6 +6,7 @@
 #include "nested.h"
 #include "pmu.h"
 #include "tdx.h"
+#include "tdx_arch.h"
 
 static bool enable_tdx __ro_after_init;
 module_param_named(tdx, enable_tdx, bool, 0444);

@@ -29,6 +30,17 @@ static bool vt_is_vm_type_supported(unsigned long type)
 		(enable_tdx && tdx_is_vm_type_supported(type));
 }
 
+static int vt_max_vcpus(struct kvm *kvm)
+{
+	if (!kvm)
+		return KVM_MAX_VCPUS;
+
+	if (is_td(kvm))
+		return min3(kvm->max_vcpus, KVM_MAX_VCPUS, TDX_MAX_VCPUS);
+
+	return kvm->max_vcpus;
+}
+
 static __init int vt_hardware_setup(void)
 {
 	int ret;

@@ -42,6 +54,14 @@ static __init int vt_hardware_setup(void)
 	return 0;
 }
 
+static int vt_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
+{
+	if (is_td(kvm))
+		return tdx_vm_enable_cap(kvm, cap);
+
+	return -EINVAL;
+}
+
 static int vt_vm_init(struct kvm *kvm)
 {
 	if (is_td(kvm))

@@ -81,7 +101,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.has_emulated_msr = vmx_has_emulated_msr,
 
 	.is_vm_type_supported = vt_is_vm_type_supported,
+	.max_vcpus = vt_max_vcpus,
 	.vm_size = sizeof(struct kvm_vmx),
+	.vm_enable_cap = vt_vm_enable_cap,
 	.vm_init = vt_vm_init,
 	.vm_destroy = vmx_vm_destroy,

arch/x86/kvm/vmx/tdx.c +30 -0

index d759028a698e..8b02e605cfb5 100644
@@ -16,6 +16,36 @@
 		offsetof(struct tdsysinfo_struct, cpuid_configs))	\
 		/ sizeof(struct tdx_cpuid_config))
 
+int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
+{
+	int r;
+
+	switch (cap->cap) {
+	case KVM_CAP_MAX_VCPUS: {
+		if (cap->flags || cap->args[0] == 0)
+			return -EINVAL;
+		if (cap->args[0] > KVM_MAX_VCPUS)
+			return -E2BIG;
+		if (cap->args[0] > TDX_MAX_VCPUS)
+			return -E2BIG;
+
+		mutex_lock(&kvm->lock);
+		if (kvm->created_vcpus)
+			r = -EBUSY;
+		else {
+			kvm->max_vcpus = cap->args[0];
+			r = 0;
+		}
+		mutex_unlock(&kvm->lock);
+		break;
+	}
+	default:
+		r = -EINVAL;
+		break;
+	}
+	return r;
+}
+
 int tdx_hardware_enable(void)
 {
 	return tdx_cpu_enable();

arch/x86/kvm/vmx/tdx.h +3 -0

index 2210c8c1e893..3860aa351bd9 100644
@@ -3,6 +3,9 @@
 #define __KVM_X86_TDX_H
 
 #ifdef CONFIG_INTEL_TDX_HOST
+
+#include "tdx_ops.h"
+
 struct kvm_tdx {
 	struct kvm kvm;
 	/* TDX specific members follow. */

arch/x86/kvm/vmx/x86_ops.h +2 -0

index c70749114e9e..8118647aa8ca 100644
@@ -143,6 +143,7 @@ int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
 int tdx_dev_ioctl(void __user *argp);
 
+int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }

@@ -150,6 +151,7 @@ static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
 
+static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { return -EINVAL; };
 static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
 #endif

arch/x86/kvm/x86.c +4 -0

index 8687623929c3..7b02dd40ef21 100644
@@ -4500,6 +4500,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		break;
 	case KVM_CAP_MAX_VCPUS:
 		r = KVM_MAX_VCPUS;
+		if (kvm_x86_ops.max_vcpus)
+			r = static_call(kvm_x86_max_vcpus)(kvm);
 		break;
 	case KVM_CAP_MAX_VCPU_ID:
 		r = KVM_MAX_VCPU_IDS;

@@ -6439,6 +6441,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		break;
 	default:
 		r = -EINVAL;
+		if (kvm_x86_ops.vm_enable_cap)
+			r = static_call(kvm_x86_vm_enable_cap)(kvm, cap);
 		break;
 	}
 	return r;

Signature · 1 line

2.25.1

Zhi Wang

2023-03-25 18:13 UTC #60 lore raw

Re: [PATCH v13 018/113] KVM: x86, tdx: Make KVM_CAP_MAX_VCPUS backend specific

Quoted 7 lines · On Sun, 12 Mar 2023 10:55:42 -0700 [email protected] wrote:

From: Isaku Yamahata <[email protected]>
TDX has its own limitation on the maximum number of vcpus that the guest can accommodate. Allow x86 kvm backend to implement its own KVM_ENABLE_CAP handler and implement TDX backend for KVM_CAP_MAX_VCPUS. user space VMM, e.g. qemu, can specify its value instead of KVM_MAX_VCPUS.

I think enabling the cap here is actually "configuring the cap". KVM_CAP_MAX _VCPUS is actually always enabled whether userspace enables it or not. It would be nice to configure of the max VCPUS in kvm_arch_vm_ioctl() where routines of configuring a VM should belong. E.g. KVM_SET_MAX_VCPUS.

Quoted 190 lines

---
 arch/x86/include/asm/kvm-x86-ops.h |  2 ++
 arch/x86/include/asm/kvm_host.h    |  2 ++
 arch/x86/kvm/vmx/main.c            | 22 ++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.c             | 30 ++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.h             |  3 +++
 arch/x86/kvm/vmx/x86_ops.h         |  2 ++
 arch/x86/kvm/x86.c                 |  4 ++++
 7 files changed, 65 insertions(+)

arch/x86/include/asm/kvm-x86-ops.h +2 -0

index 58fbaa05fc8c..7522c193f2b4 100644
@@ -21,6 +21,8 @@ KVM_X86_OP(hardware_unsetup)
 KVM_X86_OP(has_emulated_msr)
 KVM_X86_OP(vcpu_after_set_cpuid)
 KVM_X86_OP(is_vm_type_supported)
+KVM_X86_OP_OPTIONAL(max_vcpus);
+KVM_X86_OP_OPTIONAL(vm_enable_cap)
 KVM_X86_OP(vm_init)
 KVM_X86_OP_OPTIONAL(vm_destroy)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)

arch/x86/include/asm/kvm_host.h +2 -0

index 49e3ca89aced..d98d61e5213d 100644
@@ -1562,7 +1562,9 @@ struct kvm_x86_ops {
 	void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu);
 
 	bool (*is_vm_type_supported)(unsigned long vm_type);
+	int (*max_vcpus)(struct kvm *kvm);
 	unsigned int vm_size;
+	int (*vm_enable_cap)(struct kvm *kvm, struct kvm_enable_cap *cap);
 	int (*vm_init)(struct kvm *kvm);
 	void (*vm_destroy)(struct kvm *kvm);

arch/x86/kvm/vmx/main.c +22 -0

index 8ddd263eeabc..68bb320d0b6d 100644
@@ -6,6 +6,7 @@
 #include "nested.h"
 #include "pmu.h"
 #include "tdx.h"
+#include "tdx_arch.h"
 
 static bool enable_tdx __ro_after_init;
 module_param_named(tdx, enable_tdx, bool, 0444);

@@ -29,6 +30,17 @@ static bool vt_is_vm_type_supported(unsigned long type)
 		(enable_tdx && tdx_is_vm_type_supported(type));
 }
 
+static int vt_max_vcpus(struct kvm *kvm)
+{
+	if (!kvm)
+		return KVM_MAX_VCPUS;
+
+	if (is_td(kvm))
+		return min3(kvm->max_vcpus, KVM_MAX_VCPUS, TDX_MAX_VCPUS);
+
+	return kvm->max_vcpus;
+}
+
 static __init int vt_hardware_setup(void)
 {
 	int ret;

@@ -42,6 +54,14 @@ static __init int vt_hardware_setup(void)
 	return 0;
 }
 
+static int vt_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
+{
+	if (is_td(kvm))
+		return tdx_vm_enable_cap(kvm, cap);
+
+	return -EINVAL;
+}
+
 static int vt_vm_init(struct kvm *kvm)
 {
 	if (is_td(kvm))

@@ -81,7 +101,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.has_emulated_msr = vmx_has_emulated_msr,
 
 	.is_vm_type_supported = vt_is_vm_type_supported,
+	.max_vcpus = vt_max_vcpus,
 	.vm_size = sizeof(struct kvm_vmx),
+	.vm_enable_cap = vt_vm_enable_cap,
 	.vm_init = vt_vm_init,
 	.vm_destroy = vmx_vm_destroy,

arch/x86/kvm/vmx/tdx.c +30 -0

index d759028a698e..8b02e605cfb5 100644
@@ -16,6 +16,36 @@
 		offsetof(struct tdsysinfo_struct, cpuid_configs))	\
 		/ sizeof(struct tdx_cpuid_config))
 
+int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
+{
+	int r;
+
+	switch (cap->cap) {
+	case KVM_CAP_MAX_VCPUS: {
+		if (cap->flags || cap->args[0] == 0)
+			return -EINVAL;
+		if (cap->args[0] > KVM_MAX_VCPUS)
+			return -E2BIG;
+		if (cap->args[0] > TDX_MAX_VCPUS)
+			return -E2BIG;
+
+		mutex_lock(&kvm->lock);
+		if (kvm->created_vcpus)
+			r = -EBUSY;
+		else {
+			kvm->max_vcpus = cap->args[0];
+			r = 0;
+		}
+		mutex_unlock(&kvm->lock);
+		break;
+	}
+	default:
+		r = -EINVAL;
+		break;
+	}
+	return r;
+}
+
 int tdx_hardware_enable(void)
 {
 	return tdx_cpu_enable();

arch/x86/kvm/vmx/tdx.h +3 -0

index 2210c8c1e893..3860aa351bd9 100644
@@ -3,6 +3,9 @@
 #define __KVM_X86_TDX_H
 
 #ifdef CONFIG_INTEL_TDX_HOST
+
+#include "tdx_ops.h"
+
 struct kvm_tdx {
 	struct kvm kvm;
 	/* TDX specific members follow. */

arch/x86/kvm/vmx/x86_ops.h +2 -0

index c70749114e9e..8118647aa8ca 100644
@@ -143,6 +143,7 @@ int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
 int tdx_dev_ioctl(void __user *argp);
 
+int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }

@@ -150,6 +151,7 @@ static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
 
+static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { return -EINVAL; };
 static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
 #endif

arch/x86/kvm/x86.c +4 -0

index 8687623929c3..7b02dd40ef21 100644
@@ -4500,6 +4500,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		break;
 	case KVM_CAP_MAX_VCPUS:
 		r = KVM_MAX_VCPUS;
+		if (kvm_x86_ops.max_vcpus)
+			r = static_call(kvm_x86_max_vcpus)(kvm);
 		break;
 	case KVM_CAP_MAX_VCPU_ID:
 		r = KVM_MAX_VCPU_IDS;

@@ -6439,6 +6441,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		break;
 	default:
 		r = -EINVAL;
+		if (kvm_x86_ops.vm_enable_cap)
+			r = static_call(kvm_x86_vm_enable_cap)(kvm, cap);
 		break;
 	}
 	return r;

Isaku Yamahata

2023-03-29 23:33 UTC #61 lore raw

Re: [PATCH v13 018/113] KVM: x86, tdx: Make KVM_CAP_MAX_VCPUS backend specific

Quoted 11 lines · Zhi Wang

Quoted 7 lines · On Sun, 12 Mar 2023 10:55:42 -0700 [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX has its own limitation on the maximum number of vcpus that the guest can accommodate. Allow x86 kvm backend to implement its own KVM_ENABLE_CAP handler and implement TDX backend for KVM_CAP_MAX_VCPUS. user space VMM, e.g. qemu, can specify its value instead of KVM_MAX_VCPUS.

Quoted 4 lines

I think enabling the cap here is actually "configuring the cap". KVM_CAP_MAX _VCPUS is actually always enabled whether userspace enables it or not. It would be nice to configure of the max VCPUS in kvm_arch_vm_ioctl() where routines of configuring a VM should belong. E.g. KVM_SET_MAX_VCPUS.

I'm not sure I understand your point. Although KVM_ENABLE_CAP sounds like only on/off feature, but it isn't. It's also used to set parameters. For example, KVM_CAP_MAX_VCPU_ID.

KVM_SET_XXX is for run time feature. But the maxium number of vcpus is not runtime changable. Set it at vm creation.

Signature · 1 line

Isaku Yamahata <[email protected]>

Zhi Wang

2023-04-01 07:00 UTC #62 lore raw

Re: [PATCH v13 018/113] KVM: x86, tdx: Make KVM_CAP_MAX_VCPUS backend specific

Quoted 14 lines · On Wed, 29 Mar 2023 16:32:58 -0700 Isaku Yamahata <[email protected]>…

Quoted 11 lines · Zhi Wang
Quoted 7 lines · On Sun, 12 Mar 2023 10:55:42 -0700 [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX has its own limitation on the maximum number of vcpus that the guest can accommodate. Allow x86 kvm backend to implement its own KVM_ENABLE_CAP handler and implement TDX backend for KVM_CAP_MAX_VCPUS. user space VMM, e.g. qemu, can specify its value instead of KVM_MAX_VCPUS.

Quoted 9 lines

Quoted 4 lines
I think enabling the cap here is actually "configuring the cap". KVM_CAP_MAX _VCPUS is actually always enabled whether userspace enables it or not. It would be nice to configure of the max VCPUS in kvm_arch_vm_ioctl() where routines of configuring a VM should belong. E.g. KVM_SET_MAX_VCPUS.
I'm not sure I understand your point. Although KVM_ENABLE_CAP sounds like only on/off feature, but it isn't. It's also used to set parameters. For example, KVM_CAP_MAX_VCPU_ID.

Yes, I understand your point. But what has been there might not be right as well. That doesn't smell right as "enable" is for something which is previously "disabled". I understand that there can be some caps require configuration when being enabled. But later, for those caps which don't have "on/off" state, KVM_ENABLE_CAP doesn't actually enable a feature, it is just configuring a feature. It seems like the KVM_ENABLE_CAP has been mis-used little by little in the history. Also, I don't find KVM_DISABLE_CAP accordingly. So KVM_ENABLE_CAP is actually used as "KVM_SET_CAP".

I realize it is not realistic to solve the problem in this patch series. You can keep the current approach.

Quoted 2 lines

KVM_SET_XXX is for run time feature. But the maxium number of vcpus is not runtime changable. Set it at vm creation.

[email protected]

author

2023-03-12 17:59 UTC #63 lore raw

PATCH v13 19/113 KVM: TDX: create/destroy VM structure

From: Isaku Yamahata <[email protected]>

As the first step to create TDX guest, create/destroy VM struct. Assign TDX private Host Key ID (HKID) to the TDX guest for memory encryption and allocate extra pages for the TDX guest. On destruction, free allocated pages, and HKID.

Before tearing down private page tables, TDX requires some resources of the guest TD to be destroyed (i.e. HKID must have been reclaimed, etc). Add flush_shadow_all_private callback before tearing down private page tables for it.

Add vm_free() of kvm_x86_ops hook at the end of kvm_arch_destroy_vm() because some per-VM TDX resources, e.g. TDR, need to be freed after other TDX resources, e.g. HKID, were freed.

Co-developed-by Kai Huang <[email protected]>

Signed-off-by Kai Huang <[email protected]>

Signed-off-by Sean Christopherson <[email protected]>

---
Changes v11 -> v12:
- use cpu_feature_enabled().

Changes v10 -> v11:
- Fix doule free in tdx_vm_free() by setting NULL.
- replace struct tdx_td_page tdr and tdcs from struct kvm_tdx with unsigned long

---
 arch/x86/include/asm/kvm-x86-ops.h |   2 +
 arch/x86/include/asm/kvm_host.h    |   2 +
 arch/x86/kvm/vmx/main.c            |  35 ++-
 arch/x86/kvm/vmx/tdx.c             | 442 ++++++++++++++++++++++++++++-
 arch/x86/kvm/vmx/tdx.h             |   6 +-
 arch/x86/kvm/vmx/x86_ops.h         |   9 +
 arch/x86/kvm/x86.c                 |   8 +
 7 files changed, 498 insertions(+), 6 deletions(-)

arch/x86/include/asm/kvm-x86-ops.h +2 -0

index 7522c193f2b4..c30d2d2ad686 100644
@@ -24,7 +24,9 @@ KVM_X86_OP(is_vm_type_supported)
 KVM_X86_OP_OPTIONAL(max_vcpus);
 KVM_X86_OP_OPTIONAL(vm_enable_cap)
 KVM_X86_OP(vm_init)
+KVM_X86_OP_OPTIONAL(flush_shadow_all_private)
 KVM_X86_OP_OPTIONAL(vm_destroy)
+KVM_X86_OP_OPTIONAL(vm_free)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)
 KVM_X86_OP(vcpu_create)
 KVM_X86_OP(vcpu_free)

arch/x86/include/asm/kvm_host.h +2 -0

index d98d61e5213d..15f7c0d45082 100644
@@ -1566,7 +1566,9 @@ struct kvm_x86_ops {
 	unsigned int vm_size;
 	int (*vm_enable_cap)(struct kvm *kvm, struct kvm_enable_cap *cap);
 	int (*vm_init)(struct kvm *kvm);
+	void (*flush_shadow_all_private)(struct kvm *kvm);
 	void (*vm_destroy)(struct kvm *kvm);
+	void (*vm_free)(struct kvm *kvm);
 
 	/* Create, but do not attach this VCPU */
 	int (*vcpu_precreate)(struct kvm *kvm);

arch/x86/kvm/vmx/main.c +32 -3

index 68bb320d0b6d..08ed6fc8cc3d 100644
@@ -62,14 +62,41 @@ static int vt_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 	return -EINVAL;
 }
 
+static void vt_hardware_unsetup(void)
+{
+	if (enable_tdx)
+		tdx_hardware_unsetup();
+	vmx_hardware_unsetup();
+}
+
 static int vt_vm_init(struct kvm *kvm)
 {
 	if (is_td(kvm))
-		return -EOPNOTSUPP;	/* Not ready to create guest TD yet. */
+		return tdx_vm_init(kvm);
 
 	return vmx_vm_init(kvm);
 }
 
+static void vt_flush_shadow_all_private(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		tdx_mmu_release_hkid(kvm);
+}
+
+static void vt_vm_destroy(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		return;
+
+	vmx_vm_destroy(kvm);
+}
+
+static void vt_vm_free(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		tdx_vm_free(kvm);
+}
+
 static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
 	if (!is_td(kvm))

@@ -94,7 +121,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 
 	.check_processor_compatibility = vmx_check_processor_compat,
 
-	.hardware_unsetup = vmx_hardware_unsetup,
+	.hardware_unsetup = vt_hardware_unsetup,
 
 	.hardware_enable = vt_hardware_enable,
 	.hardware_disable = vmx_hardware_disable,

@@ -105,7 +132,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.vm_size = sizeof(struct kvm_vmx),
 	.vm_enable_cap = vt_vm_enable_cap,
 	.vm_init = vt_vm_init,
-	.vm_destroy = vmx_vm_destroy,
+	.flush_shadow_all_private = vt_flush_shadow_all_private,
+	.vm_destroy = vt_vm_destroy,
+	.vm_free = vt_vm_free,
 
 	.vcpu_precreate = vmx_vcpu_precreate,
 	.vcpu_create = vmx_vcpu_create,

arch/x86/kvm/vmx/tdx.c +440 -2

489 lines not on this page · raw message

arch/x86/kvm/vmx/tdx.h +5 -1

index 3860aa351bd9..4b790503e43e 100644
@@ -8,7 +8,11 @@
 
 struct kvm_tdx {
 	struct kvm kvm;
-	/* TDX specific members follow. */
+
+	unsigned long tdr_pa;
+	unsigned long *tdcs_pa;
+
+	int hkid;
 };
 
 struct vcpu_tdx {

arch/x86/kvm/vmx/x86_ops.h +9 -0

index 8118647aa8ca..8f87eeeab97c 100644
@@ -139,19 +139,28 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
 #ifdef CONFIG_INTEL_TDX_HOST
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
+void tdx_hardware_unsetup(void);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
 int tdx_dev_ioctl(void __user *argp);
 
 int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);
+int tdx_vm_init(struct kvm *kvm);
+void tdx_mmu_release_hkid(struct kvm *kvm);
+void tdx_vm_free(struct kvm *kvm);
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
+static inline void tdx_hardware_unsetup(void) {}
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
 
 static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { return -EINVAL; };
+static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; }
+static inline void tdx_mmu_release_hkid(struct kvm *kvm) {}
+static inline void tdx_flush_shadow_all_private(struct kvm *kvm) {}
+static inline void tdx_vm_free(struct kvm *kvm) {}
 static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
 #endif

arch/x86/kvm/x86.c +8 -0

index 7b02dd40ef21..89fd8074c20c 100644
@@ -12398,6 +12398,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 	kvm_page_track_cleanup(kvm);
 	kvm_xen_destroy_vm(kvm);
 	kvm_hv_destroy_vm(kvm);
+	static_call_cond(kvm_x86_vm_free)(kvm);
 }
 
 static void memslot_rmap_free(struct kvm_memory_slot *slot)

@@ -12710,6 +12711,13 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
 
 void kvm_arch_flush_shadow_all(struct kvm *kvm)
 {
+	/*
+	 * kvm_mmu_zap_all() zaps both private and shared page tables.  Before
+	 * tearing down private page tables, TDX requires some TD resources to
+	 * be destroyed (i.e. keyID must have been reclaimed, etc).  Invoke
+	 * kvm_x86_flush_shadow_all_private() for this.
+	 */
+	static_call_cond(kvm_x86_flush_shadow_all_private)(kvm);
 	kvm_mmu_zap_all(kvm);
 }

Signature · 1 line

2.25.1

Wang, Lei

2023-03-24 02:27 UTC #64 lore raw

Re: [PATCH v13 019/113] KVM: TDX: create/destroy VM structure

Quoted 489 lines · On 3/13/2023 1:55, [email protected] wrote:

From: Isaku Yamahata <[email protected]>

Add vm_free() of kvm_x86_ops hook at the end of kvm_arch_destroy_vm() because some per-VM TDX resources, e.g. TDR, need to be freed after other TDX resources, e.g. HKID, were freed.

Co-developed-by Kai Huang <[email protected]>

Signed-off-by Kai Huang <[email protected]>

Signed-off-by Sean Christopherson <[email protected]>

---
Changes v11 -> v12:
- use cpu_feature_enabled().

Changes v10 -> v11:
- Fix doule free in tdx_vm_free() by setting NULL.
- replace struct tdx_td_page tdr and tdcs from struct kvm_tdx with unsigned long

---
 arch/x86/include/asm/kvm-x86-ops.h |   2 +
 arch/x86/include/asm/kvm_host.h    |   2 +
 arch/x86/kvm/vmx/main.c            |  35 ++-
 arch/x86/kvm/vmx/tdx.c             | 442 ++++++++++++++++++++++++++++-
 arch/x86/kvm/vmx/tdx.h             |   6 +-
 arch/x86/kvm/vmx/x86_ops.h         |   9 +
 arch/x86/kvm/x86.c                 |   8 +
 7 files changed, 498 insertions(+), 6 deletions(-)

arch/x86/include/asm/kvm-x86-ops.h +2 -0

index 7522c193f2b4..c30d2d2ad686 100644
@@ -24,7 +24,9 @@ KVM_X86_OP(is_vm_type_supported)
 KVM_X86_OP_OPTIONAL(max_vcpus);
 KVM_X86_OP_OPTIONAL(vm_enable_cap)
 KVM_X86_OP(vm_init)
+KVM_X86_OP_OPTIONAL(flush_shadow_all_private)
 KVM_X86_OP_OPTIONAL(vm_destroy)
+KVM_X86_OP_OPTIONAL(vm_free)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)
 KVM_X86_OP(vcpu_create)
 KVM_X86_OP(vcpu_free)

arch/x86/include/asm/kvm_host.h +2 -0

index d98d61e5213d..15f7c0d45082 100644
@@ -1566,7 +1566,9 @@ struct kvm_x86_ops {
 	unsigned int vm_size;
 	int (*vm_enable_cap)(struct kvm *kvm, struct kvm_enable_cap *cap);
 	int (*vm_init)(struct kvm *kvm);
+	void (*flush_shadow_all_private)(struct kvm *kvm);
 	void (*vm_destroy)(struct kvm *kvm);
+	void (*vm_free)(struct kvm *kvm);
 
 	/* Create, but do not attach this VCPU */
 	int (*vcpu_precreate)(struct kvm *kvm);

arch/x86/kvm/vmx/main.c +32 -3

index 68bb320d0b6d..08ed6fc8cc3d 100644
@@ -62,14 +62,41 @@ static int vt_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 	return -EINVAL;
 }
 
+static void vt_hardware_unsetup(void)
+{
+	if (enable_tdx)
+		tdx_hardware_unsetup();
+	vmx_hardware_unsetup();
+}
+
 static int vt_vm_init(struct kvm *kvm)
 {
 	if (is_td(kvm))
-		return -EOPNOTSUPP;	/* Not ready to create guest TD yet. */
+		return tdx_vm_init(kvm);
 
 	return vmx_vm_init(kvm);
 }
 
+static void vt_flush_shadow_all_private(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		tdx_mmu_release_hkid(kvm);
+}
+
+static void vt_vm_destroy(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		return;
+
+	vmx_vm_destroy(kvm);
+}
+
+static void vt_vm_free(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		tdx_vm_free(kvm);
+}
+
 static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
 	if (!is_td(kvm))

@@ -94,7 +121,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 
 	.check_processor_compatibility = vmx_check_processor_compat,
 
-	.hardware_unsetup = vmx_hardware_unsetup,
+	.hardware_unsetup = vt_hardware_unsetup,
 
 	.hardware_enable = vt_hardware_enable,
 	.hardware_disable = vmx_hardware_disable,

@@ -105,7 +132,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.vm_size = sizeof(struct kvm_vmx),
 	.vm_enable_cap = vt_vm_enable_cap,
 	.vm_init = vt_vm_init,
-	.vm_destroy = vmx_vm_destroy,
+	.flush_shadow_all_private = vt_flush_shadow_all_private,
+	.vm_destroy = vt_vm_destroy,
+	.vm_free = vt_vm_free,
 
 	.vcpu_precreate = vmx_vcpu_precreate,
 	.vcpu_create = vmx_vcpu_create,

arch/x86/kvm/vmx/tdx.c +326 -1

index 8b02e605cfb5..3ede8a726b47 100644
@@ -5,8 +5,9 @@
 
 #include "capabilities.h"
 #include "x86_ops.h"
-#include "x86.h"
 #include "tdx.h"
+#include "tdx_ops.h"
+#include "x86.h"
 
 #undef pr_fmt
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

@@ -46,11 +47,276 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 	return r;
 }
 
+struct tdx_info {
+	u8 nr_tdcs_pages;
+};
+
+/* Info about the TDX module. */
+static struct tdx_info tdx_info;
+
+/*
+ * Some TDX SEAMCALLs (TDH.MNG.CREATE, TDH.PHYMEM.CACHE.WB,
+ * TDH.MNG.KEY.RECLAIMID, TDH.MNG.KEY.FREEID etc) tries to acquire a global lock
+ * internally in TDX module.  If failed, TDX_OPERAND_BUSY is returned without
+ * spinning or waiting due to a constraint on execution time.  It's caller's
+ * responsibility to avoid race (or retry on TDX_OPERAND_BUSY).  Use this mutex
+ * to avoid race in TDX module because the kernel knows better about scheduling.
+ */
+static DEFINE_MUTEX(tdx_lock);
+static struct mutex *tdx_mng_key_config_lock;
+
+static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid)
+{
+	return pa | ((hpa_t)hkid << boot_cpu_data.x86_phys_bits);
+}
+
+static inline bool is_td_created(struct kvm_tdx *kvm_tdx)
+{
+	return kvm_tdx->tdr_pa;
+}
+
+static inline void tdx_hkid_free(struct kvm_tdx *kvm_tdx)
+{
+	tdx_guest_keyid_free(kvm_tdx->hkid);
+	kvm_tdx->hkid = 0;
+}
+
+static inline bool is_hkid_assigned(struct kvm_tdx *kvm_tdx)
+{
+	return kvm_tdx->hkid > 0;
+}
+
 int tdx_hardware_enable(void)
 {
 	return tdx_cpu_enable();
 }
 
+static void tdx_clear_page(unsigned long page_pa)
+{
+	const void *zero_page = (const void *) __va(page_to_phys(ZERO_PAGE(0)));
+	void *page = __va(page_pa);
+	unsigned long i;
+
+	if (!cpu_feature_enabled(X86_FEATURE_MOVDIR64B)) {
+		clear_page(page);
+		return;
+	}
+
+	/*
+	 * Zeroing the page is only necessary for systems with MKTME-i:
+	 * when re-assign one page from old keyid to a new keyid, MOVDIR64B is
+	 * required to clear/write the page with new keyid to prevent integrity
+	 * error when read on the page with new keyid.
+	 *
+	 * clflush doesn't flush cache with HKID set.
+	 * The cache line could be poisoned (even without MKTME-i), clear the
+	 * poison bit.
+	 */
+	for (i = 0; i < PAGE_SIZE; i += 64)
+		movdir64b(page + i, zero_page);
+	/*
+	 * MOVDIR64B store uses WC buffer.  Prevent following memory reads
+	 * from seeing potentially poisoned cache.
+	 */
+	__mb();
+}
+
+static int tdx_reclaim_page(hpa_t pa, bool do_wb, u16 hkid)
+{
+	struct tdx_module_output out;
+	u64 err;
+
+	do {
+		err = tdh_phymem_page_reclaim(pa, &out);
+		/*
+		 * TDH.PHYMEM.PAGE.RECLAIM is allowed only when TD is shutdown.
+		 * state.  i.e. destructing TD.
+		 * TDH.PHYMEM.PAGE.RECLAIM requires TDR and target page.
+		 * Because we're destructing TD, it's rare to contend with TDR.
+		 */
+	} while (err == (TDX_OPERAND_BUSY | TDX_OPERAND_ID_RCX));
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_PHYMEM_PAGE_RECLAIM, err, &out);
+		return -EIO;
+	}
+
+	if (do_wb) {
+		/*
+		 * Only TDR page gets into this path.  No contention is expected
+		 * because of the last page of TD.
+		 */
+		err = tdh_phymem_page_wbinvd(set_hkid_to_hpa(pa, hkid));
+		if (WARN_ON_ONCE(err)) {
+			pr_tdx_error(TDH_PHYMEM_PAGE_WBINVD, err, NULL);
+			return -EIO;
+		}
+	}
+
+	tdx_clear_page(pa);
+	return 0;
+}
+
+static void tdx_reclaim_td_page(unsigned long td_page_pa)
+{
+	if (!td_page_pa)
+		return;
+	/*
+	 * TDCX are being reclaimed.  TDX module maps TDCX with HKID
+	 * assigned to the TD.  Here the cache associated to the TD
+	 * was already flushed by TDH.PHYMEM.CACHE.WB before here, So
+	 * cache doesn't need to be flushed again.
+	 */
+	if (tdx_reclaim_page(td_page_pa, false, 0))
+		/*
+		 * Leak the page on failure:
+		 * tdx_reclaim_page() returns an error if and only if there's an
+		 * unexpected, fatal error, e.g. a SEAMCALL with bad params,
+		 * incorrect concurrency in KVM, a TDX Module bug, etc.
+		 * Retrying at a later point is highly unlikely to be
+		 * successful.
+		 * No log here as tdx_reclaim_page() already did.
+		 */
+		return;
+	free_page((unsigned long)__va(td_page_pa));
+}
+
+static int tdx_do_tdh_phymem_cache_wb(void *param)
+{
+	u64 err = 0;
+
+	do {
+		err = tdh_phymem_cache_wb(!!err);
+	} while (err == TDX_INTERRUPTED_RESUMABLE);
+
+	/* Other thread may have done for us. */
+	if (err == TDX_NO_HKID_READY_TO_WBCACHE)
+		err = TDX_SUCCESS;
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_PHYMEM_CACHE_WB, err, NULL);
+		return -EIO;
+	}
+
+	return 0;
+}
+
+void tdx_mmu_release_hkid(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	cpumask_var_t packages;
+	bool cpumask_allocated;
+	u64 err;
+	int ret;
+	int i;
+
+	if (!is_hkid_assigned(kvm_tdx))
+		return;
+
+	if (!is_td_created(kvm_tdx))
+		goto free_hkid;
+
+	cpumask_allocated = zalloc_cpumask_var(&packages, GFP_KERNEL);
+	cpus_read_lock();
+	for_each_online_cpu(i) {
+		if (cpumask_allocated &&
+			cpumask_test_and_set_cpu(topology_physical_package_id(i),
+						packages))
+			continue;
+
+		/*
+		 * We can destroy multiple the guest TDs simultaneously.
+		 * Prevent tdh_phymem_cache_wb from returning TDX_BUSY by
+		 * serialization.
+		 */
+		mutex_lock(&tdx_lock);
+		ret = smp_call_on_cpu(i, tdx_do_tdh_phymem_cache_wb, NULL, 1);
+		mutex_unlock(&tdx_lock);
+		if (ret)
+			break;
+	}
+	cpus_read_unlock();
+	free_cpumask_var(packages);
+
+	mutex_lock(&tdx_lock);
+	err = tdh_mng_key_freeid(kvm_tdx->tdr_pa);
+	mutex_unlock(&tdx_lock);
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_MNG_KEY_FREEID, err, NULL);
+		pr_err("tdh_mng_key_freeid failed. HKID %d is leaked.\n",
+			kvm_tdx->hkid);
+		return;
+	}
+
+free_hkid:
+	tdx_hkid_free(kvm_tdx);
+}
+
+void tdx_vm_free(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	int i;
+
+	/* Can't reclaim or free TD pages if teardown failed. */
+	if (is_hkid_assigned(kvm_tdx))
+		return;
+
+	if (kvm_tdx->tdcs_pa) {
+		for (i = 0; i < tdx_info.nr_tdcs_pages; i++)
+			tdx_reclaim_td_page(kvm_tdx->tdcs_pa[i]);
+		kfree(kvm_tdx->tdcs_pa);
+		kvm_tdx->tdcs_pa = NULL;
+	}
+
+	if (!kvm_tdx->tdr_pa)
+		return;
+	/*
+	 * TDX module maps TDR with TDX global HKID.  TDX module may access TDR
+	 * while operating on TD (Especially reclaiming TDCS).  Cache flush with
+	 * TDX global HKID is needed.
+	 */
+	if (tdx_reclaim_page(kvm_tdx->tdr_pa, true, tdx_global_keyid))
+		return;
+
+	free_page((unsigned long)__va(kvm_tdx->tdr_pa));
+	kvm_tdx->tdr_pa = 0;
+}
+
+static int tdx_do_tdh_mng_key_config(void *param)
+{
+	hpa_t *tdr_p = param;
+	u64 err;
+
+	do {
+		err = tdh_mng_key_config(*tdr_p);
+
+		/*
+		 * If it failed to generate a random key, retry it because this
+		 * is typically caused by an entropy error of the CPU's random
+		 * number generator.
+		 */
+	} while (err == TDX_KEY_GENERATION_FAILED);
+
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_MNG_KEY_CONFIG, err, NULL);
+		return -EIO;
+	}
+
+	return 0;
+}
+
+static int __tdx_td_init(struct kvm *kvm);
+
+int tdx_vm_init(struct kvm *kvm)
+{
+	/*
+	 * TDX has its own limit of the number of vcpus in addition to
+	 * KVM_MAX_VCPUS.
+	 */
+	kvm->max_vcpus = min(kvm->max_vcpus, TDX_MAX_VCPUS);
+
+	/* Place holder for TDX specific logic. */
+	return __tdx_td_init(kvm);
+}
+
 int tdx_dev_ioctl(void __user *argp)
 {
 	struct kvm_tdx_capabilities __user *user_caps;

@@ -102,6 +368,160 @@ int tdx_dev_ioctl(void __user *argp)
 	return 0;
 }
 
+static int __tdx_td_init(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	cpumask_var_t packages;
+	unsigned long *tdcs_pa = NULL;
+	unsigned long tdr_pa = 0;
+	unsigned long va;
+	int ret, i;
+	u64 err;
+
+	ret = tdx_guest_keyid_alloc();
+	if (ret < 0)
+		return ret;
+	kvm_tdx->hkid = ret;
+
+	va = __get_free_page(GFP_KERNEL_ACCOUNT);
+	if (!va)
+		goto free_hkid;
+	tdr_pa = __pa(va);
+
+	tdcs_pa = kcalloc(tdx_info.nr_tdcs_pages, sizeof(*kvm_tdx->tdcs_pa),
+			  GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+	if (!tdcs_pa)
+		goto free_tdr;
+	for (i = 0; i < tdx_info.nr_tdcs_pages; i++) {
+		va = __get_free_page(GFP_KERNEL_ACCOUNT);
+		if (!va)
+			goto free_tdcs;
+		tdcs_pa[i] = __pa(va);
+	}
+
+	if (!zalloc_cpumask_var(&packages, GFP_KERNEL)) {
+		ret = -ENOMEM;
+		goto free_tdcs;
+	}
+	cpus_read_lock();
+	/*
+	 * Need at least one CPU of the package to be online in order to
+	 * program all packages for host key id.  Check it.
+	 */
+	for_each_present_cpu(i)
+		cpumask_set_cpu(topology_physical_package_id(i), packages);
+	for_each_online_cpu(i)
+		cpumask_clear_cpu(topology_physical_package_id(i), packages);
+	if (!cpumask_empty(packages)) {
+		ret = -EIO;
+		/*
+		 * Because it's hard for human operator to figure out the
+		 * reason, warn it.
+		 */
+		pr_warn("All packages need to have online CPU to create TD. Online CPU and retry.\n");
+		goto free_packages;
+	}
+
+	/*
+	 * Acquire global lock to avoid TDX_OPERAND_BUSY:
+	 * TDH.MNG.CREATE and other APIs try to lock the global Key Owner
+	 * Table (KOT) to track the assigned TDX private HKID.  It doesn't spin
+	 * to acquire the lock, returns TDX_OPERAND_BUSY instead, and let the

arch/x86/kvm/vmx/tdx.c int tdx_dev_ioctl(void __user *argp) · line 439

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c +10 -0

+	 * caller to handle the contention.  This is because of time limitation
+	 * usable inside the TDX module and OS/VMM knows better about process
+	 * scheduling.
+	 *
+	 * APIs to acquire the lock of KOT:
+	 * TDH.MNG.CREATE, TDH.MNG.KEY.FREEID, TDH.MNG.VPFLUSHDONE, and
+	 * TDH.PHYMEM.CACHE.WB.
+	 */
+	mutex_lock(&tdx_lock);
+	err = tdh_mng_create(tdr_pa, kvm_tdx->hkid);

In the latest (March 2023 ver.) Intel® TDX Module v1.5 ABI Specification, error code 0x80000203 represents TDX_RND_NO_ENTROPY:

Random number generation (e.g., RDRAND or RDSEED) failed because the hardware random number generator did not have enough entropy. The caller should retry the operation.

Quoted 211 lines

arch/x86/kvm/vmx/tdx.c +104 -1

+	mutex_unlock(&tdx_lock);
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_MNG_CREATE, err, NULL);
+		ret = -EIO;
+		goto free_packages;
+	}
+	kvm_tdx->tdr_pa = tdr_pa;
+
+	for_each_online_cpu(i) {
+		int pkg = topology_physical_package_id(i);
+
+		if (cpumask_test_and_set_cpu(pkg, packages))
+			continue;
+
+		/*
+		 * Program the memory controller in the package with an
+		 * encryption key associated to a TDX private host key id
+		 * assigned to this TDR.  Concurrent operations on same memory
+		 * controller results in TDX_OPERAND_BUSY.  Avoid this race by
+		 * mutex.
+		 */
+		mutex_lock(&tdx_mng_key_config_lock[pkg]);
+		ret = smp_call_on_cpu(i, tdx_do_tdh_mng_key_config,
+				      &kvm_tdx->tdr_pa, true);
+		mutex_unlock(&tdx_mng_key_config_lock[pkg]);
+		if (ret)
+			break;
+	}
+	cpus_read_unlock();
+	free_cpumask_var(packages);
+	if (ret)
+		goto teardown;
+
+	kvm_tdx->tdcs_pa = tdcs_pa;
+	for (i = 0; i < tdx_info.nr_tdcs_pages; i++) {
+		err = tdh_mng_addcx(kvm_tdx->tdr_pa, tdcs_pa[i]);
+		if (WARN_ON_ONCE(err)) {
+			pr_tdx_error(TDH_MNG_ADDCX, err, NULL);
+			for (i++; i < tdx_info.nr_tdcs_pages; i++) {
+				free_page((unsigned long)__va(tdcs_pa[i]));
+				tdcs_pa[i] = 0;
+			}
+			ret = -EIO;
+			goto teardown;
+		}
+	}
+
+	/*
+	 * Note, TDH_MNG_INIT cannot be invoked here.  TDH_MNG_INIT requires a dedicated
+	 * ioctl() to define the configure CPUID values for the TD.
+	 */
+	return 0;
+
+	/*
+	 * The sequence for freeing resources from a partially initialized TD
+	 * varies based on where in the initialization flow failure occurred.
+	 * Simply use the full teardown and destroy, which naturally play nice
+	 * with partial initialization.
+	 */
+teardown:
+	tdx_mmu_release_hkid(kvm);
+	tdx_vm_free(kvm);
+	return ret;
+
+free_packages:
+	cpus_read_unlock();
+	free_cpumask_var(packages);
+free_tdcs:
+	for (i = 0; i < tdx_info.nr_tdcs_pages; i++) {
+		if (tdcs_pa[i])
+			free_page((unsigned long)__va(tdcs_pa[i]));
+	}
+	kfree(tdcs_pa);
+	kvm_tdx->tdcs_pa = NULL;
+
+free_tdr:
+	if (tdr_pa)
+		free_page((unsigned long)__va(tdr_pa));
+	kvm_tdx->tdr_pa = 0;
+free_hkid:
+	if (is_hkid_assigned(kvm_tdx))
+		tdx_hkid_free(kvm_tdx);
+	return ret;
+}
+
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)
 {
 	struct kvm_tdx_cmd tdx_cmd;

@@ -142,9 +562,11 @@ static int __init tdx_module_setup(void)
 		return ret;
 	}
 
-	/* Sanitary check just in case. */
 	tdsysinfo = tdx_get_sysinfo();
 	WARN_ON(tdsysinfo->num_cpuid_config > TDX_MAX_NR_CPUID_CONFIGS);
+	tdx_info = (struct tdx_info) {
+		.nr_tdcs_pages = tdsysinfo->tdcs_base_size / PAGE_SIZE,
+	};
 
 	pr_info("TDX is supported.\n");
 	return 0;

@@ -163,6 +585,8 @@ static int __init tdx_cpu_enable_cpu(void *unused)
 
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
 {
+	int max_pkgs;
+	int i;
 	int r;
 
 	if (!enable_ept) {

@@ -170,6 +594,14 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
 		return -EINVAL;
 	}
 
+	max_pkgs = topology_max_packages();
+	tdx_mng_key_config_lock = kcalloc(max_pkgs, sizeof(*tdx_mng_key_config_lock),
+				   GFP_KERNEL);
+	if (!tdx_mng_key_config_lock)
+		return -ENOMEM;
+	for (i = 0; i < max_pkgs; i++)
+		mutex_init(&tdx_mng_key_config_lock[i]);
+
 	/* tdx_enable() in tdx_module_setup() requires cpus lock. */
 	cpus_read_lock();
 	/* TDX requires VMX. */

@@ -193,3 +625,9 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
 
 	return r;
 }
+
+void tdx_hardware_unsetup(void)
+{
+	/* kfree accepts NULL. */
+	kfree(tdx_mng_key_config_lock);
+}

arch/x86/kvm/vmx/tdx.h +5 -1

index 3860aa351bd9..4b790503e43e 100644
@@ -8,7 +8,11 @@
 
 struct kvm_tdx {
 	struct kvm kvm;
-	/* TDX specific members follow. */
+
+	unsigned long tdr_pa;
+	unsigned long *tdcs_pa;
+
+	int hkid;
 };
 
 struct vcpu_tdx {

arch/x86/kvm/vmx/x86_ops.h +9 -0

index 8118647aa8ca..8f87eeeab97c 100644
@@ -139,19 +139,28 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
 #ifdef CONFIG_INTEL_TDX_HOST
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
+void tdx_hardware_unsetup(void);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
 int tdx_dev_ioctl(void __user *argp);
 
 int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);
+int tdx_vm_init(struct kvm *kvm);
+void tdx_mmu_release_hkid(struct kvm *kvm);
+void tdx_vm_free(struct kvm *kvm);
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
+static inline void tdx_hardware_unsetup(void) {}
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
 
 static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { return -EINVAL; };
+static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; }
+static inline void tdx_mmu_release_hkid(struct kvm *kvm) {}
+static inline void tdx_flush_shadow_all_private(struct kvm *kvm) {}
+static inline void tdx_vm_free(struct kvm *kvm) {}
 static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
 #endif

arch/x86/kvm/x86.c +8 -0

index 7b02dd40ef21..89fd8074c20c 100644
@@ -12398,6 +12398,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 	kvm_page_track_cleanup(kvm);
 	kvm_xen_destroy_vm(kvm);
 	kvm_hv_destroy_vm(kvm);
+	static_call_cond(kvm_x86_vm_free)(kvm);
 }
 
 static void memslot_rmap_free(struct kvm_memory_slot *slot)

@@ -12710,6 +12711,13 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
 
 void kvm_arch_flush_shadow_all(struct kvm *kvm)
 {
+	/*
+	 * kvm_mmu_zap_all() zaps both private and shared page tables.  Before
+	 * tearing down private page tables, TDX requires some TD resources to
+	 * be destroyed (i.e. keyID must have been reclaimed, etc).  Invoke
+	 * kvm_x86_flush_shadow_all_private() for this.
+	 */
+	static_call_cond(kvm_x86_flush_shadow_all_private)(kvm);
 	kvm_mmu_zap_all(kvm);
 }

Wang, Wei W

2023-03-24 05:12 UTC #65 lore raw

RE: [PATCH v13 019/113] KVM: TDX: create/destroy VM structure

Quoted 68 lines · Lei

Quoted 33 lines

arch/x86/kvm/vmx/tdx.c +33 -0

+static int __tdx_td_init(struct kvm *kvm) {
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	cpumask_var_t packages;
+	unsigned long *tdcs_pa = NULL;
+	unsigned long tdr_pa = 0;
+	unsigned long va;
+	int ret, i;
+	u64 err;
+
+	ret = tdx_guest_keyid_alloc();
+	if (ret < 0)
+		return ret;
+	kvm_tdx->hkid = ret;
+
+	va = __get_free_page(GFP_KERNEL_ACCOUNT);
+	if (!va)
+		goto free_hkid;
+	tdr_pa = __pa(va);
+
+	tdcs_pa = kcalloc(tdx_info.nr_tdcs_pages, sizeof(*kvm_tdx->tdcs_pa),
+			  GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+	if (!tdcs_pa)
+		goto free_tdr;
+	for (i = 0; i < tdx_info.nr_tdcs_pages; i++) {
+		va = __get_free_page(GFP_KERNEL_ACCOUNT);
+		if (!va)
+			goto free_tdcs;
+		tdcs_pa[i] = __pa(va);
+	}
+
+	if (!zalloc_cpumask_var(&packages, GFP_KERNEL)) {
+		ret = -ENOMEM;
+		goto free_tdcs;

arch/x86/kvm/vmx/tdx.c int tdx_dev_ioctl(void __user *argp)

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c +10 -0

+	}
+	cpus_read_lock();
+	/*
+	 * Need at least one CPU of the package to be online in order to
+	 * program all packages for host key id.  Check it.
+	 */
+	for_each_present_cpu(i)
+		cpumask_set_cpu(topology_physical_package_id(i), packages);
+	for_each_online_cpu(i)
+		cpumask_clear_cpu(topology_physical_package_id(i),

packages);

arch/x86/kvm/vmx/tdx.c int tdx_dev_ioctl(void __user *argp)

Quoted 7 lines

arch/x86/kvm/vmx/tdx.c +7 -0

+	if (!cpumask_empty(packages)) {
+		ret = -EIO;
+		/*
+		 * Because it's hard for human operator to figure out the
+		 * reason, warn it.
+		 */
+		pr_warn("All packages need to have online CPU to create TD.

Online CPU and retry.\n");

arch/x86/kvm/vmx/tdx.c int tdx_dev_ioctl(void __user *argp)

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c +10 -0

+		goto free_packages;
+	}
+
+	/*
+	 * Acquire global lock to avoid TDX_OPERAND_BUSY:
+	 * TDH.MNG.CREATE and other APIs try to lock the global Key Owner
+	 * Table (KOT) to track the assigned TDX private HKID.  It doesn't spin
+	 * to acquire the lock, returns TDX_OPERAND_BUSY instead, and let the
+	 * caller to handle the contention.  This is because of time limitation
+	 * usable inside the TDX module and OS/VMM knows better about

process

arch/x86/kvm/vmx/tdx.c int tdx_dev_ioctl(void __user *argp)

Quoted 4 lines

arch/x86/kvm/vmx/tdx.c +4 -0

+	 * scheduling.
+	 *
+	 * APIs to acquire the lock of KOT:
+	 * TDH.MNG.CREATE, TDH.MNG.KEY.FREEID, TDH.MNG.VPFLUSHDONE,

and

Quoted 7 lines

arch/x86/kvm/vmx/tdx.c int tdx_dev_ioctl(void __user *argp) · line 439
Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +4 -0
+	 * TDH.PHYMEM.CACHE.WB.
+	 */
+	mutex_lock(&tdx_lock);
+	err = tdh_mng_create(tdr_pa, kvm_tdx->hkid);
In the latest (March 2023 ver.) Intel® TDX Module v1.5 ABI Specification, error code 0x80000203 represents TDX_RND_NO_ENTROPY:

This patchset is written for what's documented in the TDX 1.0 spec. We'll update it for 1.5 later.

Zhi Wang

2023-03-26 11:09 UTC #66 lore raw

Re: [PATCH v13 019/113] KVM: TDX: create/destroy VM structure

Quoted 197 lines · On Sun, 12 Mar 2023 10:55:43 -0700 [email protected] wrote:

From: Isaku Yamahata <[email protected]>

Add vm_free() of kvm_x86_ops hook at the end of kvm_arch_destroy_vm() because some per-VM TDX resources, e.g. TDR, need to be freed after other TDX resources, e.g. HKID, were freed.

Co-developed-by Kai Huang <[email protected]>

Signed-off-by Kai Huang <[email protected]>

Signed-off-by Sean Christopherson <[email protected]>

---
Changes v11 -> v12:
- use cpu_feature_enabled().

Changes v10 -> v11:
- Fix doule free in tdx_vm_free() by setting NULL.
- replace struct tdx_td_page tdr and tdcs from struct kvm_tdx with unsigned long

---
 arch/x86/include/asm/kvm-x86-ops.h |   2 +
 arch/x86/include/asm/kvm_host.h    |   2 +
 arch/x86/kvm/vmx/main.c            |  35 ++-
 arch/x86/kvm/vmx/tdx.c             | 442 ++++++++++++++++++++++++++++-
 arch/x86/kvm/vmx/tdx.h             |   6 +-
 arch/x86/kvm/vmx/x86_ops.h         |   9 +
 arch/x86/kvm/x86.c                 |   8 +
 7 files changed, 498 insertions(+), 6 deletions(-)

arch/x86/include/asm/kvm-x86-ops.h +2 -0

index 7522c193f2b4..c30d2d2ad686 100644
@@ -24,7 +24,9 @@ KVM_X86_OP(is_vm_type_supported)
 KVM_X86_OP_OPTIONAL(max_vcpus);
 KVM_X86_OP_OPTIONAL(vm_enable_cap)
 KVM_X86_OP(vm_init)
+KVM_X86_OP_OPTIONAL(flush_shadow_all_private)
 KVM_X86_OP_OPTIONAL(vm_destroy)
+KVM_X86_OP_OPTIONAL(vm_free)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)
 KVM_X86_OP(vcpu_create)
 KVM_X86_OP(vcpu_free)

arch/x86/include/asm/kvm_host.h +2 -0

index d98d61e5213d..15f7c0d45082 100644
@@ -1566,7 +1566,9 @@ struct kvm_x86_ops {
 	unsigned int vm_size;
 	int (*vm_enable_cap)(struct kvm *kvm, struct kvm_enable_cap *cap);
 	int (*vm_init)(struct kvm *kvm);
+	void (*flush_shadow_all_private)(struct kvm *kvm);
 	void (*vm_destroy)(struct kvm *kvm);
+	void (*vm_free)(struct kvm *kvm);
 
 	/* Create, but do not attach this VCPU */
 	int (*vcpu_precreate)(struct kvm *kvm);

arch/x86/kvm/vmx/main.c +32 -3

index 68bb320d0b6d..08ed6fc8cc3d 100644
@@ -62,14 +62,41 @@ static int vt_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 	return -EINVAL;
 }
 
+static void vt_hardware_unsetup(void)
+{
+	if (enable_tdx)
+		tdx_hardware_unsetup();
+	vmx_hardware_unsetup();
+}
+
 static int vt_vm_init(struct kvm *kvm)
 {
 	if (is_td(kvm))
-		return -EOPNOTSUPP;	/* Not ready to create guest TD yet. */
+		return tdx_vm_init(kvm);
 
 	return vmx_vm_init(kvm);
 }
 
+static void vt_flush_shadow_all_private(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		tdx_mmu_release_hkid(kvm);
+}
+
+static void vt_vm_destroy(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		return;
+
+	vmx_vm_destroy(kvm);
+}
+
+static void vt_vm_free(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		tdx_vm_free(kvm);
+}
+
 static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
 	if (!is_td(kvm))

@@ -94,7 +121,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 
 	.check_processor_compatibility = vmx_check_processor_compat,
 
-	.hardware_unsetup = vmx_hardware_unsetup,
+	.hardware_unsetup = vt_hardware_unsetup,
 
 	.hardware_enable = vt_hardware_enable,
 	.hardware_disable = vmx_hardware_disable,

@@ -105,7 +132,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.vm_size = sizeof(struct kvm_vmx),
 	.vm_enable_cap = vt_vm_enable_cap,
 	.vm_init = vt_vm_init,
-	.vm_destroy = vmx_vm_destroy,
+	.flush_shadow_all_private = vt_flush_shadow_all_private,
+	.vm_destroy = vt_vm_destroy,
+	.vm_free = vt_vm_free,
 
 	.vcpu_precreate = vmx_vcpu_precreate,
 	.vcpu_create = vmx_vcpu_create,

arch/x86/kvm/vmx/tdx.c +41 -1

index 8b02e605cfb5..3ede8a726b47 100644
@@ -5,8 +5,9 @@
 
 #include "capabilities.h"
 #include "x86_ops.h"
-#include "x86.h"
 #include "tdx.h"
+#include "tdx_ops.h"
+#include "x86.h"
 
 #undef pr_fmt
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

@@ -46,11 +47,276 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 	return r;
 }
 
+struct tdx_info {
+	u8 nr_tdcs_pages;
+};
+
+/* Info about the TDX module. */
+static struct tdx_info tdx_info;
+
+/*
+ * Some TDX SEAMCALLs (TDH.MNG.CREATE, TDH.PHYMEM.CACHE.WB,
+ * TDH.MNG.KEY.RECLAIMID, TDH.MNG.KEY.FREEID etc) tries to acquire a global lock
+ * internally in TDX module.  If failed, TDX_OPERAND_BUSY is returned without
+ * spinning or waiting due to a constraint on execution time.  It's caller's
+ * responsibility to avoid race (or retry on TDX_OPERAND_BUSY).  Use this mutex
+ * to avoid race in TDX module because the kernel knows better about scheduling.
+ */
+static DEFINE_MUTEX(tdx_lock);
+static struct mutex *tdx_mng_key_config_lock;
+
+static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid)
+{
+	return pa | ((hpa_t)hkid << boot_cpu_data.x86_phys_bits);
+}
+
+static inline bool is_td_created(struct kvm_tdx *kvm_tdx)
+{
+	return kvm_tdx->tdr_pa;
+}
+
+static inline void tdx_hkid_free(struct kvm_tdx *kvm_tdx)
+{
+	tdx_guest_keyid_free(kvm_tdx->hkid);
+	kvm_tdx->hkid = 0;
+}
+
+static inline bool is_hkid_assigned(struct kvm_tdx *kvm_tdx)
+{
+	return kvm_tdx->hkid > 0;
+}
+
 int tdx_hardware_enable(void)
 {
 	return tdx_cpu_enable();
 }

arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 103

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c +10 -0

+static void tdx_clear_page(unsigned long page_pa)
+{
+	const void *zero_page = (const void *) __va(page_to_phys(ZERO_PAGE(0)));
+	void *page = __va(page_pa);
+	unsigned long i;
+
+	if (!cpu_feature_enabled(X86_FEATURE_MOVDIR64B)) {
+		clear_page(page);
+		return;
+	}

Is it possbile to have a TDX machine without MOVDIR64B support? I am not sure if there is any other way for the kernel to clear the posioned cache line.

If not, there should be a warn/bug at least and check if MOVDIR64B support when initializing the TDX.

Quoted 110 lines

arch/x86/kvm/vmx/tdx.c +110 -0

+
+	/*
+	 * Zeroing the page is only necessary for systems with MKTME-i:
+	 * when re-assign one page from old keyid to a new keyid, MOVDIR64B is
+	 * required to clear/write the page with new keyid to prevent integrity
+	 * error when read on the page with new keyid.
+	 *
+	 * clflush doesn't flush cache with HKID set.
+	 * The cache line could be poisoned (even without MKTME-i), clear the
+	 * poison bit.
+	 */
+	for (i = 0; i < PAGE_SIZE; i += 64)
+		movdir64b(page + i, zero_page);
+	/*
+	 * MOVDIR64B store uses WC buffer.  Prevent following memory reads
+	 * from seeing potentially poisoned cache.
+	 */
+	__mb();
+}
+
+static int tdx_reclaim_page(hpa_t pa, bool do_wb, u16 hkid)
+{
+	struct tdx_module_output out;
+	u64 err;
+
+	do {
+		err = tdh_phymem_page_reclaim(pa, &out);
+		/*
+		 * TDH.PHYMEM.PAGE.RECLAIM is allowed only when TD is shutdown.
+		 * state.  i.e. destructing TD.
+		 * TDH.PHYMEM.PAGE.RECLAIM requires TDR and target page.
+		 * Because we're destructing TD, it's rare to contend with TDR.
+		 */
+	} while (err == (TDX_OPERAND_BUSY | TDX_OPERAND_ID_RCX));
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_PHYMEM_PAGE_RECLAIM, err, &out);
+		return -EIO;
+	}
+
+	if (do_wb) {
+		/*
+		 * Only TDR page gets into this path.  No contention is expected
+		 * because of the last page of TD.
+		 */
+		err = tdh_phymem_page_wbinvd(set_hkid_to_hpa(pa, hkid));
+		if (WARN_ON_ONCE(err)) {
+			pr_tdx_error(TDH_PHYMEM_PAGE_WBINVD, err, NULL);
+			return -EIO;
+		}
+	}
+
+	tdx_clear_page(pa);
+	return 0;
+}
+
+static void tdx_reclaim_td_page(unsigned long td_page_pa)
+{
+	if (!td_page_pa)
+		return;
+	/*
+	 * TDCX are being reclaimed.  TDX module maps TDCX with HKID
+	 * assigned to the TD.  Here the cache associated to the TD
+	 * was already flushed by TDH.PHYMEM.CACHE.WB before here, So
+	 * cache doesn't need to be flushed again.
+	 */
+	if (tdx_reclaim_page(td_page_pa, false, 0))
+		/*
+		 * Leak the page on failure:
+		 * tdx_reclaim_page() returns an error if and only if there's an
+		 * unexpected, fatal error, e.g. a SEAMCALL with bad params,
+		 * incorrect concurrency in KVM, a TDX Module bug, etc.
+		 * Retrying at a later point is highly unlikely to be
+		 * successful.
+		 * No log here as tdx_reclaim_page() already did.
+		 */
+		return;
+	free_page((unsigned long)__va(td_page_pa));
+}
+
+static int tdx_do_tdh_phymem_cache_wb(void *param)
+{
+	u64 err = 0;
+
+	do {
+		err = tdh_phymem_cache_wb(!!err);
+	} while (err == TDX_INTERRUPTED_RESUMABLE);
+
+	/* Other thread may have done for us. */
+	if (err == TDX_NO_HKID_READY_TO_WBCACHE)
+		err = TDX_SUCCESS;
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_PHYMEM_CACHE_WB, err, NULL);
+		return -EIO;
+	}
+
+	return 0;
+}
+
+void tdx_mmu_release_hkid(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	cpumask_var_t packages;
+	bool cpumask_allocated;
+	u64 err;
+	int ret;
+	int i;
+
+	if (!is_hkid_assigned(kvm_tdx))
+		return;
+

arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 223

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c +10 -0

+	if (!is_td_created(kvm_tdx))
+		goto free_hkid;
+
+	cpumask_allocated = zalloc_cpumask_var(&packages, GFP_KERNEL);
+	cpus_read_lock();
+	for_each_online_cpu(i) {
+		if (cpumask_allocated &&
+			cpumask_test_and_set_cpu(topology_physical_package_id(i),
+						packages))
+			continue;

Is this necessary to check cpumask_allocated in the while loop? if cpumask is not succefully allocated, wouldn't it be better to bail out just after it?

Quoted 28 lines

arch/x86/kvm/vmx/tdx.c +28 -0

+
+		/*
+		 * We can destroy multiple the guest TDs simultaneously.
+		 * Prevent tdh_phymem_cache_wb from returning TDX_BUSY by
+		 * serialization.
+		 */
+		mutex_lock(&tdx_lock);
+		ret = smp_call_on_cpu(i, tdx_do_tdh_phymem_cache_wb, NULL, 1);
+		mutex_unlock(&tdx_lock);
+		if (ret)
+			break;
+	}
+	cpus_read_unlock();
+	free_cpumask_var(packages);
+
+	mutex_lock(&tdx_lock);
+	err = tdh_mng_key_freeid(kvm_tdx->tdr_pa);
+	mutex_unlock(&tdx_lock);
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_MNG_KEY_FREEID, err, NULL);
+		pr_err("tdh_mng_key_freeid failed. HKID %d is leaked.\n",
+			kvm_tdx->hkid);
+		return;
+	}
+
+free_hkid:
+	tdx_hkid_free(kvm_tdx);
+}

arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 261

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c +10 -0

+
+void tdx_vm_free(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	int i;
+
+	/* Can't reclaim or free TD pages if teardown failed. */
+	if (is_hkid_assigned(kvm_tdx))
+		return;
+

Better to explain why, as it is common to think even the teardown failed, we should still try to reclaim the pages as many as we can.

Quoted 345 lines

arch/x86/kvm/vmx/tdx.c +231 -1

+	if (kvm_tdx->tdcs_pa) {
+		for (i = 0; i < tdx_info.nr_tdcs_pages; i++)
+			tdx_reclaim_td_page(kvm_tdx->tdcs_pa[i]);
+		kfree(kvm_tdx->tdcs_pa);
+		kvm_tdx->tdcs_pa = NULL;
+	}
+
+	if (!kvm_tdx->tdr_pa)
+		return;
+	/*
+	 * TDX module maps TDR with TDX global HKID.  TDX module may access TDR
+	 * while operating on TD (Especially reclaiming TDCS).  Cache flush with
+	 * TDX global HKID is needed.
+	 */
+	if (tdx_reclaim_page(kvm_tdx->tdr_pa, true, tdx_global_keyid))
+		return;
+
+	free_page((unsigned long)__va(kvm_tdx->tdr_pa));
+	kvm_tdx->tdr_pa = 0;
+}
+
+static int tdx_do_tdh_mng_key_config(void *param)
+{
+	hpa_t *tdr_p = param;
+	u64 err;
+
+	do {
+		err = tdh_mng_key_config(*tdr_p);
+
+		/*
+		 * If it failed to generate a random key, retry it because this
+		 * is typically caused by an entropy error of the CPU's random
+		 * number generator.
+		 */
+	} while (err == TDX_KEY_GENERATION_FAILED);
+
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_MNG_KEY_CONFIG, err, NULL);
+		return -EIO;
+	}
+
+	return 0;
+}
+
+static int __tdx_td_init(struct kvm *kvm);
+
+int tdx_vm_init(struct kvm *kvm)
+{
+	/*
+	 * TDX has its own limit of the number of vcpus in addition to
+	 * KVM_MAX_VCPUS.
+	 */
+	kvm->max_vcpus = min(kvm->max_vcpus, TDX_MAX_VCPUS);
+
+	/* Place holder for TDX specific logic. */
+	return __tdx_td_init(kvm);
+}
+
 int tdx_dev_ioctl(void __user *argp)
 {
 	struct kvm_tdx_capabilities __user *user_caps;

@@ -102,6 +368,160 @@ int tdx_dev_ioctl(void __user *argp)
 	return 0;
 }
 
+static int __tdx_td_init(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	cpumask_var_t packages;
+	unsigned long *tdcs_pa = NULL;
+	unsigned long tdr_pa = 0;
+	unsigned long va;
+	int ret, i;
+	u64 err;
+
+	ret = tdx_guest_keyid_alloc();
+	if (ret < 0)
+		return ret;
+	kvm_tdx->hkid = ret;
+
+	va = __get_free_page(GFP_KERNEL_ACCOUNT);
+	if (!va)
+		goto free_hkid;
+	tdr_pa = __pa(va);
+
+	tdcs_pa = kcalloc(tdx_info.nr_tdcs_pages, sizeof(*kvm_tdx->tdcs_pa),
+			  GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+	if (!tdcs_pa)
+		goto free_tdr;
+	for (i = 0; i < tdx_info.nr_tdcs_pages; i++) {
+		va = __get_free_page(GFP_KERNEL_ACCOUNT);
+		if (!va)
+			goto free_tdcs;
+		tdcs_pa[i] = __pa(va);
+	}
+
+	if (!zalloc_cpumask_var(&packages, GFP_KERNEL)) {
+		ret = -ENOMEM;
+		goto free_tdcs;
+	}
+	cpus_read_lock();
+	/*
+	 * Need at least one CPU of the package to be online in order to
+	 * program all packages for host key id.  Check it.
+	 */
+	for_each_present_cpu(i)
+		cpumask_set_cpu(topology_physical_package_id(i), packages);
+	for_each_online_cpu(i)
+		cpumask_clear_cpu(topology_physical_package_id(i), packages);
+	if (!cpumask_empty(packages)) {
+		ret = -EIO;
+		/*
+		 * Because it's hard for human operator to figure out the
+		 * reason, warn it.
+		 */
+		pr_warn("All packages need to have online CPU to create TD. Online CPU and retry.\n");
+		goto free_packages;
+	}
+
+	/*
+	 * Acquire global lock to avoid TDX_OPERAND_BUSY:
+	 * TDH.MNG.CREATE and other APIs try to lock the global Key Owner
+	 * Table (KOT) to track the assigned TDX private HKID.  It doesn't spin
+	 * to acquire the lock, returns TDX_OPERAND_BUSY instead, and let the
+	 * caller to handle the contention.  This is because of time limitation
+	 * usable inside the TDX module and OS/VMM knows better about process
+	 * scheduling.
+	 *
+	 * APIs to acquire the lock of KOT:
+	 * TDH.MNG.CREATE, TDH.MNG.KEY.FREEID, TDH.MNG.VPFLUSHDONE, and
+	 * TDH.PHYMEM.CACHE.WB.
+	 */
+	mutex_lock(&tdx_lock);
+	err = tdh_mng_create(tdr_pa, kvm_tdx->hkid);
+	mutex_unlock(&tdx_lock);
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_MNG_CREATE, err, NULL);
+		ret = -EIO;
+		goto free_packages;
+	}
+	kvm_tdx->tdr_pa = tdr_pa;
+
+	for_each_online_cpu(i) {
+		int pkg = topology_physical_package_id(i);
+
+		if (cpumask_test_and_set_cpu(pkg, packages))
+			continue;
+
+		/*
+		 * Program the memory controller in the package with an
+		 * encryption key associated to a TDX private host key id
+		 * assigned to this TDR.  Concurrent operations on same memory
+		 * controller results in TDX_OPERAND_BUSY.  Avoid this race by
+		 * mutex.
+		 */
+		mutex_lock(&tdx_mng_key_config_lock[pkg]);
+		ret = smp_call_on_cpu(i, tdx_do_tdh_mng_key_config,
+				      &kvm_tdx->tdr_pa, true);
+		mutex_unlock(&tdx_mng_key_config_lock[pkg]);
+		if (ret)
+			break;
+	}
+	cpus_read_unlock();
+	free_cpumask_var(packages);
+	if (ret)
+		goto teardown;
+
+	kvm_tdx->tdcs_pa = tdcs_pa;
+	for (i = 0; i < tdx_info.nr_tdcs_pages; i++) {
+		err = tdh_mng_addcx(kvm_tdx->tdr_pa, tdcs_pa[i]);
+		if (WARN_ON_ONCE(err)) {
+			pr_tdx_error(TDH_MNG_ADDCX, err, NULL);
+			for (i++; i < tdx_info.nr_tdcs_pages; i++) {
+				free_page((unsigned long)__va(tdcs_pa[i]));
+				tdcs_pa[i] = 0;
+			}
+			ret = -EIO;
+			goto teardown;
+		}
+	}
+
+	/*
+	 * Note, TDH_MNG_INIT cannot be invoked here.  TDH_MNG_INIT requires a dedicated
+	 * ioctl() to define the configure CPUID values for the TD.
+	 */
+	return 0;
+
+	/*
+	 * The sequence for freeing resources from a partially initialized TD
+	 * varies based on where in the initialization flow failure occurred.
+	 * Simply use the full teardown and destroy, which naturally play nice
+	 * with partial initialization.
+	 */
+teardown:
+	tdx_mmu_release_hkid(kvm);
+	tdx_vm_free(kvm);
+	return ret;
+
+free_packages:
+	cpus_read_unlock();
+	free_cpumask_var(packages);
+free_tdcs:
+	for (i = 0; i < tdx_info.nr_tdcs_pages; i++) {
+		if (tdcs_pa[i])
+			free_page((unsigned long)__va(tdcs_pa[i]));
+	}
+	kfree(tdcs_pa);
+	kvm_tdx->tdcs_pa = NULL;
+
+free_tdr:
+	if (tdr_pa)
+		free_page((unsigned long)__va(tdr_pa));
+	kvm_tdx->tdr_pa = 0;
+free_hkid:
+	if (is_hkid_assigned(kvm_tdx))
+		tdx_hkid_free(kvm_tdx);
+	return ret;
+}
+
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)
 {
 	struct kvm_tdx_cmd tdx_cmd;

@@ -142,9 +562,11 @@ static int __init tdx_module_setup(void)
 		return ret;
 	}
 
-	/* Sanitary check just in case. */
 	tdsysinfo = tdx_get_sysinfo();
 	WARN_ON(tdsysinfo->num_cpuid_config > TDX_MAX_NR_CPUID_CONFIGS);
+	tdx_info = (struct tdx_info) {
+		.nr_tdcs_pages = tdsysinfo->tdcs_base_size / PAGE_SIZE,
+	};
 
 	pr_info("TDX is supported.\n");
 	return 0;

@@ -163,6 +585,8 @@ static int __init tdx_cpu_enable_cpu(void *unused)
 
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
 {
+	int max_pkgs;
+	int i;
 	int r;
 
 	if (!enable_ept) {

@@ -170,6 +594,14 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
 		return -EINVAL;
 	}
 
+	max_pkgs = topology_max_packages();
+	tdx_mng_key_config_lock = kcalloc(max_pkgs, sizeof(*tdx_mng_key_config_lock),
+				   GFP_KERNEL);
+	if (!tdx_mng_key_config_lock)
+		return -ENOMEM;
+	for (i = 0; i < max_pkgs; i++)
+		mutex_init(&tdx_mng_key_config_lock[i]);
+
 	/* tdx_enable() in tdx_module_setup() requires cpus lock. */
 	cpus_read_lock();
 	/* TDX requires VMX. */

@@ -193,3 +625,9 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)
 
 	return r;
 }
+
+void tdx_hardware_unsetup(void)
+{
+	/* kfree accepts NULL. */
+	kfree(tdx_mng_key_config_lock);
+}

arch/x86/kvm/vmx/tdx.h +5 -1

index 3860aa351bd9..4b790503e43e 100644
@@ -8,7 +8,11 @@
 
 struct kvm_tdx {
 	struct kvm kvm;
-	/* TDX specific members follow. */
+
+	unsigned long tdr_pa;
+	unsigned long *tdcs_pa;
+
+	int hkid;
 };
 
 struct vcpu_tdx {

arch/x86/kvm/vmx/x86_ops.h +9 -0

index 8118647aa8ca..8f87eeeab97c 100644
@@ -139,19 +139,28 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
 #ifdef CONFIG_INTEL_TDX_HOST
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
+void tdx_hardware_unsetup(void);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
 int tdx_dev_ioctl(void __user *argp);
 
 int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);
+int tdx_vm_init(struct kvm *kvm);
+void tdx_mmu_release_hkid(struct kvm *kvm);
+void tdx_vm_free(struct kvm *kvm);
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
+static inline void tdx_hardware_unsetup(void) {}
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
 
 static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { return -EINVAL; };
+static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; }
+static inline void tdx_mmu_release_hkid(struct kvm *kvm) {}
+static inline void tdx_flush_shadow_all_private(struct kvm *kvm) {}
+static inline void tdx_vm_free(struct kvm *kvm) {}
 static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
 #endif

arch/x86/kvm/x86.c +8 -0

index 7b02dd40ef21..89fd8074c20c 100644
@@ -12398,6 +12398,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 	kvm_page_track_cleanup(kvm);
 	kvm_xen_destroy_vm(kvm);
 	kvm_hv_destroy_vm(kvm);
+	static_call_cond(kvm_x86_vm_free)(kvm);
 }
 
 static void memslot_rmap_free(struct kvm_memory_slot *slot)

@@ -12710,6 +12711,13 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
 
 void kvm_arch_flush_shadow_all(struct kvm *kvm)
 {
+	/*
+	 * kvm_mmu_zap_all() zaps both private and shared page tables.  Before
+	 * tearing down private page tables, TDX requires some TD resources to
+	 * be destroyed (i.e. keyID must have been reclaimed, etc).  Invoke
+	 * kvm_x86_flush_shadow_all_private() for this.
+	 */
+	static_call_cond(kvm_x86_flush_shadow_all_private)(kvm);
 	kvm_mmu_zap_all(kvm);
 }

Isaku Yamahata

2023-03-30 01:02 UTC #67 lore raw

Re: [PATCH v13 019/113] KVM: TDX: create/destroy VM structure

Hi, thanks for review.

Quoted 80 lines · Zhi Wang

Quoted 63 lines · On Sun, 12 Mar 2023 10:55:43 -0700 [email protected] wrote:

arch/x86/kvm/vmx/tdx.c +41 -1

index 8b02e605cfb5..3ede8a726b47 100644
@@ -5,8 +5,9 @@
 
 #include "capabilities.h"
 #include "x86_ops.h"
-#include "x86.h"
 #include "tdx.h"
+#include "tdx_ops.h"
+#include "x86.h"
 
 #undef pr_fmt
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

@@ -46,11 +47,276 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 	return r;
 }
 
+struct tdx_info {
+	u8 nr_tdcs_pages;
+};
+
+/* Info about the TDX module. */
+static struct tdx_info tdx_info;
+
+/*
+ * Some TDX SEAMCALLs (TDH.MNG.CREATE, TDH.PHYMEM.CACHE.WB,
+ * TDH.MNG.KEY.RECLAIMID, TDH.MNG.KEY.FREEID etc) tries to acquire a global lock
+ * internally in TDX module.  If failed, TDX_OPERAND_BUSY is returned without
+ * spinning or waiting due to a constraint on execution time.  It's caller's
+ * responsibility to avoid race (or retry on TDX_OPERAND_BUSY).  Use this mutex
+ * to avoid race in TDX module because the kernel knows better about scheduling.
+ */
+static DEFINE_MUTEX(tdx_lock);
+static struct mutex *tdx_mng_key_config_lock;
+
+static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid)
+{
+	return pa | ((hpa_t)hkid << boot_cpu_data.x86_phys_bits);
+}
+
+static inline bool is_td_created(struct kvm_tdx *kvm_tdx)
+{
+	return kvm_tdx->tdr_pa;
+}
+
+static inline void tdx_hkid_free(struct kvm_tdx *kvm_tdx)
+{
+	tdx_guest_keyid_free(kvm_tdx->hkid);
+	kvm_tdx->hkid = 0;
+}
+
+static inline bool is_hkid_assigned(struct kvm_tdx *kvm_tdx)
+{
+	return kvm_tdx->hkid > 0;
+}
+
 int tdx_hardware_enable(void)
 {
 	return tdx_cpu_enable();
 }

arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 103

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c +10 -0

+static void tdx_clear_page(unsigned long page_pa)
+{
+	const void *zero_page = (const void *) __va(page_to_phys(ZERO_PAGE(0)));
+	void *page = __va(page_pa);
+	unsigned long i;
+
+	if (!cpu_feature_enabled(X86_FEATURE_MOVDIR64B)) {
+		clear_page(page);
+		return;
+	}

Is it possbile to have a TDX machine without MOVDIR64B support? I am not sure if there is any other way for the kernel to clear the posioned cache line.

Quoted 2 lines

If not, there should be a warn/bug at least and check if MOVDIR64B support when initializing the TDX.

Because the latest TDX specification uses movdir64b, it's safe for TDX to assume movdir64b.
I'll add the check to TDX initialization part and drop it from here.

Quoted 18 lines

arch/x86/kvm/vmx/tdx.c +18 -0

+void tdx_mmu_release_hkid(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	cpumask_var_t packages;
+	bool cpumask_allocated;
+	u64 err;
+	int ret;
+	int i;
+
+	if (!is_hkid_assigned(kvm_tdx))
+		return;
+
+	if (!is_td_created(kvm_tdx))
+		goto free_hkid;
+
+	cpumask_allocated = zalloc_cpumask_var(&packages, GFP_KERNEL);
+	cpus_read_lock();
+	for_each_online_cpu(i) {

Quoted 8 lines

arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 223
Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +4 -0
+		if (cpumask_allocated &&
+			cpumask_test_and_set_cpu(topology_physical_package_id(i),
+						packages))
+			continue;
Is this necessary to check cpumask_allocated in the while loop? if cpumask is not succefully allocated, wouldn't it be better to bail out just after it?

No because we can't return error here. It's better to do in-efficiently freeing resources instead of leak.

We can move the check out of loop. But it would be ugly
(if () {cpu loop} else {cpu loop} ) and this function isn't performance critical. Also I think it's okay to depend on compiler optimization for loop invariant. My compiler didn't optimize it in this case, though.

Quoted 33 lines

arch/x86/kvm/vmx/tdx.c +33 -0

+
+		/*
+		 * We can destroy multiple the guest TDs simultaneously.
+		 * Prevent tdh_phymem_cache_wb from returning TDX_BUSY by
+		 * serialization.
+		 */
+		mutex_lock(&tdx_lock);
+		ret = smp_call_on_cpu(i, tdx_do_tdh_phymem_cache_wb, NULL, 1);
+		mutex_unlock(&tdx_lock);
+		if (ret)
+			break;
+	}
+	cpus_read_unlock();
+	free_cpumask_var(packages);
+
+	mutex_lock(&tdx_lock);
+	err = tdh_mng_key_freeid(kvm_tdx->tdr_pa);
+	mutex_unlock(&tdx_lock);
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_MNG_KEY_FREEID, err, NULL);
+		pr_err("tdh_mng_key_freeid failed. HKID %d is leaked.\n",
+			kvm_tdx->hkid);
+		return;
+	}
+
+free_hkid:
+	tdx_hkid_free(kvm_tdx);
+}
+
+void tdx_vm_free(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	int i;

Quoted 8 lines

arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 261
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5 -0
+
+	/* Can't reclaim or free TD pages if teardown failed. */
+	if (is_hkid_assigned(kvm_tdx))
+		return;
+
Better to explain why, as it is common to think even the teardown failed, we should still try to reclaim the pages as many as we can.

Ok, here is the updated comment.
/*
* tdx_mmu_release_hkid() failed to reclaim HKID. Something went wrong
* heavily with TDX module. Give up freeing TD pages. As the function
* already warned, don't warn it again.
*/

Signature · 1 line

Isaku Yamahata <[email protected]>

Zhi Wang

2023-04-02 08:42 UTC #68 lore raw

Re: [PATCH v13 019/113] KVM: TDX: create/destroy VM structure

Quoted 120 lines · On Wed, 29 Mar 2023 18:01:53 -0700 Isaku Yamahata <[email protected]>…

Hi, thanks for review.

Quoted 80 lines · Zhi Wang

Quoted 63 lines · On Sun, 12 Mar 2023 10:55:43 -0700 [email protected] wrote:

arch/x86/kvm/vmx/tdx.c +41 -1

index 8b02e605cfb5..3ede8a726b47 100644
@@ -5,8 +5,9 @@
 
 #include "capabilities.h"
 #include "x86_ops.h"
-#include "x86.h"
 #include "tdx.h"
+#include "tdx_ops.h"
+#include "x86.h"
 
 #undef pr_fmt
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

@@ -46,11 +47,276 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
 	return r;
 }
 
+struct tdx_info {
+	u8 nr_tdcs_pages;
+};
+
+/* Info about the TDX module. */
+static struct tdx_info tdx_info;
+
+/*
+ * Some TDX SEAMCALLs (TDH.MNG.CREATE, TDH.PHYMEM.CACHE.WB,
+ * TDH.MNG.KEY.RECLAIMID, TDH.MNG.KEY.FREEID etc) tries to acquire a global lock
+ * internally in TDX module.  If failed, TDX_OPERAND_BUSY is returned without
+ * spinning or waiting due to a constraint on execution time.  It's caller's
+ * responsibility to avoid race (or retry on TDX_OPERAND_BUSY).  Use this mutex
+ * to avoid race in TDX module because the kernel knows better about scheduling.
+ */
+static DEFINE_MUTEX(tdx_lock);
+static struct mutex *tdx_mng_key_config_lock;
+
+static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid)
+{
+	return pa | ((hpa_t)hkid << boot_cpu_data.x86_phys_bits);
+}
+
+static inline bool is_td_created(struct kvm_tdx *kvm_tdx)
+{
+	return kvm_tdx->tdr_pa;
+}
+
+static inline void tdx_hkid_free(struct kvm_tdx *kvm_tdx)
+{
+	tdx_guest_keyid_free(kvm_tdx->hkid);
+	kvm_tdx->hkid = 0;
+}
+
+static inline bool is_hkid_assigned(struct kvm_tdx *kvm_tdx)
+{
+	return kvm_tdx->hkid > 0;
+}
+
 int tdx_hardware_enable(void)
 {
 	return tdx_cpu_enable();
 }

arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 103

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c +10 -0

+static void tdx_clear_page(unsigned long page_pa)
+{
+	const void *zero_page = (const void *) __va(page_to_phys(ZERO_PAGE(0)));
+	void *page = __va(page_pa);
+	unsigned long i;
+
+	if (!cpu_feature_enabled(X86_FEATURE_MOVDIR64B)) {
+		clear_page(page);
+		return;
+	}

Is it possbile to have a TDX machine without MOVDIR64B support? I am not sure if there is any other way for the kernel to clear the posioned cache line.

Quoted 2 lines

If not, there should be a warn/bug at least and check if MOVDIR64B support when initializing the TDX.

Because the latest TDX specification uses movdir64b, it's safe for TDX to assume movdir64b.
I'll add the check to TDX initialization part and drop it from here.

Quoted 18 lines

arch/x86/kvm/vmx/tdx.c +18 -0

+void tdx_mmu_release_hkid(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	cpumask_var_t packages;
+	bool cpumask_allocated;
+	u64 err;
+	int ret;
+	int i;
+
+	if (!is_hkid_assigned(kvm_tdx))
+		return;
+
+	if (!is_td_created(kvm_tdx))
+		goto free_hkid;
+
+	cpumask_allocated = zalloc_cpumask_var(&packages, GFP_KERNEL);
+	cpus_read_lock();
+	for_each_online_cpu(i) {

Quoted 8 lines

arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 223
Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +4 -0
+		if (cpumask_allocated &&
+			cpumask_test_and_set_cpu(topology_physical_package_id(i),
+						packages))
+			continue;
Is this necessary to check cpumask_allocated in the while loop? if cpumask is not succefully allocated, wouldn't it be better to bail out just after it?

Quoted 8 lines

No because we can't return error here. It's better to do in-efficiently freeing resources instead of leak.
We can move the check out of loop. But it would be ugly
(if () {cpu loop} else {cpu loop} ) and this function isn't performance critical. Also I think it's okay to depend on compiler optimization for loop invariant. My compiler didn't optimize it in this case, though.

Do you mean the tdh_mng_key_freeid() is still required if failing to allocate the cpumask var and do TDH.PHYMEM_CACHE_WB(WBINVD) on each CPU?

Out of curiosity, I took a look on the TDX module source code [1], it seems TDX module has an additional check in TDH.MNG.KEY.FREEID. TDH.MNG.VPFLUSHDONE [2] will mark the pending wbinvd in a bitmap:

...
/**

     * Create the WBINVD_BITMAP per-package.
     * Set to 1 num_of_pkgs bits from the LSB
     */
    global_data_ptr->kot.entries[curr_hkid].wbinvd_bitmap = global_data_ptr->pkg_config_bitmap; /* <----HERE */

    // Set new TD life cycle state
    tdr_ptr->management_fields.lifecycle_state = TD_BLOCKED;

    // Set the proper new KOT entry state
    global_data_ptr->kot.entries[curr_hkid].state = (uint8_t)KOT_STATE_HKID_FLUSHED;

...

And TDH.MNG.KEY.FREEID [3] will check if the pending WBINVD has been performed:

...

    /**
     * If TDH_PHYMEM_CACHE_WB was executed on all packages/cores,
     * set the KOT entry, set the KOT entry state to HKID_FREE.
     */
    curr_hkid = tdr_ptr->key_management_fields.hkid;
    tdx_debug_assert(global_data_ptr->kot.entr/ies[curr_hkid].state == KOT_STATE_HKID_FLUSHED);
    if (global_data_ptr->kot.entries[curr_hkid].wbinvd_bitmap != 0) /* HERE */
    {
        TDX_ERROR("CACHEWB is not complete for this HKID (=%x)\n", curr_hkid);
        return_val = TDX_WBCACHE_NOT_COMPLETE;
        goto EXIT;
    }

...

Guess the conclusion is: if TDH.PHYMEM.CACHE.WB is not performed on each required CPU correctly, TDH.MNG.KEY.FREEID will fail as well. A leak seems the only option (none of us likes a leak, but...).

It would be better that:

1) Leave a comment about the finding above in the code to explain why leak happens (It is always nice to explain the reason of a leak). One sentence will be good enough.

2) If failing to allocate the cpumask, bail out (with the findings above)

...
cpumask_allocated = zalloc_cpumask_var(&packages, GFP_KERNEL); if (!cpumask_allocated)

	return;

...

3) As the reason of the leak will be explained in tdx_mmu_release_hkid(), leaving a pointer in the comment in tdx_vm_free() to refer to tdx_mmu_release_hkid() would be enough, like:

...
/*
* Can't reclaim or free TD pages if teardown failed in
* tdx_mmu_release_hkid().
*/
if (is_hkid_assigned(kvm_tdx))

	return;

...

[1] https://downloadmirror.intel.com/738876/tdx-module-v1.0.01.01.zip
[2] tdx-module/src/vmm_dispatcher/api_calls/tdh_mng_vpflushdone.c
[3] tdx-module/src/vmm_dispatcher/api_calls/tdh_mng_key_freeid.c

Quoted 49 lines

Quoted 34 lines
Quoted 34 lines
arch/x86/kvm/vmx/tdx.c +33 -0
+
+		/*
+		 * We can destroy multiple the guest TDs simultaneously.
+		 * Prevent tdh_phymem_cache_wb from returning TDX_BUSY by
+		 * serialization.
+		 */
+		mutex_lock(&tdx_lock);
+		ret = smp_call_on_cpu(i, tdx_do_tdh_phymem_cache_wb, NULL, 1);
+		mutex_unlock(&tdx_lock);
+		if (ret)
+			break;
+	}
+	cpus_read_unlock();
+	free_cpumask_var(packages);
+
+	mutex_lock(&tdx_lock);
+	err = tdh_mng_key_freeid(kvm_tdx->tdr_pa);
+	mutex_unlock(&tdx_lock);
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_MNG_KEY_FREEID, err, NULL);
+		pr_err("tdh_mng_key_freeid failed. HKID %d is leaked.\n",
+			kvm_tdx->hkid);
+		return;
+	}
+
+free_hkid:
+	tdx_hkid_free(kvm_tdx);
+}
+
+void tdx_vm_free(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	int i;
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 261
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5 -0
+
+	/* Can't reclaim or free TD pages if teardown failed. */
+	if (is_hkid_assigned(kvm_tdx))
+		return;
+
Better to explain why, as it is common to think even the teardown failed, we should still try to reclaim the pages as many as we can.
Ok, here is the updated comment.
/*
* tdx_mmu_release_hkid() failed to reclaim HKID. Something went wrong
* heavily with TDX module. Give up freeing TD pages. As the function
* already warned, don't warn it again.
*/

Isaku Yamahata

2023-04-05 18:16 UTC #69 lore raw

Re: [PATCH v13 019/113] KVM: TDX: create/destroy VM structure

Quoted 73 lines · Zhi Wang

Quoted 27 lines
Quoted 18 lines
Quoted 18 lines
arch/x86/kvm/vmx/tdx.c +18 -0
+void tdx_mmu_release_hkid(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	cpumask_var_t packages;
+	bool cpumask_allocated;
+	u64 err;
+	int ret;
+	int i;
+
+	if (!is_hkid_assigned(kvm_tdx))
+		return;
+
+	if (!is_td_created(kvm_tdx))
+		goto free_hkid;
+
+	cpumask_allocated = zalloc_cpumask_var(&packages, GFP_KERNEL);
+	cpus_read_lock();
+	for_each_online_cpu(i) {
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 223
Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +4 -0
+		if (cpumask_allocated &&
+			cpumask_test_and_set_cpu(topology_physical_package_id(i),
+						packages))
+			continue;
Is this necessary to check cpumask_allocated in the while loop? if cpumask is not succefully allocated, wouldn't it be better to bail out just after it?
Quoted 8 lines
No because we can't return error here. It's better to do in-efficiently freeing resources instead of leak.
We can move the check out of loop. But it would be ugly
(if () {cpu loop} else {cpu loop} ) and this function isn't performance critical. Also I think it's okay to depend on compiler optimization for loop invariant. My compiler didn't optimize it in this case, though.
Do you mean the tdh_mng_key_freeid() is still required if failing to allocate the cpumask var and do TDH.PHYMEM_CACHE_WB(WBINVD) on each CPU?
Out of curiosity, I took a look on the TDX module source code [1], it seems TDX module has an additional check in TDH.MNG.KEY.FREEID. TDH.MNG.VPFLUSHDONE [2] will mark the pending wbinvd in a bitmap:
...
/**
     * Create the WBINVD_BITMAP per-package.
     * Set to 1 num_of_pkgs bits from the LSB
     */
    global_data_ptr->kot.entries[curr_hkid].wbinvd_bitmap = global_data_ptr->pkg_config_bitmap; /* <----HERE */

    // Set new TD life cycle state
    tdr_ptr->management_fields.lifecycle_state = TD_BLOCKED;

    // Set the proper new KOT entry state
    global_data_ptr->kot.entries[curr_hkid].state = (uint8_t)KOT_STATE_HKID_FLUSHED; 
...
And TDH.MNG.KEY.FREEID [3] will check if the pending WBINVD has been performed:
...
    /**
     * If TDH_PHYMEM_CACHE_WB was executed on all packages/cores,
     * set the KOT entry, set the KOT entry state to HKID_FREE.
     */
    curr_hkid = tdr_ptr->key_management_fields.hkid;
    tdx_debug_assert(global_data_ptr->kot.entr/ies[curr_hkid].state == KOT_STATE_HKID_FLUSHED);
    if (global_data_ptr->kot.entries[curr_hkid].wbinvd_bitmap != 0) /* HERE */
    {
        TDX_ERROR("CACHEWB is not complete for this HKID (=%x)\n", curr_hkid);
        return_val = TDX_WBCACHE_NOT_COMPLETE;
        goto EXIT;
    }

Quoted 5 lines

...
Guess the conclusion is: if TDH.PHYMEM.CACHE.WB is not performed on each required CPU correctly, TDH.MNG.KEY.FREEID will fail as well. A leak seems the only option (none of us likes a leak, but...).

Why do we need to leak key? If we fails to allocate cpumask, we can issue TDH.PHYMEM.CACHE.WB on all pCPUs instead of all packages.
If we call TDH.PHYMEM.CACHE.WB multiple times on the same package, it may return error. It's benign. It is suboptimal, but it's much better than leaking hkid.

Signature · 1 line

Isaku Yamahata <[email protected]>

Zhi Wang

2023-04-06 16:05 UTC #70 lore raw

Re: [PATCH v13 019/113] KVM: TDX: create/destroy VM structure

Quoted 78 lines · On Wed, 5 Apr 2023 11:16:18 -0700 Isaku Yamahata <[email protected]>…

Quoted 75 lines · Zhi Wang
Quoted 27 lines
Quoted 18 lines
Quoted 18 lines
arch/x86/kvm/vmx/tdx.c +18 -0
+void tdx_mmu_release_hkid(struct kvm *kvm)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	cpumask_var_t packages;
+	bool cpumask_allocated;
+	u64 err;
+	int ret;
+	int i;
+
+	if (!is_hkid_assigned(kvm_tdx))
+		return;
+
+	if (!is_td_created(kvm_tdx))
+		goto free_hkid;
+
+	cpumask_allocated = zalloc_cpumask_var(&packages, GFP_KERNEL);
+	cpus_read_lock();
+	for_each_online_cpu(i) {
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 223
Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +4 -0
+		if (cpumask_allocated &&
+			cpumask_test_and_set_cpu(topology_physical_package_id(i),
+						packages))
+			continue;
Is this necessary to check cpumask_allocated in the while loop? if cpumask is not succefully allocated, wouldn't it be better to bail out just after it?
Quoted 8 lines
No because we can't return error here. It's better to do in-efficiently freeing resources instead of leak.
We can move the check out of loop. But it would be ugly
(if () {cpu loop} else {cpu loop} ) and this function isn't performance critical. Also I think it's okay to depend on compiler optimization for loop invariant. My compiler didn't optimize it in this case, though.
Do you mean the tdh_mng_key_freeid() is still required if failing to allocate the cpumask var and do TDH.PHYMEM_CACHE_WB(WBINVD) on each CPU?
Out of curiosity, I took a look on the TDX module source code [1], it seems TDX module has an additional check in TDH.MNG.KEY.FREEID. TDH.MNG.VPFLUSHDONE [2] will mark the pending wbinvd in a bitmap:
...
/**
     * Create the WBINVD_BITMAP per-package.
     * Set to 1 num_of_pkgs bits from the LSB
     */
    global_data_ptr->kot.entries[curr_hkid].wbinvd_bitmap = global_data_ptr->pkg_config_bitmap; /* <----HERE */

    // Set new TD life cycle state
    tdr_ptr->management_fields.lifecycle_state = TD_BLOCKED;

    // Set the proper new KOT entry state
    global_data_ptr->kot.entries[curr_hkid].state = (uint8_t)KOT_STATE_HKID_FLUSHED; 
...
And TDH.MNG.KEY.FREEID [3] will check if the pending WBINVD has been performed:
...
    /**
     * If TDH_PHYMEM_CACHE_WB was executed on all packages/cores,
     * set the KOT entry, set the KOT entry state to HKID_FREE.
     */
    curr_hkid = tdr_ptr->key_management_fields.hkid;
    tdx_debug_assert(global_data_ptr->kot.entr/ies[curr_hkid].state == KOT_STATE_HKID_FLUSHED);
    if (global_data_ptr->kot.entries[curr_hkid].wbinvd_bitmap != 0) /* HERE */
    {
        TDX_ERROR("CACHEWB is not complete for this HKID (=%x)\n", curr_hkid);
        return_val = TDX_WBCACHE_NOT_COMPLETE;
        goto EXIT;
    }
...

Quoted 8 lines

Quoted 3 lines
Guess the conclusion is: if TDH.PHYMEM.CACHE.WB is not performed on each required CPU correctly, TDH.MNG.KEY.FREEID will fail as well. A leak seems the only option (none of us likes a leak, but...).
Why do we need to leak key? If we fails to allocate cpumask, we can issue TDH.PHYMEM.CACHE.WB on all pCPUs instead of all packages.
If we call TDH.PHYMEM.CACHE.WB multiple times on the same package, it may return error. It's benign. It is suboptimal, but it's much better than leaking hkid.

I guess I misunderstood the following sentence in the previous email. Now I get it. It is a combination of failure-resolving and normal-resolving.

Quoted 3 lines

Quoted 3 lines
We can move the check out of loop. But it would be ugly
(if () {cpu loop} else {cpu loop} ) and this function isn't performance critical.

Wen, Qian

2023-05-23 08:19 UTC #71 lore raw

Re: [PATCH v13 019/113] KVM: TDX: create/destroy VM structure

Quoted 38 lines · On 3/13/2023 1:55 AM, [email protected] wrote:

From: Isaku Yamahata <[email protected]>
As the first step to create TDX guest, create/destroy VM struct. Assign TDX private Host Key ID (HKID) to the TDX guest for memory encryption and allocate extra pages for the TDX guest. On destruction, free allocated pages, and HKID.
Before tearing down private page tables, TDX requires some resources of the guest TD to be destroyed (i.e. HKID must have been reclaimed, etc). Add flush_shadow_all_private callback before tearing down private page tables for it.
Add vm_free() of kvm_x86_ops hook at the end of kvm_arch_destroy_vm() because some per-VM TDX resources, e.g. TDR, need to be freed after other TDX resources, e.g. HKID, were freed.
Co-developed-by Kai Huang <[email protected]>
Signed-off-by Kai Huang <[email protected]>
Signed-off-by Sean Christopherson <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>
---
Changes v11 -> v12:
- use cpu_feature_enabled().
Changes v10 -> v11:
- Fix doule free in tdx_vm_free() by setting NULL.
- replace struct tdx_td_page tdr and tdcs from struct kvm_tdx with unsigned long
---
 arch/x86/include/asm/kvm-x86-ops.h |   2 +
 arch/x86/include/asm/kvm_host.h    |   2 +
 arch/x86/kvm/vmx/main.c            |  35 ++-
 arch/x86/kvm/vmx/tdx.c             | 442 ++++++++++++++++++++++++++++-
 arch/x86/kvm/vmx/tdx.h             |   6 +-
 arch/x86/kvm/vmx/x86_ops.h         |   9 +
 arch/x86/kvm/x86.c                 |   8 +
 7 files changed, 498 insertions(+), 6 deletions(-)

...

Quoted 19 lines

arch/x86/kvm/vmx/x86_ops.h +4 -0

index 8118647aa8ca..8f87eeeab97c 100644
@@ -139,19 +139,28 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
 #ifdef CONFIG_INTEL_TDX_HOST
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
+void tdx_hardware_unsetup(void);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
 int tdx_dev_ioctl(void __user *argp);
 
 int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);
+int tdx_vm_init(struct kvm *kvm);
+void tdx_mmu_release_hkid(struct kvm *kvm);
+void tdx_vm_free(struct kvm *kvm);
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 #else

arch/x86/kvm/vmx/x86_ops.h void vmx_setup_mce(struct kvm_vcpu *vcpu); · line 162

Quoted 10 lines

arch/x86/kvm/vmx/x86_ops.h +4 -0

 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
+static inline void tdx_hardware_unsetup(void) {}
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
 
 static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { return -EINVAL; };
+static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; }
+static inline void tdx_mmu_release_hkid(struct kvm *kvm) {}
+static inline void tdx_flush_shadow_all_private(struct kvm *kvm) {}

typo? This function tdx_flush_shadow_all_private is not used anywhere.

Quoted 30 lines

arch/x86/kvm/vmx/x86_ops.h +1 -0

+static inline void tdx_vm_free(struct kvm *kvm) {}
 static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
 #endif

arch/x86/kvm/x86.c +8 -0

index 7b02dd40ef21..89fd8074c20c 100644
@@ -12398,6 +12398,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 	kvm_page_track_cleanup(kvm);
 	kvm_xen_destroy_vm(kvm);
 	kvm_hv_destroy_vm(kvm);
+	static_call_cond(kvm_x86_vm_free)(kvm);
 }
 
 static void memslot_rmap_free(struct kvm_memory_slot *slot)

@@ -12710,6 +12711,13 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
 
 void kvm_arch_flush_shadow_all(struct kvm *kvm)
 {
+	/*
+	 * kvm_mmu_zap_all() zaps both private and shared page tables.  Before
+	 * tearing down private page tables, TDX requires some TD resources to
+	 * be destroyed (i.e. keyID must have been reclaimed, etc).  Invoke
+	 * kvm_x86_flush_shadow_all_private() for this.
+	 */
+	static_call_cond(kvm_x86_flush_shadow_all_private)(kvm);
 	kvm_mmu_zap_all(kvm);
 }

Isaku Yamahata

2023-05-28 08:21 UTC #72 lore raw

Re: [PATCH v13 019/113] KVM: TDX: create/destroy VM structure

Quoted 25 lines · Qian

Quoted 25 lines

arch/x86/kvm/vmx/x86_ops.h +5 -0

index 8118647aa8ca..8f87eeeab97c 100644
@@ -139,19 +139,28 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);
 
 #ifdef CONFIG_INTEL_TDX_HOST
 int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);
+void tdx_hardware_unsetup(void);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
 int tdx_dev_ioctl(void __user *argp);
 
 int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);
+int tdx_vm_init(struct kvm *kvm);
+void tdx_mmu_release_hkid(struct kvm *kvm);
+void tdx_vm_free(struct kvm *kvm);
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
+static inline void tdx_hardware_unsetup(void) {}
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };

Quoted 6 lines

arch/x86/kvm/vmx/x86_ops.h void vmx_setup_mce(struct kvm_vcpu *vcpu); · line 162
Quoted 4 lines
arch/x86/kvm/vmx/x86_ops.h +3 -0
 static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { return -EINVAL; };
+static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; }
+static inline void tdx_mmu_release_hkid(struct kvm *kvm) {}
+static inline void tdx_flush_shadow_all_private(struct kvm *kvm) {}
typo? This function tdx_flush_shadow_all_private is not used anywhere.

Right. It was carry over from the old versions. Will remove the line.

Thanks,

Signature · 1 line

Isaku Yamahata <[email protected]>

[email protected]

author

2023-03-12 17:59 UTC #73 lore raw

PATCH v13 20/113 KVM: TDX: initialize VM with TDX specific parameters

From: Isaku Yamahata <[email protected]>

TDX requires additional parameters for TDX VM for confidential execution to protect its confidentiality of its memory contents and its CPU state from any other software, including VMM. When creating guest TD VM before creating vcpu, the number of vcpu, TSC frequency (that is same among vcpus. and it can't be changed.) CPUIDs which is emulated by the TDX module. It means guest can trust those CPUIDs. and sha384 values for measurement.

Add new subcommand, KVM_TDX_INIT_VM, to pass parameters for TDX guest. It assigns encryption key to the TDX guest for memory encryption. TDX encrypts memory per-guest bases. It assigns device model passes per-VM parameters for the TDX guest. The maximum number of vcpus, tsc frequency (TDX guest has fised VM-wide TSC frequency. not per-vcpu. The TDX guest can not change it.), attributes (production or debug), available extended features (which is reflected into guest XCR0, IA32_XSS MSR), cpuids, sha384 measurements, and etc.

This subcommand is called before creating vcpu and KVM_SET_CPUID2, i.e. cpuids configurations aren't available yet. So CPUIDs configuration values needs to be passed in struct kvm_tdx_init_vm. It's device model responsibility to make this cpuid config for KVM_TDX_INIT_VM and KVM_SET_CPUID2.

Signed-off-by Xiaoyao Li <[email protected]>

---
Changes from v11 to v12
- ABI change. Changes struct kvm_tdx_init_vm layout

---
 arch/x86/include/asm/tdx.h            |   3 +
 arch/x86/include/uapi/asm/kvm.h       |  24 +++
 arch/x86/kvm/cpuid.c                  |   7 +
 arch/x86/kvm/cpuid.h                  |   2 +
 arch/x86/kvm/vmx/tdx.c                | 247 ++++++++++++++++++++++++--
 arch/x86/kvm/vmx/tdx.h                |  18 ++
 tools/arch/x86/include/uapi/asm/kvm.h |  33 ++++
 7 files changed, 324 insertions(+), 10 deletions(-)

arch/x86/include/asm/tdx.h +3 -0

index 05870e5ed131..bd09b03d7edd 100644
@@ -101,6 +101,9 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,
 #endif /* CONFIG_INTEL_TDX_GUEST && CONFIG_KVM_GUEST */
 
 #ifdef CONFIG_INTEL_TDX_HOST
+
+/* -1 indicates CPUID leaf with no sub-leaves. */
+#define TDX_CPUID_NO_SUBLEAF	((u32)-1)
 struct tdx_cpuid_config {
 	u32	leaf;
 	u32	sub_leaf;

arch/x86/include/uapi/asm/kvm.h +24 -0

index af4c5bd0af1c..68e8d544afe5 100644
@@ -565,6 +565,7 @@ struct kvm_pmu_event_filter {
 /* Trust Domain eXtension sub-ioctl() commands. */
 enum kvm_tdx_cmd_id {
 	KVM_TDX_CAPABILITIES = 0,
+	KVM_TDX_INIT_VM,
 
 	KVM_TDX_CMD_NR_MAX,
 };

@@ -610,4 +611,27 @@ struct kvm_tdx_capabilities {
 	struct kvm_tdx_cpuid_config cpuid_configs[0];
 };
 
+struct kvm_tdx_init_vm {
+	__u64 attributes;
+	__u64 mrconfigid[6];	/* sha384 digest */
+	__u64 mrowner[6];	/* sha384 digest */
+	__u64 mrownerconfig[6];	/* sha348 digest */
+	/*
+	 * For future extensibility to make sizeof(struct kvm_tdx_init_vm) = 8KB.
+	 * This should be enough given sizeof(TD_PARAMS) = 1024.
+	 * 8KB was chosen given because
+	 * sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES(=256) = 8KB.
+	 */
+	__u64 reserved[1004];
+
+	/*
+	 * KVM_TDX_INIT_VM is called before vcpu creation, thus before
+	 * KVM_SET_CPUID2.
+	 * This configuration supersedes KVM_SET_CPUID2s for VCPUs. The user
+	 * space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with this
+	 * values.
+	 */
+	struct kvm_cpuid2 cpuid;
+};
+
 #endif /* _ASM_X86_KVM_H */

arch/x86/kvm/cpuid.c +7 -0

index 599aebec2d52..9cda9a3a558a 100644
@@ -1386,6 +1386,13 @@ int kvm_dev_ioctl_get_cpuid(struct kvm_cpuid2 *cpuid,
 	return r;
 }
 
+struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2( struct kvm_cpuid2 *cpuid,
+						u32 function, u32 index)
+{
+	return cpuid_entry2_find(cpuid->entries, cpuid->nent, function, index);
+}
+EXPORT_SYMBOL_GPL(kvm_find_cpuid_entry2);
+
 struct kvm_cpuid_entry2 *kvm_find_cpuid_entry_index(struct kvm_vcpu *vcpu,
 						    u32 function, u32 index)
 {

arch/x86/kvm/cpuid.h +2 -0

index b1658c0de847..a0e799297629 100644
@@ -13,6 +13,8 @@ void kvm_set_cpu_caps(void);
 
 void kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu);
 void kvm_update_pv_runtime(struct kvm_vcpu *vcpu);
+struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2(struct kvm_cpuid2 *cpuid,
+					       u32 function, u32 index);
 struct kvm_cpuid_entry2 *kvm_find_cpuid_entry_index(struct kvm_vcpu *vcpu,
 						    u32 function, u32 index);
 struct kvm_cpuid_entry2 *kvm_find_cpuid_entry(struct kvm_vcpu *vcpu,

arch/x86/kvm/vmx/tdx.c +237 ~~-10~~

293 lines not on this page · raw message

arch/x86/kvm/vmx/tdx.h +18 -0

index 4b790503e43e..1e00e75b1c5e 100644
@@ -12,7 +12,11 @@ struct kvm_tdx {
 	unsigned long tdr_pa;
 	unsigned long *tdcs_pa;
 
+	u64 attributes;
+	u64 xfam;
 	int hkid;
+
+	u64 tsc_offset;
 };
 
 struct vcpu_tdx {

@@ -39,6 +43,20 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu)
 {
 	return container_of(vcpu, struct vcpu_tdx, vcpu);
 }
+
+static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 field)
+{
+	struct tdx_module_output out;
+	u64 err;
+
+	err = tdh_mng_rd(kvm_tdx->tdr_pa, TDCS_EXEC(field), &out);
+	if (unlikely(err)) {
+		pr_err("TDH_MNG_RD[EXEC.0x%x] failed: 0x%llx\n", field, err);
+		return 0;
+	}
+	return out.r8;
+}
+
 #else
 struct kvm_tdx {
 	struct kvm kvm;

tools/arch/x86/include/uapi/asm/kvm.h +33 -0

index af4c5bd0af1c..c0f011384934 100644
@@ -565,6 +565,7 @@ struct kvm_pmu_event_filter {
 /* Trust Domain eXtension sub-ioctl() commands. */
 enum kvm_tdx_cmd_id {
 	KVM_TDX_CAPABILITIES = 0,
+	KVM_TDX_INIT_VM,
 
 	KVM_TDX_CMD_NR_MAX,
 };

@@ -610,4 +611,36 @@ struct kvm_tdx_capabilities {
 	struct kvm_tdx_cpuid_config cpuid_configs[0];
 };
 
+struct kvm_tdx_init_vm {
+	__u64 attributes;
+	__u32 max_vcpus;
+	__u32 padding;
+	__u64 mrconfigid[6];    /* sha384 digest */
+	__u64 mrowner[6];       /* sha384 digest */
+	__u64 mrownerconfig[6]; /* sha348 digest */
+	union {
+		/*
+		 * KVM_TDX_INIT_VM is called before vcpu creation, thus before
+		 * KVM_SET_CPUID2.  CPUID configurations needs to be passed.
+		 *
+		 * This configuration supersedes KVM_SET_CPUID{,2}.
+		 * The user space VMM, e.g. qemu, should make them consistent
+		 * with this values.
+		 * sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES(256)
+		 * = 8KB.
+		 */
+		struct {
+			struct kvm_cpuid2 cpuid;
+			/* 8KB with KVM_MAX_CPUID_ENTRIES. */
+			struct kvm_cpuid_entry2 entries[];
+		};
+		/*
+		 * For future extensibility.
+		 * The size(struct kvm_tdx_init_vm) = 16KB.
+		 * This should be enough given sizeof(TD_PARAMS) = 1024
+		 */
+		__u64 reserved[2028];
+	};
+};
+
 #endif /* _ASM_X86_KVM_H */

Signature · 1 line

2.25.1

Zhi Wang

2023-04-04 19:24 UTC #74 lore raw

Re: [PATCH v13 020/113] KVM: TDX: initialize VM with TDX specific parameters

Quoted 7 lines · On Sun, 12 Mar 2023 10:55:44 -0700 [email protected] wrote:

From: Isaku Yamahata <[email protected]>
TDX requires additional parameters for TDX VM for confidential execution to protect its confidentiality of its memory contents and its CPU state from any other software, including VMM. When creating guest TD VM before creating vcpu, the number of vcpu, TSC frequency (that is same among vcpus. and it can't be changed.) CPUIDs which is emulated by the TDX

^ This sentence seems broken. "

Quoted 72 lines

module. It means guest can trust those CPUIDs. and sha384 values for measurement.
Add new subcommand, KVM_TDX_INIT_VM, to pass parameters for TDX guest. It assigns encryption key to the TDX guest for memory encryption. TDX encrypts memory per-guest bases. It assigns device model passes per-VM parameters for the TDX guest. The maximum number of vcpus, tsc frequency (TDX guest has fised VM-wide TSC frequency. not per-vcpu. The TDX guest can not change it.), attributes (production or debug), available extended features (which is reflected into guest XCR0, IA32_XSS MSR), cpuids, sha384 measurements, and etc.
This subcommand is called before creating vcpu and KVM_SET_CPUID2, i.e. cpuids configurations aren't available yet. So CPUIDs configuration values needs to be passed in struct kvm_tdx_init_vm. It's device model responsibility to make this cpuid config for KVM_TDX_INIT_VM and KVM_SET_CPUID2.
Signed-off-by Xiaoyao Li <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>
---
Changes from v11 to v12
- ABI change. Changes struct kvm_tdx_init_vm layout
---
 arch/x86/include/asm/tdx.h            |   3 +
 arch/x86/include/uapi/asm/kvm.h       |  24 +++
 arch/x86/kvm/cpuid.c                  |   7 +
 arch/x86/kvm/cpuid.h                  |   2 +
 arch/x86/kvm/vmx/tdx.c                | 247 ++++++++++++++++++++++++--
 arch/x86/kvm/vmx/tdx.h                |  18 ++
 tools/arch/x86/include/uapi/asm/kvm.h |  33 ++++
 7 files changed, 324 insertions(+), 10 deletions(-)
arch/x86/include/asm/tdx.h +3 -0
index 05870e5ed131..bd09b03d7edd 100644
@@ -101,6 +101,9 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,
 #endif /* CONFIG_INTEL_TDX_GUEST && CONFIG_KVM_GUEST */
 
 #ifdef CONFIG_INTEL_TDX_HOST
+
+/* -1 indicates CPUID leaf with no sub-leaves. */
+#define TDX_CPUID_NO_SUBLEAF	((u32)-1)
 struct tdx_cpuid_config {
 	u32	leaf;
 	u32	sub_leaf;
arch/x86/include/uapi/asm/kvm.h +10 -0
index af4c5bd0af1c..68e8d544afe5 100644
@@ -565,6 +565,7 @@ struct kvm_pmu_event_filter {
 /* Trust Domain eXtension sub-ioctl() commands. */
 enum kvm_tdx_cmd_id {
 	KVM_TDX_CAPABILITIES = 0,
+	KVM_TDX_INIT_VM,
 
 	KVM_TDX_CMD_NR_MAX,
 };
@@ -610,4 +611,27 @@ struct kvm_tdx_capabilities {
 	struct kvm_tdx_cpuid_config cpuid_configs[0];
 };
 
+struct kvm_tdx_init_vm {
+	__u64 attributes;
+	__u64 mrconfigid[6];	/* sha384 digest */
+	__u64 mrowner[6];	/* sha384 digest */
+	__u64 mrownerconfig[6];	/* sha348 digest */
+	/*
+	 * For future extensibility to make sizeof(struct kvm_tdx_init_vm) = 8KB.
+	 * This should be enough given sizeof(TD_PARAMS) = 1024.
+	 * 8KB was chosen given because

arch/x86/include/uapi/asm/kvm.h struct kvm_tdx_capabilities { · line 632

Quoted 10 lines

arch/x86/include/uapi/asm/kvm.h +10 -0

+	 * sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES(=256) = 8KB.
+	 */
+	__u64 reserved[1004];
+
+	/*
+	 * KVM_TDX_INIT_VM is called before vcpu creation, thus before
+	 * KVM_SET_CPUID2.
+	 * This configuration supersedes KVM_SET_CPUID2s for VCPUs. The user
+	 * space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with this
+	 * values.

^
It would be nice to have one more sentence to talk about the
consequence if QEMU configures different CPUIDs in KVM_TDX_INIT_VM and the later SET_CPUID2.

Quoted 425 lines

arch/x86/include/uapi/asm/kvm.h +4 -0

+	 */
+	struct kvm_cpuid2 cpuid;
+};
+
 #endif /* _ASM_X86_KVM_H */

arch/x86/kvm/cpuid.c +7 -0

index 599aebec2d52..9cda9a3a558a 100644
@@ -1386,6 +1386,13 @@ int kvm_dev_ioctl_get_cpuid(struct kvm_cpuid2 *cpuid,
 	return r;
 }
 
+struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2( struct kvm_cpuid2 *cpuid,
+						u32 function, u32 index)
+{
+	return cpuid_entry2_find(cpuid->entries, cpuid->nent, function, index);
+}
+EXPORT_SYMBOL_GPL(kvm_find_cpuid_entry2);
+
 struct kvm_cpuid_entry2 *kvm_find_cpuid_entry_index(struct kvm_vcpu *vcpu,
 						    u32 function, u32 index)
 {

arch/x86/kvm/cpuid.h +2 -0

index b1658c0de847..a0e799297629 100644
@@ -13,6 +13,8 @@ void kvm_set_cpu_caps(void);
 
 void kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu);
 void kvm_update_pv_runtime(struct kvm_vcpu *vcpu);
+struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2(struct kvm_cpuid2 *cpuid,
+					       u32 function, u32 index);
 struct kvm_cpuid_entry2 *kvm_find_cpuid_entry_index(struct kvm_vcpu *vcpu,
 						    u32 function, u32 index);
 struct kvm_cpuid_entry2 *kvm_find_cpuid_entry(struct kvm_vcpu *vcpu,

arch/x86/kvm/vmx/tdx.c +237 ~~-10~~

index 3ede8a726b47..496410ec6334 100644
@@ -6,7 +6,6 @@
 #include "capabilities.h"
 #include "x86_ops.h"
 #include "tdx.h"
-#include "tdx_ops.h"
 #include "x86.h"
 
 #undef pr_fmt

@@ -303,18 +302,21 @@ static int tdx_do_tdh_mng_key_config(void *param)
 	return 0;
 }
 
-static int __tdx_td_init(struct kvm *kvm);
-
 int tdx_vm_init(struct kvm *kvm)
 {
+	/*
+	 * This function initializes only KVM software construct.  It doesn't
+	 * initialize TDX stuff, e.g. TDCS, TDR, TDCX, HKID etc.
+	 * It is handled by KVM_TDX_INIT_VM, __tdx_td_init().
+	 */
+
 	/*
 	 * TDX has its own limit of the number of vcpus in addition to
 	 * KVM_MAX_VCPUS.
 	 */
 	kvm->max_vcpus = min(kvm->max_vcpus, TDX_MAX_VCPUS);
 
-	/* Place holder for TDX specific logic. */
-	return __tdx_td_init(kvm);
+	return 0;
 }
 
 int tdx_dev_ioctl(void __user *argp)

@@ -368,9 +370,162 @@ int tdx_dev_ioctl(void __user *argp)
 	return 0;
 }
 
-static int __tdx_td_init(struct kvm *kvm)
+static void setup_tdparams_eptp_controls(struct kvm_cpuid2 *cpuid, struct td_params *td_params)
+{
+	const struct kvm_cpuid_entry2 *entry;
+	int max_pa = 36;
+
+	entry = kvm_find_cpuid_entry2(cpuid, 0x80000008, 0);
+	if (entry)
+		max_pa = entry->eax & 0xff;
+
+	td_params->eptp_controls = VMX_EPTP_MT_WB;
+	/*
+	 * No CPU supports 4-level && max_pa > 48.
+	 * "5-level paging and 5-level EPT" section 4.1 4-level EPT
+	 * "4-level EPT is limited to translating 48-bit guest-physical
+	 *  addresses."
+	 * cpu_has_vmx_ept_5levels() check is just in case.
+	 */
+	if (cpu_has_vmx_ept_5levels() && max_pa > 48) {
+		td_params->eptp_controls |= VMX_EPTP_PWL_5;
+		td_params->exec_controls |= TDX_EXEC_CONTROL_MAX_GPAW;
+	} else {
+		td_params->eptp_controls |= VMX_EPTP_PWL_4;
+	}
+}
+
+static void setup_tdparams_cpuids(const struct tdsysinfo_struct *tdsysinfo,
+				  struct kvm_cpuid2 *cpuid,
+				  struct td_params *td_params)
+{
+	int i;
+
+	/*
+	 * td_params.cpuid_values: The number and the order of cpuid_value must
+	 * be same to the one of struct tdsysinfo.{num_cpuid_config, cpuid_configs}
+	 * It's assumed that td_params was zeroed.
+	 */
+	for (i = 0; i < tdsysinfo->num_cpuid_config; i++) {
+		const struct tdx_cpuid_config *config = &tdsysinfo->cpuid_configs[i];
+		/* TDX_CPUID_NO_SUBLEAF in TDX CPUID_CONFIG means index = 0. */
+		u32 index = config->sub_leaf == TDX_CPUID_NO_SUBLEAF ? 0: config->sub_leaf;
+		const struct kvm_cpuid_entry2 *entry =
+			kvm_find_cpuid_entry2(cpuid, config->leaf, index);
+		struct tdx_cpuid_value *value = &td_params->cpuid_values[i];
+
+		if (!entry)
+			continue;
+
+		/*
+		 * tdsysinfo.cpuid_configs[].{eax, ebx, ecx, edx}
+		 * bit 1 means it can be configured to zero or one.
+		 * bit 0 means it must be zero.
+		 * Mask out non-configurable bits.
+		 */
+		value->eax = entry->eax & config->eax;
+		value->ebx = entry->ebx & config->ebx;
+		value->ecx = entry->ecx & config->ecx;
+		value->edx = entry->edx & config->edx;
+	}
+}
+
+static int setup_tdparams_xfam(struct kvm_cpuid2 *cpuid, struct td_params *td_params)
+{
+	const struct kvm_cpuid_entry2 *entry;
+	u64 guest_supported_xcr0;
+	u64 guest_supported_xss;
+
+	/* Setup td_params.xfam */
+	entry = kvm_find_cpuid_entry2(cpuid, 0xd, 0);
+	if (entry)
+		guest_supported_xcr0 = (entry->eax | ((u64)entry->edx << 32));
+	else
+		guest_supported_xcr0 = 0;
+	guest_supported_xcr0 &= kvm_caps.supported_xcr0;
+
+	entry = kvm_find_cpuid_entry2(cpuid, 0xd, 1);
+	if (entry)
+		guest_supported_xss = (entry->ecx | ((u64)entry->edx << 32));
+	else
+		guest_supported_xss = 0;
+	/* PT can be exposed to TD guest regardless of KVM's XSS support */
+	guest_supported_xss &= (kvm_caps.supported_xss | XFEATURE_MASK_PT);
+
+	td_params->xfam = guest_supported_xcr0 | guest_supported_xss;
+	if (td_params->xfam & XFEATURE_MASK_LBR) {
+		/*
+		 * TODO: once KVM supports LBR(save/restore LBR related
+		 * registers around TDENTER), remove this guard.
+		 */
+		pr_warn("TD doesn't support LBR yet. KVM needs to save/restore "
+			"IA32_LBR_DEPTH properly.\n");
+		return -EOPNOTSUPP;
+	}
+
+	if (td_params->xfam & XFEATURE_MASK_XTILE) {
+		/*
+		 * TODO: once KVM supports AMX(save/restore AMX related
+		 * registers around TDENTER), remove this guard.
+		 */
+		pr_warn("TD doesn't support AMX yet. KVM needs to save/restore "
+			"IA32_XFD, IA32_XFD_ERR properly.\n");
+		return -EOPNOTSUPP;
+	}
+
+	return 0;
+}
+
+static int setup_tdparams(struct kvm *kvm, struct td_params *td_params,
+			struct kvm_tdx_init_vm *init_vm)
+{
+	struct kvm_cpuid2 *cpuid = &init_vm->cpuid;
+	const struct tdsysinfo_struct *tdsysinfo;
+	int ret;
+
+	tdsysinfo = tdx_get_sysinfo();
+	if (!tdsysinfo)
+		return -ENOTSUPP;
+	if (kvm->created_vcpus)
+		return -EBUSY;
+
+	if (td_params->attributes & TDX_TD_ATTRIBUTE_PERFMON) {
+		/*
+		 * TODO: save/restore PMU related registers around TDENTER.
+		 * Once it's done, remove this guard.
+		 */
+		pr_warn("TD doesn't support perfmon yet. KVM needs to save/restore "
+			"host perf registers properly.\n");
+		return -EOPNOTSUPP;
+	}
+
+	td_params->max_vcpus = kvm->max_vcpus;
+	td_params->attributes = init_vm->attributes;
+	td_params->tsc_frequency = TDX_TSC_KHZ_TO_25MHZ(kvm->arch.default_tsc_khz);
+
+	setup_tdparams_eptp_controls(cpuid, td_params);
+	setup_tdparams_cpuids(tdsysinfo, cpuid, td_params);
+	ret = setup_tdparams_xfam(cpuid, td_params);
+	if (ret)
+		return ret;
+
+#define MEMCPY_SAME_SIZE(dst, src)				\
+	do {							\
+		BUILD_BUG_ON(sizeof(dst) != sizeof(src));	\
+		memcpy((dst), (src), sizeof(dst));		\
+	} while (0)
+
+	MEMCPY_SAME_SIZE(td_params->mrconfigid, init_vm->mrconfigid);
+	MEMCPY_SAME_SIZE(td_params->mrowner, init_vm->mrowner);
+	MEMCPY_SAME_SIZE(td_params->mrownerconfig, init_vm->mrownerconfig);
+
+	return 0;
+}
+
+static int __tdx_td_init(struct kvm *kvm, struct td_params *td_params)
 {
 	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	struct tdx_module_output out;
 	cpumask_var_t packages;
 	unsigned long *tdcs_pa = NULL;
 	unsigned long tdr_pa = 0;

@@ -484,10 +639,13 @@ static int __tdx_td_init(struct kvm *kvm)
 		}
 	}
 
-	/*
-	 * Note, TDH_MNG_INIT cannot be invoked here.  TDH_MNG_INIT requires a dedicated
-	 * ioctl() to define the configure CPUID values for the TD.
-	 */
+	err = tdh_mng_init(kvm_tdx->tdr_pa, __pa(td_params), &out);
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_MNG_INIT, err, &out);
+		ret = -EIO;
+		goto teardown;
+	}
+
 	return 0;
 
 	/*

@@ -522,6 +680,72 @@ static int __tdx_td_init(struct kvm *kvm)
 	return ret;
 }
 
+static int tdx_td_init(struct kvm *kvm, struct kvm_tdx_cmd *cmd)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
+	struct kvm_tdx_init_vm *init_vm = NULL;
+	struct td_params *td_params = NULL;
+	int ret;
+
+	BUILD_BUG_ON(sizeof(*init_vm) != 8 * 1024);
+	BUILD_BUG_ON(sizeof(struct td_params) != 1024);
+
+	if (is_hkid_assigned(kvm_tdx))
+		return -EINVAL;
+
+	if (cmd->flags)
+		return -EINVAL;
+
+	init_vm = kzalloc(sizeof(*init_vm) +
+			  sizeof(init_vm->cpuid.entries[0]) * KVM_MAX_CPUID_ENTRIES,
+			  GFP_KERNEL);
+	if (!init_vm)
+		return -ENOMEM;
+	if (copy_from_user(init_vm, (void __user *)cmd->data, sizeof(*init_vm))) {
+		ret = -EFAULT;
+		goto out;
+	}
+	if (init_vm->cpuid.nent > KVM_MAX_CPUID_ENTRIES) {
+		ret = -E2BIG;
+		goto out;
+	}
+	if (copy_from_user(init_vm->cpuid.entries,
+			   (void __user *)cmd->data + sizeof(*init_vm),
+			   sizeof(init_vm->cpuid.entries[0]) * init_vm->cpuid.nent)) {
+		ret = -EFAULT;
+		goto out;
+	}
+
+	if (init_vm->cpuid.padding) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	td_params = kzalloc(sizeof(struct td_params), GFP_KERNEL);
+	if (!td_params) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	ret = setup_tdparams(kvm, td_params, init_vm);
+	if (ret)
+		goto out;
+
+	ret = __tdx_td_init(kvm, td_params);
+	if (ret)
+		goto out;
+
+	kvm_tdx->tsc_offset = td_tdcs_exec_read64(kvm_tdx, TD_TDCS_EXEC_TSC_OFFSET);
+	kvm_tdx->attributes = td_params->attributes;
+	kvm_tdx->xfam = td_params->xfam;
+
+out:
+	/* kfree() accepts NULL. */
+	kfree(init_vm);
+	kfree(td_params);
+	return ret;
+}
+
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)
 {
 	struct kvm_tdx_cmd tdx_cmd;

@@ -535,6 +759,9 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)
 	mutex_lock(&kvm->lock);
 
 	switch (tdx_cmd.id) {
+	case KVM_TDX_INIT_VM:
+		r = tdx_td_init(kvm, &tdx_cmd);
+		break;
 	default:
 		r = -EINVAL;
 		goto out;

arch/x86/kvm/vmx/tdx.h +18 -0

index 4b790503e43e..1e00e75b1c5e 100644
@@ -12,7 +12,11 @@ struct kvm_tdx {
 	unsigned long tdr_pa;
 	unsigned long *tdcs_pa;
 
+	u64 attributes;
+	u64 xfam;
 	int hkid;
+
+	u64 tsc_offset;
 };
 
 struct vcpu_tdx {

@@ -39,6 +43,20 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu)
 {
 	return container_of(vcpu, struct vcpu_tdx, vcpu);
 }
+
+static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 field)
+{
+	struct tdx_module_output out;
+	u64 err;
+
+	err = tdh_mng_rd(kvm_tdx->tdr_pa, TDCS_EXEC(field), &out);
+	if (unlikely(err)) {
+		pr_err("TDH_MNG_RD[EXEC.0x%x] failed: 0x%llx\n", field, err);
+		return 0;
+	}
+	return out.r8;
+}
+
 #else
 struct kvm_tdx {
 	struct kvm kvm;

tools/arch/x86/include/uapi/asm/kvm.h +33 -0

index af4c5bd0af1c..c0f011384934 100644
@@ -565,6 +565,7 @@ struct kvm_pmu_event_filter {
 /* Trust Domain eXtension sub-ioctl() commands. */
 enum kvm_tdx_cmd_id {
 	KVM_TDX_CAPABILITIES = 0,
+	KVM_TDX_INIT_VM,
 
 	KVM_TDX_CMD_NR_MAX,
 };

@@ -610,4 +611,36 @@ struct kvm_tdx_capabilities {
 	struct kvm_tdx_cpuid_config cpuid_configs[0];
 };
 
+struct kvm_tdx_init_vm {
+	__u64 attributes;
+	__u32 max_vcpus;
+	__u32 padding;
+	__u64 mrconfigid[6];    /* sha384 digest */
+	__u64 mrowner[6];       /* sha384 digest */
+	__u64 mrownerconfig[6]; /* sha348 digest */
+	union {
+		/*
+		 * KVM_TDX_INIT_VM is called before vcpu creation, thus before
+		 * KVM_SET_CPUID2.  CPUID configurations needs to be passed.
+		 *
+		 * This configuration supersedes KVM_SET_CPUID{,2}.
+		 * The user space VMM, e.g. qemu, should make them consistent
+		 * with this values.
+		 * sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES(256)
+		 * = 8KB.
+		 */
+		struct {
+			struct kvm_cpuid2 cpuid;
+			/* 8KB with KVM_MAX_CPUID_ENTRIES. */
+			struct kvm_cpuid_entry2 entries[];
+		};
+		/*
+		 * For future extensibility.
+		 * The size(struct kvm_tdx_init_vm) = 16KB.
+		 * This should be enough given sizeof(TD_PARAMS) = 1024
+		 */
+		__u64 reserved[2028];
+	};
+};
+
 #endif /* _ASM_X86_KVM_H */

Isaku Yamahata

2023-04-05 18:50 UTC #75 lore raw

Re: [PATCH v13 020/113] KVM: TDX: initialize VM with TDX specific parameters

Quoted 11 lines · Zhi Wang

Quoted 7 lines · On Sun, 12 Mar 2023 10:55:44 -0700 [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX requires additional parameters for TDX VM for confidential execution to protect its confidentiality of its memory contents and its CPU state from any other software, including VMM. When creating guest TD VM before creating vcpu, the number of vcpu, TSC frequency (that is same among vcpus. and it can't be changed.) CPUIDs which is emulated by the TDX
^ This sentence seems broken. "

Here is the revised one.

TDX requires additional parameters for TDX VM for confidential execution to protect the confidentiality of its memory contents and CPU state from any other software, including VMM. When creating a guest TD VM before creating vcpu, the number of vcpu, TSC frequency (the values are the same among vcpus, and it can't change.) CPUIDs which the TDX module emulates. Guest TDs can trust those CPUIDs and sha384 values for measurement.

Add a new subcommand, KVM_TDX_INIT_VM, to pass parameters for the TDX guest. It assigns an encryption key to the TDX guest for memory encryption. TDX encrypts memory per guest basis. The device model, say qemu, passes per-VM parameters for the TDX guest. The maximum number of vcpus, TSC frequency (TDX guest has fixed VM-wide TSC frequency, not per vcpu. The TDX guest can not change it.), attributes (production or debug), available extended features (which configure guest XCR0, IA32_XSS MSR), CPUIDs, sha384 measurements, etc.

Call this subcommand before creating vcpu and KVM_SET_CPUID2, i.e. CPUID configurations aren't available yet. So CPUIDs configuration values need to be passed in struct kvm_tdx_init_vm. The device model's responsibility to make this CPUID config for KVM_TDX_INIT_VM and KVM_SET_CPUID2.

Quoted 31 lines

arch/x86/include/uapi/asm/kvm.h +16 -0

index af4c5bd0af1c..68e8d544afe5 100644
@@ -565,6 +565,7 @@ struct kvm_pmu_event_filter {
 /* Trust Domain eXtension sub-ioctl() commands. */
 enum kvm_tdx_cmd_id {
 	KVM_TDX_CAPABILITIES = 0,
+	KVM_TDX_INIT_VM,
 
 	KVM_TDX_CMD_NR_MAX,
 };

@@ -610,4 +611,27 @@ struct kvm_tdx_capabilities {
 	struct kvm_tdx_cpuid_config cpuid_configs[0];
 };
 
+struct kvm_tdx_init_vm {
+	__u64 attributes;
+	__u64 mrconfigid[6];	/* sha384 digest */
+	__u64 mrowner[6];	/* sha384 digest */
+	__u64 mrownerconfig[6];	/* sha348 digest */
+	/*
+	 * For future extensibility to make sizeof(struct kvm_tdx_init_vm) = 8KB.
+	 * This should be enough given sizeof(TD_PARAMS) = 1024.
+	 * 8KB was chosen given because
+	 * sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES(=256) = 8KB.
+	 */
+	__u64 reserved[1004];
+
+	/*
+	 * KVM_TDX_INIT_VM is called before vcpu creation, thus before

Quoted 8 lines

arch/x86/include/uapi/asm/kvm.h struct kvm_tdx_capabilities { · line 632
Quoted 4 lines
arch/x86/include/uapi/asm/kvm.h +4 -0
+	 * KVM_SET_CPUID2.
+	 * This configuration supersedes KVM_SET_CPUID2s for VCPUs. The user
+	 * space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with this
+	 * values.
^
It would be nice to have one more sentence to talk about the
consequence if QEMU configures different CPUIDs in KVM_TDX_INIT_VM and the later SET_CPUID2.

Here is the updated one.
/*
* Call KVM_TDX_INIT_VM before vcpu creation, thus before
* KVM_SET_CPUID2.
* This configuration supersedes KVM_SET_CPUID2s for VCPUs because the
* TDX module directly virtualizes those CPUIDs without VMM. The user
* space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with
* those values. If it doesn't, KVM may have wrong idea of vCPUIDs of
* the guest, and KVM may wrongly emulate CPUIDs or MSRs that the TDX
* module doesn't virtualize.
*/

Signature · 1 line

Isaku Yamahata <[email protected]>

[email protected]

author

2023-03-12 17:59 UTC #76 lore raw

PATCH v13 21/113 KVM: TDX: Make pmu_intel.c ignore guest TD case

From: Isaku Yamahata <[email protected]>

Because TDX KVM doesn't support PMU yet (it's future work of TDX KVM support as another patch series) and pmu_intel.c touches vmx specific structure in vcpu initialization, as workaround add dummy structure to struct vcpu_tdx and pmu_intel.c can ignore TDX case.

---
 arch/x86/kvm/vmx/pmu_intel.c | 46 +++++++++++++++++++++++++++++++++++-
 arch/x86/kvm/vmx/pmu_intel.h | 28 ++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.h       |  8 ++++++-
 arch/x86/kvm/vmx/vmx.c       |  2 +-
 arch/x86/kvm/vmx/vmx.h       | 32 +------------------------
 5 files changed, 82 insertions(+), 34 deletions(-)
 create mode 100644 arch/x86/kvm/vmx/pmu_intel.h

arch/x86/kvm/vmx/pmu_intel.c +45 -1

index e8a3be0b9df9..df1f4ddfa72d 100644
@@ -19,6 +19,7 @@
 #include "lapic.h"
 #include "nested.h"
 #include "pmu.h"
+#include "tdx.h"
 
 #define MSR_PMC_FULL_WIDTH_BIT      (MSR_IA32_PMC0 - MSR_IA32_PERFCTR0)

@@ -40,6 +41,26 @@ static struct {
 /* mapping between fixed pmc index and intel_arch_events array */
 static int fixed_pmc_events[] = {1, 0, 7};
 
+struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_INTEL_TDX_HOST
+	if (is_td_vcpu(vcpu))
+		return &to_tdx(vcpu)->lbr_desc;
+#endif
+
+	return &to_vmx(vcpu)->lbr_desc;
+}
+
+struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_INTEL_TDX_HOST
+	if (is_td_vcpu(vcpu))
+		return &to_tdx(vcpu)->lbr_desc.records;
+#endif
+
+	return &to_vmx(vcpu)->lbr_desc.records;
+}
+
 static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 {
 	struct kvm_pmc *pmc;

@@ -172,6 +193,23 @@ static inline struct kvm_pmc *get_fw_gp_pmc(struct kvm_pmu *pmu, u32 msr)
 	return get_gp_pmc(pmu, msr, MSR_IA32_PMC0);
 }
 
+bool intel_pmu_lbr_is_compatible(struct kvm_vcpu *vcpu)
+{
+	if (is_td_vcpu(vcpu))
+		return false;
+	return cpuid_model_is_consistent(vcpu);
+}
+
+bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu)
+{
+	struct x86_pmu_lbr *lbr = vcpu_to_lbr_records(vcpu);
+
+	if (is_td_vcpu(vcpu))
+		return false;
+
+	return lbr->nr && (vcpu_get_perf_capabilities(vcpu) & PMU_CAP_LBR_FMT);
+}
+
 static bool intel_pmu_is_valid_lbr_msr(struct kvm_vcpu *vcpu, u32 index)
 {
 	struct x86_pmu_lbr *records = vcpu_to_lbr_records(vcpu);

@@ -282,6 +320,9 @@ int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu)
 					PERF_SAMPLE_BRANCH_USER,
 	};
 
+	if (WARN_ON_ONCE(is_td_vcpu(vcpu)))
+		return 0;
+
 	if (unlikely(lbr_desc->event)) {
 		__set_bit(INTEL_PMC_IDX_FIXED_VLBR, pmu->pmc_in_use);
 		return 0;

@@ -591,7 +632,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 		INTEL_PMC_MAX_GENERIC, pmu->nr_arch_fixed_counters);
 
 	perf_capabilities = vcpu_get_perf_capabilities(vcpu);
-	if (cpuid_model_is_consistent(vcpu) &&
+	if (intel_pmu_lbr_is_compatible(vcpu) &&
 	    (perf_capabilities & PMU_CAP_LBR_FMT))
 		x86_perf_get_lbr(&lbr_desc->records);
 	else

@@ -647,6 +688,9 @@ static void intel_pmu_reset(struct kvm_vcpu *vcpu)
 	struct kvm_pmc *pmc = NULL;
 	int i;
 
+	if (is_td_vcpu(vcpu))
+		return;
+
 	for (i = 0; i < KVM_INTEL_PMC_MAX_GENERIC; i++) {
 		pmc = &pmu->gp_counters[i];

arch/x86/kvm/vmx/pmu_intel.h new file +28 -0

new file mode 100644
index 000000000000..66bba47c1269
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __KVM_X86_VMX_PMU_INTEL_H
+#define  __KVM_X86_VMX_PMU_INTEL_H
+
+struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu);
+struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu);
+
+bool intel_pmu_lbr_is_compatible(struct kvm_vcpu *vcpu);
+bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu);
+int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu);
+
+struct lbr_desc {
+	/* Basic info about guest LBR records. */
+	struct x86_pmu_lbr records;
+
+	/*
+	 * Emulate LBR feature via passthrough LBR registers when the
+	 * per-vcpu guest LBR event is scheduled on the current pcpu.
+	 *
+	 * The records may be inaccurate if the host reclaims the LBR.
+	 */
+	struct perf_event *event;
+
+	/* True if LBRs are marked as not intercepted in the MSR bitmap */
+	bool msr_passthrough;
+};
+
+#endif /* __KVM_X86_VMX_PMU_INTEL_H */

arch/x86/kvm/vmx/tdx.h +7 -1

index 1e00e75b1c5e..5728820fed5e 100644
@@ -4,6 +4,7 @@
 
 #ifdef CONFIG_INTEL_TDX_HOST
 
+#include "pmu_intel.h"
 #include "tdx_ops.h"
 
 struct kvm_tdx {

@@ -21,7 +22,12 @@ struct kvm_tdx {
 
 struct vcpu_tdx {
 	struct kvm_vcpu	vcpu;
-	/* TDX specific members follow. */
+
+	/*
+	 * Dummy to make pmu_intel not corrupt memory.
+	 * TODO: Support PMU for TDX.  Future work.
+	 */
+	struct lbr_desc lbr_desc;
 };
 
 static inline bool is_td(struct kvm *kvm)

arch/x86/kvm/vmx/vmx.c +1 -1

index d23830d92f61..f9e9fd7fde2c 100644
@@ -2432,7 +2432,7 @@ int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			if ((data & PMU_CAP_LBR_FMT) !=
 			    (kvm_caps.supported_perf_cap & PMU_CAP_LBR_FMT))
 				return 1;
-			if (!cpuid_model_is_consistent(vcpu))
+			if (!intel_pmu_lbr_is_compatible(vcpu))
 				return 1;
 		}
 		if (data & PERF_CAP_PEBS_FORMAT) {

arch/x86/kvm/vmx/vmx.h +1 ~~-31~~

index 2acdc54bc34b..1d15c3c2751b 100644
@@ -11,6 +11,7 @@
 #include "capabilities.h"
 #include "../kvm_cache_regs.h"
 #include "posted_intr.h"
+#include "pmu_intel.h"
 #include "vmcs.h"
 #include "vmx_ops.h"
 #include "../cpuid.h"

@@ -105,22 +106,6 @@ static inline bool intel_pmu_has_perf_global_ctrl(struct kvm_pmu *pmu)
 	return pmu->version > 1;
 }
 
-struct lbr_desc {
-	/* Basic info about guest LBR records. */
-	struct x86_pmu_lbr records;
-
-	/*
-	 * Emulate LBR feature via passthrough LBR registers when the
-	 * per-vcpu guest LBR event is scheduled on the current pcpu.
-	 *
-	 * The records may be inaccurate if the host reclaims the LBR.
-	 */
-	struct perf_event *event;
-
-	/* True if LBRs are marked as not intercepted in the MSR bitmap */
-	bool msr_passthrough;
-};
-
 /*
  * The nested_vmx structure is part of vcpu_vmx, and holds information we need
  * for correct emulation of VMX (i.e., nested VMX) on this vcpu.

@@ -650,21 +635,6 @@ static __always_inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu)
 	return container_of(vcpu, struct vcpu_vmx, vcpu);
 }
 
-static inline struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu)
-{
-	return &to_vmx(vcpu)->lbr_desc;
-}
-
-static inline struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu)
-{
-	return &vcpu_to_lbr_desc(vcpu)->records;
-}
-
-static inline bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu)
-{
-	return !!vcpu_to_lbr_records(vcpu)->nr;
-}
-
 void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu);
 int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu);
 void vmx_passthrough_lbr_msrs(struct kvm_vcpu *vcpu);

Signature · 1 line

2.25.1

Zhi Wang

2023-04-02 08:50 UTC #77 lore raw

Re: [PATCH v13 021/113] KVM: TDX: Make pmu_intel.c ignore guest TD case

Hi Like:

Would you mind to take a look on this patch? It would be nice to have a r-b also from you. :)

Quoted 239 lines · On Sun, 12 Mar 2023 10:55:45 -0700 [email protected] wrote:

From: Isaku Yamahata <[email protected]>

---
 arch/x86/kvm/vmx/pmu_intel.c | 46 +++++++++++++++++++++++++++++++++++-
 arch/x86/kvm/vmx/pmu_intel.h | 28 ++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.h       |  8 ++++++-
 arch/x86/kvm/vmx/vmx.c       |  2 +-
 arch/x86/kvm/vmx/vmx.h       | 32 +------------------------
 5 files changed, 82 insertions(+), 34 deletions(-)
 create mode 100644 arch/x86/kvm/vmx/pmu_intel.h

arch/x86/kvm/vmx/pmu_intel.c +45 -1

index e8a3be0b9df9..df1f4ddfa72d 100644
@@ -19,6 +19,7 @@
 #include "lapic.h"
 #include "nested.h"
 #include "pmu.h"
+#include "tdx.h"
 
 #define MSR_PMC_FULL_WIDTH_BIT      (MSR_IA32_PMC0 - MSR_IA32_PERFCTR0)

@@ -40,6 +41,26 @@ static struct {
 /* mapping between fixed pmc index and intel_arch_events array */
 static int fixed_pmc_events[] = {1, 0, 7};
 
+struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_INTEL_TDX_HOST
+	if (is_td_vcpu(vcpu))
+		return &to_tdx(vcpu)->lbr_desc;
+#endif
+
+	return &to_vmx(vcpu)->lbr_desc;
+}
+
+struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_INTEL_TDX_HOST
+	if (is_td_vcpu(vcpu))
+		return &to_tdx(vcpu)->lbr_desc.records;
+#endif
+
+	return &to_vmx(vcpu)->lbr_desc.records;
+}
+
 static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 {
 	struct kvm_pmc *pmc;

@@ -172,6 +193,23 @@ static inline struct kvm_pmc *get_fw_gp_pmc(struct kvm_pmu *pmu, u32 msr)
 	return get_gp_pmc(pmu, msr, MSR_IA32_PMC0);
 }
 
+bool intel_pmu_lbr_is_compatible(struct kvm_vcpu *vcpu)
+{
+	if (is_td_vcpu(vcpu))
+		return false;
+	return cpuid_model_is_consistent(vcpu);
+}
+
+bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu)
+{
+	struct x86_pmu_lbr *lbr = vcpu_to_lbr_records(vcpu);
+
+	if (is_td_vcpu(vcpu))
+		return false;
+
+	return lbr->nr && (vcpu_get_perf_capabilities(vcpu) & PMU_CAP_LBR_FMT);
+}
+
 static bool intel_pmu_is_valid_lbr_msr(struct kvm_vcpu *vcpu, u32 index)
 {
 	struct x86_pmu_lbr *records = vcpu_to_lbr_records(vcpu);

@@ -282,6 +320,9 @@ int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu)
 					PERF_SAMPLE_BRANCH_USER,
 	};
 
+	if (WARN_ON_ONCE(is_td_vcpu(vcpu)))
+		return 0;
+
 	if (unlikely(lbr_desc->event)) {
 		__set_bit(INTEL_PMC_IDX_FIXED_VLBR, pmu->pmc_in_use);
 		return 0;

@@ -591,7 +632,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 		INTEL_PMC_MAX_GENERIC, pmu->nr_arch_fixed_counters);
 
 	perf_capabilities = vcpu_get_perf_capabilities(vcpu);
-	if (cpuid_model_is_consistent(vcpu) &&
+	if (intel_pmu_lbr_is_compatible(vcpu) &&
 	    (perf_capabilities & PMU_CAP_LBR_FMT))
 		x86_perf_get_lbr(&lbr_desc->records);
 	else

@@ -647,6 +688,9 @@ static void intel_pmu_reset(struct kvm_vcpu *vcpu)
 	struct kvm_pmc *pmc = NULL;
 	int i;
 
+	if (is_td_vcpu(vcpu))
+		return;
+
 	for (i = 0; i < KVM_INTEL_PMC_MAX_GENERIC; i++) {
 		pmc = &pmu->gp_counters[i];

arch/x86/kvm/vmx/pmu_intel.h new file +28 -0

new file mode 100644
index 000000000000..66bba47c1269
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __KVM_X86_VMX_PMU_INTEL_H
+#define  __KVM_X86_VMX_PMU_INTEL_H
+
+struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu);
+struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu);
+
+bool intel_pmu_lbr_is_compatible(struct kvm_vcpu *vcpu);
+bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu);
+int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu);
+
+struct lbr_desc {
+	/* Basic info about guest LBR records. */
+	struct x86_pmu_lbr records;
+
+	/*
+	 * Emulate LBR feature via passthrough LBR registers when the
+	 * per-vcpu guest LBR event is scheduled on the current pcpu.
+	 *
+	 * The records may be inaccurate if the host reclaims the LBR.
+	 */
+	struct perf_event *event;
+
+	/* True if LBRs are marked as not intercepted in the MSR bitmap */
+	bool msr_passthrough;
+};
+
+#endif /* __KVM_X86_VMX_PMU_INTEL_H */

arch/x86/kvm/vmx/tdx.h +7 -1

index 1e00e75b1c5e..5728820fed5e 100644
@@ -4,6 +4,7 @@
 
 #ifdef CONFIG_INTEL_TDX_HOST
 
+#include "pmu_intel.h"
 #include "tdx_ops.h"
 
 struct kvm_tdx {

@@ -21,7 +22,12 @@ struct kvm_tdx {
 
 struct vcpu_tdx {
 	struct kvm_vcpu	vcpu;
-	/* TDX specific members follow. */
+
+	/*
+	 * Dummy to make pmu_intel not corrupt memory.
+	 * TODO: Support PMU for TDX.  Future work.
+	 */
+	struct lbr_desc lbr_desc;
 };
 
 static inline bool is_td(struct kvm *kvm)

arch/x86/kvm/vmx/vmx.c +1 -1

index d23830d92f61..f9e9fd7fde2c 100644
@@ -2432,7 +2432,7 @@ int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			if ((data & PMU_CAP_LBR_FMT) !=
 			    (kvm_caps.supported_perf_cap & PMU_CAP_LBR_FMT))
 				return 1;
-			if (!cpuid_model_is_consistent(vcpu))
+			if (!intel_pmu_lbr_is_compatible(vcpu))
 				return 1;
 		}
 		if (data & PERF_CAP_PEBS_FORMAT) {

arch/x86/kvm/vmx/vmx.h +1 ~~-31~~

index 2acdc54bc34b..1d15c3c2751b 100644
@@ -11,6 +11,7 @@
 #include "capabilities.h"
 #include "../kvm_cache_regs.h"
 #include "posted_intr.h"
+#include "pmu_intel.h"
 #include "vmcs.h"
 #include "vmx_ops.h"
 #include "../cpuid.h"

@@ -105,22 +106,6 @@ static inline bool intel_pmu_has_perf_global_ctrl(struct kvm_pmu *pmu)
 	return pmu->version > 1;
 }
 
-struct lbr_desc {
-	/* Basic info about guest LBR records. */
-	struct x86_pmu_lbr records;
-
-	/*
-	 * Emulate LBR feature via passthrough LBR registers when the
-	 * per-vcpu guest LBR event is scheduled on the current pcpu.
-	 *
-	 * The records may be inaccurate if the host reclaims the LBR.
-	 */
-	struct perf_event *event;
-
-	/* True if LBRs are marked as not intercepted in the MSR bitmap */
-	bool msr_passthrough;
-};
-
 /*
  * The nested_vmx structure is part of vcpu_vmx, and holds information we need
  * for correct emulation of VMX (i.e., nested VMX) on this vcpu.

@@ -650,21 +635,6 @@ static __always_inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu)
 	return container_of(vcpu, struct vcpu_vmx, vcpu);
 }
 
-static inline struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu)
-{
-	return &to_vmx(vcpu)->lbr_desc;
-}
-
-static inline struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu)
-{
-	return &vcpu_to_lbr_desc(vcpu)->records;
-}
-
-static inline bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu)
-{
-	return !!vcpu_to_lbr_records(vcpu)->nr;
-}
-
 void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu);
 int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu);
 void vmx_passthrough_lbr_msrs(struct kvm_vcpu *vcpu);

Like Xu

2023-04-19 08:21 UTC #78 lore raw

Re: [PATCH v13 021/113] KVM: TDX: Make pmu_intel.c ignore guest TD case

Quoted 12 lines · Zhi Wang

Hi Like:
Would you mind to take a look on this patch? It would be nice to have a r-b also from you. :)
Quoted 4 lines · On Sun, 12 Mar 2023 10:55:45 -0700 [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Because TDX KVM doesn't support PMU yet (it's future work of TDX KVM support as another patch series) and pmu_intel.c touches vmx specific

It would be nice to have pmu support for tdx-guest from the very beginning. If you guys are more open on the tdx development model, I'd like to help on those features.

Quoted 2 lines

Quoted 2 lines
structure in vcpu initialization, as workaround add dummy structure to struct vcpu_tdx and pmu_intel.c can ignore TDX case.

If the target is not to provide a workaround, how about other variants:
- struct lbr_desc lbr_desc;
- pebs ds_buffer;
?

We also need tdx selftest to verify the unavailability of these features.
Also, it would be great to have TDX's "System Profiling Mode" featue back in the specification.

Quoted 235 lines

Quoted 233 lines

---
  arch/x86/kvm/vmx/pmu_intel.c | 46 +++++++++++++++++++++++++++++++++++-
  arch/x86/kvm/vmx/pmu_intel.h | 28 ++++++++++++++++++++++
  arch/x86/kvm/vmx/tdx.h       |  8 ++++++-
  arch/x86/kvm/vmx/vmx.c       |  2 +-
  arch/x86/kvm/vmx/vmx.h       | 32 +------------------------
  5 files changed, 82 insertions(+), 34 deletions(-)

create mode 100644 arch/x86/kvm/vmx/pmu_intel.h

arch/x86/kvm/vmx/pmu_intel.c +45 -1

index e8a3be0b9df9..df1f4ddfa72d 100644
@@ -19,6 +19,7 @@
  #include "lapic.h"
  #include "nested.h"
  #include "pmu.h"
+#include "tdx.h"
 
  #define MSR_PMC_FULL_WIDTH_BIT      (MSR_IA32_PMC0 - MSR_IA32_PERFCTR0)

@@ -40,6 +41,26 @@ static struct {
  /* mapping between fixed pmc index and intel_arch_events array */
  static int fixed_pmc_events[] = {1, 0, 7};
 
+struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_INTEL_TDX_HOST
+	if (is_td_vcpu(vcpu))
+		return &to_tdx(vcpu)->lbr_desc;
+#endif
+
+	return &to_vmx(vcpu)->lbr_desc;
+}
+
+struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_INTEL_TDX_HOST
+	if (is_td_vcpu(vcpu))
+		return &to_tdx(vcpu)->lbr_desc.records;
+#endif
+
+	return &to_vmx(vcpu)->lbr_desc.records;
+}
+
  static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
  {
  	struct kvm_pmc *pmc;

@@ -172,6 +193,23 @@ static inline struct kvm_pmc *get_fw_gp_pmc(struct kvm_pmu *pmu, u32 msr)
  	return get_gp_pmc(pmu, msr, MSR_IA32_PMC0);
  }
 
+bool intel_pmu_lbr_is_compatible(struct kvm_vcpu *vcpu)
+{
+	if (is_td_vcpu(vcpu))
+		return false;
+	return cpuid_model_is_consistent(vcpu);
+}
+
+bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu)
+{
+	struct x86_pmu_lbr *lbr = vcpu_to_lbr_records(vcpu);
+
+	if (is_td_vcpu(vcpu))
+		return false;
+
+	return lbr->nr && (vcpu_get_perf_capabilities(vcpu) & PMU_CAP_LBR_FMT);
+}
+
  static bool intel_pmu_is_valid_lbr_msr(struct kvm_vcpu *vcpu, u32 index)
  {
  	struct x86_pmu_lbr *records = vcpu_to_lbr_records(vcpu);

@@ -282,6 +320,9 @@ int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu)
  					PERF_SAMPLE_BRANCH_USER,
  	};
 
+	if (WARN_ON_ONCE(is_td_vcpu(vcpu)))
+		return 0;
+
  	if (unlikely(lbr_desc->event)) {
  		__set_bit(INTEL_PMC_IDX_FIXED_VLBR, pmu->pmc_in_use);
  		return 0;

@@ -591,7 +632,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
  		INTEL_PMC_MAX_GENERIC, pmu->nr_arch_fixed_counters);
 
  	perf_capabilities = vcpu_get_perf_capabilities(vcpu);
-	if (cpuid_model_is_consistent(vcpu) &&
+	if (intel_pmu_lbr_is_compatible(vcpu) &&
  	    (perf_capabilities & PMU_CAP_LBR_FMT))
  		x86_perf_get_lbr(&lbr_desc->records);
  	else

@@ -647,6 +688,9 @@ static void intel_pmu_reset(struct kvm_vcpu *vcpu)
  	struct kvm_pmc *pmc = NULL;
  	int i;
 
+	if (is_td_vcpu(vcpu))
+		return;
+
  	for (i = 0; i < KVM_INTEL_PMC_MAX_GENERIC; i++) {
  		pmc = &pmu->gp_counters[i];

arch/x86/kvm/vmx/pmu_intel.h new file +28 -0

new file mode 100644
index 000000000000..66bba47c1269
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __KVM_X86_VMX_PMU_INTEL_H
+#define  __KVM_X86_VMX_PMU_INTEL_H
+
+struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu);
+struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu);
+
+bool intel_pmu_lbr_is_compatible(struct kvm_vcpu *vcpu);
+bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu);
+int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu);
+
+struct lbr_desc {
+	/* Basic info about guest LBR records. */
+	struct x86_pmu_lbr records;
+
+	/*
+	 * Emulate LBR feature via passthrough LBR registers when the
+	 * per-vcpu guest LBR event is scheduled on the current pcpu.
+	 *
+	 * The records may be inaccurate if the host reclaims the LBR.
+	 */
+	struct perf_event *event;
+
+	/* True if LBRs are marked as not intercepted in the MSR bitmap */
+	bool msr_passthrough;
+};
+
+#endif /* __KVM_X86_VMX_PMU_INTEL_H */

arch/x86/kvm/vmx/tdx.h +7 -1

index 1e00e75b1c5e..5728820fed5e 100644
@@ -4,6 +4,7 @@
 
  #ifdef CONFIG_INTEL_TDX_HOST
 
+#include "pmu_intel.h"
  #include "tdx_ops.h"
 
  struct kvm_tdx {

@@ -21,7 +22,12 @@ struct kvm_tdx {
 
  struct vcpu_tdx {
  	struct kvm_vcpu	vcpu;
-	/* TDX specific members follow. */
+
+	/*
+	 * Dummy to make pmu_intel not corrupt memory.
+	 * TODO: Support PMU for TDX.  Future work.
+	 */
+	struct lbr_desc lbr_desc;
  };
 
  static inline bool is_td(struct kvm *kvm)

arch/x86/kvm/vmx/vmx.c +1 -1

index d23830d92f61..f9e9fd7fde2c 100644
@@ -2432,7 +2432,7 @@ int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
  			if ((data & PMU_CAP_LBR_FMT) !=
  			    (kvm_caps.supported_perf_cap & PMU_CAP_LBR_FMT))
  				return 1;
-			if (!cpuid_model_is_consistent(vcpu))
+			if (!intel_pmu_lbr_is_compatible(vcpu))
  				return 1;
  		}
  		if (data & PERF_CAP_PEBS_FORMAT) {

arch/x86/kvm/vmx/vmx.h +1 ~~-31~~

index 2acdc54bc34b..1d15c3c2751b 100644
@@ -11,6 +11,7 @@
  #include "capabilities.h"
  #include "../kvm_cache_regs.h"
  #include "posted_intr.h"
+#include "pmu_intel.h"
  #include "vmcs.h"
  #include "vmx_ops.h"
  #include "../cpuid.h"

@@ -105,22 +106,6 @@ static inline bool intel_pmu_has_perf_global_ctrl(struct kvm_pmu *pmu)
  	return pmu->version > 1;
  }
 
-struct lbr_desc {
-	/* Basic info about guest LBR records. */
-	struct x86_pmu_lbr records;
-
-	/*
-	 * Emulate LBR feature via passthrough LBR registers when the
-	 * per-vcpu guest LBR event is scheduled on the current pcpu.
-	 *
-	 * The records may be inaccurate if the host reclaims the LBR.
-	 */
-	struct perf_event *event;
-
-	/* True if LBRs are marked as not intercepted in the MSR bitmap */
-	bool msr_passthrough;
-};
-
  /*
   * The nested_vmx structure is part of vcpu_vmx, and holds information we need
   * for correct emulation of VMX (i.e., nested VMX) on this vcpu.

@@ -650,21 +635,6 @@ static __always_inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu)
  	return container_of(vcpu, struct vcpu_vmx, vcpu);
  }
 
-static inline struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu)
-{
-	return &to_vmx(vcpu)->lbr_desc;
-}
-
-static inline struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu)
-{
-	return &vcpu_to_lbr_desc(vcpu)->records;
-}
-
-static inline bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu)
-{
-	return !!vcpu_to_lbr_records(vcpu)->nr;
-}
-
  void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu);
  int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu);
  void vmx_passthrough_lbr_msrs(struct kvm_vcpu *vcpu);

Isaku Yamahata

2023-05-28 08:26 UTC #79 lore raw

Re: [PATCH v13 021/113] KVM: TDX: Make pmu_intel.c ignore guest TD case

Quoted 9 lines · Like Xu

Quoted 8 lines · Zhi Wang
Hi Like:
Would you mind to take a look on this patch? It would be nice to have a r-b also from you. :)
On Sun, 12 Mar 2023 10:55:45 -0700
[email protected] wrote:

Quoted 6 lines

Quoted 4 lines
Quoted 4 lines
From: Isaku Yamahata <[email protected]>
Because TDX KVM doesn't support PMU yet (it's future work of TDX KVM support as another patch series) and pmu_intel.c touches vmx specific
It would be nice to have pmu support for tdx-guest from the very beginning.

It's supported in the public github repo.
https://github.com/intel/tdx/tree/kvm-upstream-workaround
As this patch series has 100+ patches, I don't want to bloat this patch more.

Quoted 2 lines

If you guys are more open on the tdx development model, I'd like to help on those features.

This mainling list is the place.

Quoted 11 lines

Quoted 2 lines
Quoted 2 lines
structure in vcpu initialization, as workaround add dummy structure to struct vcpu_tdx and pmu_intel.c can ignore TDX case.
If the target is not to provide a workaround, how about other variants:
- struct lbr_desc lbr_desc;
- pebs ds_buffer;
?
We also need tdx selftest to verify the unavailability of these features. Also, it would be great to have TDX's "System Profiling Mode" featue back in the specification.

I don't think it's productive. Once merging this patch series, we can move on to TDX PMU support (or whatever still missing feature) as second (or later) step.

Signature · 1 line

Isaku Yamahata <[email protected]>

Like Xu

2023-05-29 14:19 UTC #80 lore raw

Re: [PATCH v13 021/113] KVM: TDX: Make pmu_intel.c ignore guest TD case

Quoted 14 lines · Isaku Yamahata

Quoted 11 lines · Like Xu
Quoted 10 lines · Zhi Wang
Hi Like:
Would you mind to take a look on this patch? It would be nice to have a r-b also from you. :)
Quoted 2 lines · On Sun, 12 Mar 2023 10:55:45 -0700 [email protected] wrote:
From: Isaku Yamahata <[email protected]>

Quoted 8 lines

Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Because TDX KVM doesn't support PMU yet (it's future work of TDX KVM support as another patch series) and pmu_intel.c touches vmx specific
It would be nice to have pmu support for tdx-guest from the very beginning.
It's supported in the public github repo.
https://github.com/intel/tdx/tree/kvm-upstream-workaround
As this patch series has 100+ patches, I don't want to bloat this patch more.

I presume we are talking about 873e2391e729...63761adbf5aa for TD pmu:

A quick glance brought me at least these comments:

(1) how does intel_pmu_save/restore() handle the enabled host LBR/PEBS ?
(2) guest PMI injection may be malicious and could the current guest pmu driver handle it ?
(3) how do we handle the case when host counters can be enabled before TDENTER for debuggable TD and support the case like "perf-kvm for both guest and host" ?

My point is actually, changes to perf/core should be CC to the perf reviewers as early as possible to prevent key player from killing the direction.

Quoted 6 lines

Quoted 4 lines
If you guys are more open on the tdx development model, I'd like to help on those features.
This mainling list is the place.

Yeah, plus the PUCK and LPC KVM micro-conference.

Quoted 5 lines

Quoted 5 lines
Quoted 4 lines
Quoted 4 lines
structure in vcpu initialization, as workaround add dummy structure to struct vcpu_tdx and pmu_intel.c can ignore TDX case.

Quoted 8 lines

Quoted 8 lines
If the target is not to provide a workaround, how about other variants:
- struct lbr_desc lbr_desc;
- pebs ds_buffer;
?
We also need tdx selftest to verify the unavailability of these features. Also, it would be great to have TDX's "System Profiling Mode" featue back in the specification.

Detailed TD (plus debuggable) PMU selftest would clearly speed up the review process.

Quoted 4 lines

I don't think it's productive. Once merging this patch series, we can move on to TDX PMU support (or whatever still missing feature) as second (or later) step.

I totally agree that TD PMU (plus debuggable) support should not be the first part of the code to be merged in, but the related discussion should be kicked off on day 0. As a scenario for users using Intel TDX, it's expected to support "System Profiling Mode", which was first introduced in 344425-001US but disappeared since version 003.

Isaku Yamahata

2023-05-30 23:00 UTC #81 lore raw

Re: [PATCH v13 021/113] KVM: TDX: Make pmu_intel.c ignore guest TD case

Quoted 24 lines · Like Xu

Quoted 14 lines · Isaku Yamahata
Quoted 11 lines · Like Xu
Quoted 10 lines · Zhi Wang
Hi Like:
Would you mind to take a look on this patch? It would be nice to have a r-b also from you. :)
Quoted 2 lines · On Sun, 12 Mar 2023 10:55:45 -0700 [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Because TDX KVM doesn't support PMU yet (it's future work of TDX KVM support as another patch series) and pmu_intel.c touches vmx specific
It would be nice to have pmu support for tdx-guest from the very beginning.
It's supported in the public github repo.
https://github.com/intel/tdx/tree/kvm-upstream-workaround
As this patch series has 100+ patches, I don't want to bloat this patch more.

Quoted 5 lines

I presume we are talking about 873e2391e729...63761adbf5aa for TD pmu:
A quick glance brought me at least these comments:
(1) how does intel_pmu_save/restore() handle the enabled host LBR/PEBS ?

It's not handled yet. We need to save/restore those MSRs.

Quoted 2 lines

(2) guest PMI injection may be malicious and could the current guest pmu driver handle it ?

This isn't specific to PMI. Malicious VMM can inject any interrupt to the guest at any time. Guest should be prepared for it.

Quoted 2 lines

(3) how do we handle the case when host counters can be enabled before TDENTER for debuggable TD and support the case like "perf-kvm for both guest and host" ?

On TDEXIT, those are disabled. VMM has to restore MSRs and enable it again. There is a window where events can be missed.

Quoted 2 lines

My point is actually, changes to perf/core should be CC to the perf reviewers as early as possible to prevent key player from killing the direction.

Sure, agreed.

Quoted 8 lines

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
structure in vcpu initialization, as workaround add dummy structure to struct vcpu_tdx and pmu_intel.c can ignore TDX case.
If the target is not to provide a workaround, how about other variants:
- struct lbr_desc lbr_desc;
- pebs ds_buffer;
?

Quoted 6 lines

Quoted 3 lines
Quoted 3 lines
We also need tdx selftest to verify the unavailability of these features. Also, it would be great to have TDX's "System Profiling Mode" featue back in the specification.
Detailed TD (plus debuggable) PMU selftest would clearly speed up the review process.

The existing KVM PMU selftest can be utilized. Or do you have something else in mind?

Signature · 1 line

Isaku Yamahata <[email protected]>

[email protected]

author

2023-03-12 18:00 UTC #82 lore raw

PATCH v13 22/113 KVM: TDX: Refuse to unplug the last cpu on the package

From: Isaku Yamahata <[email protected]>

In order to reclaim TDX HKID, (i.e. when deleting guest TD), needs to call TDH.PHYMEM.PAGE.WBINVD on all packages. If we have active TDX HKID, refuse to offline the last online cpu to guarantee at least one CPU online per package. Add arch callback for cpu offline.
Because TDX doesn't support suspend by the TDX 1.0 spec, this also refuses suspend if TDs are running. If no TD is running, suspend is allowed.

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  1 +
 arch/x86/kvm/vmx/main.c            |  1 +
 arch/x86/kvm/vmx/tdx.c             | 43 +++++++++++++++++++++++++++++-
 arch/x86/kvm/vmx/x86_ops.h         |  2 ++
 arch/x86/kvm/x86.c                 |  5 ++++
 include/linux/kvm_host.h           |  1 +
 virt/kvm/kvm_main.c                | 12 +++++++--
 8 files changed, 63 insertions(+), 3 deletions(-)

arch/x86/include/asm/kvm-x86-ops.h +1 -0

index c30d2d2ad686..f763981b7dbc 100644
@@ -18,6 +18,7 @@ KVM_X86_OP(check_processor_compatibility)
 KVM_X86_OP(hardware_enable)
 KVM_X86_OP(hardware_disable)
 KVM_X86_OP(hardware_unsetup)
+KVM_X86_OP_OPTIONAL_RET0(offline_cpu)
 KVM_X86_OP(has_emulated_msr)
 KVM_X86_OP(vcpu_after_set_cpuid)
 KVM_X86_OP(is_vm_type_supported)

arch/x86/include/asm/kvm_host.h +1 -0

index 15f7c0d45082..f30543caddbf 100644
@@ -1558,6 +1558,7 @@ struct kvm_x86_ops {
 	int (*hardware_enable)(void);
 	void (*hardware_disable)(void);
 	void (*hardware_unsetup)(void);
+	int (*offline_cpu)(void);
 	bool (*has_emulated_msr)(struct kvm *kvm, u32 index);
 	void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu);

arch/x86/kvm/vmx/main.c +1 -0

index 08ed6fc8cc3d..e57b36902313 100644
@@ -122,6 +122,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.check_processor_compatibility = vmx_check_processor_compat,
 
 	.hardware_unsetup = vt_hardware_unsetup,
+	.offline_cpu = tdx_offline_cpu,
 
 	.hardware_enable = vt_hardware_enable,
 	.hardware_disable = vmx_hardware_disable,

arch/x86/kvm/vmx/tdx.c +42 -1

64 lines not on this page · raw message

arch/x86/kvm/vmx/x86_ops.h +2 -0

index 8f87eeeab97c..544f99141f8f 100644
@@ -143,6 +143,7 @@ void tdx_hardware_unsetup(void);
 int tdx_hardware_enable(void);
 bool tdx_is_vm_type_supported(unsigned long type);
 int tdx_dev_ioctl(void __user *argp);
+int tdx_offline_cpu(void);
 
 int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);
 int tdx_vm_init(struct kvm *kvm);

@@ -155,6 +156,7 @@ static inline void tdx_hardware_unsetup(void) {}
 static inline int tdx_hardware_enable(void) { return -EOPNOTSUPP; }
 static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; }
 static inline int tdx_dev_ioctl(void __user *argp) { return -EOPNOTSUPP; };
+static inline int tdx_offline_cpu(void) { return 0; }
 
 static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { return -EINVAL; };
 static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; }

arch/x86/kvm/x86.c +5 -0

index 89fd8074c20c..a1d5d920302b 100644
@@ -12161,6 +12161,11 @@ void kvm_arch_hardware_disable(void)
 	drop_user_return_notifiers();
 }
 
+int kvm_arch_offline_cpu(unsigned int cpu)
+{
+	return static_call(kvm_x86_offline_cpu)();
+}
+
 bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu)
 {
 	return vcpu->kvm->arch.bsp_vcpu_id == vcpu->vcpu_id;

include/linux/kvm_host.h +1 -0

index 4ae46e6f93af..115cdca6384f 100644
@@ -1478,6 +1478,7 @@ static inline void kvm_create_vcpu_debugfs(struct kvm_vcpu *vcpu) {}
 int kvm_arch_hardware_enable(void);
 void kvm_arch_hardware_disable(void);
 #endif
+int kvm_arch_offline_cpu(unsigned int cpu);
 int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu);
 bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu);
 int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu);

virt/kvm/kvm_main.c +10 -2

index 3f1a55834440..f8495e27d210 100644
@@ -5509,13 +5509,21 @@ static void hardware_disable_nolock(void *junk)
 	__this_cpu_write(hardware_enabled, false);
 }
 
+__weak int kvm_arch_offline_cpu(unsigned int cpu)
+{
+	return 0;
+}
+
 static int kvm_offline_cpu(unsigned int cpu)
 {
+	int r = 0;
+
 	mutex_lock(&kvm_lock);
-	if (kvm_usage_count)
+	r = kvm_arch_offline_cpu(cpu);
+	if (!r && kvm_usage_count)
 		hardware_disable_nolock(NULL);
 	mutex_unlock(&kvm_lock);
-	return 0;
+	return r;
 }
 
 static void hardware_disable_all_nolock(void)

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:00 UTC #83 lore raw

PATCH v13 23/113 The start of TDX KVM patch series: TD vcpu creation/destruction

From: Isaku Yamahata <[email protected]>

This empty commit is to mark the start of patch series of TD vcpu creation/destruction.

---
 Documentation/virt/kvm/intel-tdx-layer-status.rst | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Documentation/virt/kvm/intel-tdx-layer-status.rst +3 -3

index 098150da6ea2..25082e9c0b20 100644
@@ -9,7 +9,7 @@ Layer status
 What qemu can do
 ----------------
 - TDX VM TYPE is exposed to Qemu.
-- Qemu can try to create VM of TDX VM type and then fails.
+- Qemu can create/destroy guest of TDX vm type.
 
 Patch Layer status
 ------------------

@@ -17,8 +17,8 @@ Patch Layer status
 
 * TDX, VMX coexistence:                 Applied
 * TDX architectural definitions:        Applied
-* TD VM creation/destruction:           Applying
-* TD vcpu creation/destruction:         Not yet
+* TD VM creation/destruction:           Applied
+* TD vcpu creation/destruction:         Applying
 * TDX EPT violation:                    Not yet
 * TD finalization:                      Not yet
 * TD vcpu enter/exit:                   Not yet

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:00 UTC #84 lore raw

PATCH v13 24/113 KVM: TDX: allocate/free TDX vcpu structure

From: Isaku Yamahata <[email protected]>

The next step of TDX guest creation is to create vcpu. Allocate TDX vcpu structures, initialize it that doesn't require TDX SEAMCALL. TDX specific vcpu initialization will be implemented as independent KVM_TDX_INIT_VCPU so that when error occurs it's easy to determine which component has the issue, KVM or TDX.

---
Changes v11 -> v12:
- add more comments in tdx_vcpu_reset().
- use KVM_BUG_ON()

Changes v10 -> v11:
- NULL check of kvmalloc_array() in tdx_vcpu_reset. Move it to tdx_vcpu_create()

---
 arch/x86/kvm/vmx/main.c    | 44 ++++++++++++++++++++++++++++++----
 arch/x86/kvm/vmx/tdx.c     | 49 ++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h | 10 ++++++++
 arch/x86/kvm/x86.c         |  2 ++
 4 files changed, 101 insertions(+), 4 deletions(-)

arch/x86/kvm/vmx/main.c +40 -4

index e57b36902313..e9fd4e80b67d 100644
@@ -97,6 +97,42 @@ static void vt_vm_free(struct kvm *kvm)
 		tdx_vm_free(kvm);
 }
 
+static int vt_vcpu_precreate(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		return 0;
+
+	return vmx_vcpu_precreate(kvm);
+}
+
+static int vt_vcpu_create(struct kvm_vcpu *vcpu)
+{
+	if (is_td_vcpu(vcpu))
+		return tdx_vcpu_create(vcpu);
+
+	return vmx_vcpu_create(vcpu);
+}
+
+static void vt_vcpu_free(struct kvm_vcpu *vcpu)
+{
+	if (is_td_vcpu(vcpu)) {
+		tdx_vcpu_free(vcpu);
+		return;
+	}
+
+	vmx_vcpu_free(vcpu);
+}
+
+static void vt_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
+{
+	if (is_td_vcpu(vcpu)) {
+		tdx_vcpu_reset(vcpu, init_event);
+		return;
+	}
+
+	vmx_vcpu_reset(vcpu, init_event);
+}
+
 static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
 	if (!is_td(kvm))

@@ -137,10 +173,10 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.vm_destroy = vt_vm_destroy,
 	.vm_free = vt_vm_free,
 
-	.vcpu_precreate = vmx_vcpu_precreate,
-	.vcpu_create = vmx_vcpu_create,
-	.vcpu_free = vmx_vcpu_free,
-	.vcpu_reset = vmx_vcpu_reset,
+	.vcpu_precreate = vt_vcpu_precreate,
+	.vcpu_create = vt_vcpu_create,
+	.vcpu_free = vt_vcpu_free,
+	.vcpu_reset = vt_vcpu_reset,
 
 	.prepare_switch_to_guest = vmx_prepare_switch_to_guest,
 	.vcpu_load = vmx_vcpu_load,

arch/x86/kvm/vmx/tdx.c +49 -0

55 lines not on this page · raw message

arch/x86/kvm/vmx/x86_ops.h +10 -0

index 544f99141f8f..1669c95d8249 100644
@@ -149,7 +149,12 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);
 int tdx_vm_init(struct kvm *kvm);
 void tdx_mmu_release_hkid(struct kvm *kvm);
 void tdx_vm_free(struct kvm *kvm);
+
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
+
+int tdx_vcpu_create(struct kvm_vcpu *vcpu);
+void tdx_vcpu_free(struct kvm_vcpu *vcpu);
+void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
 static inline void tdx_hardware_unsetup(void) {}

@@ -163,7 +168,12 @@ static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; }
 static inline void tdx_mmu_release_hkid(struct kvm *kvm) {}
 static inline void tdx_flush_shadow_all_private(struct kvm *kvm) {}
 static inline void tdx_vm_free(struct kvm *kvm) {}
+
 static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
+
+static inline int tdx_vcpu_create(struct kvm_vcpu *vcpu) { return -EOPNOTSUPP; }
+static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {}
+static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {}
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

arch/x86/kvm/x86.c +2 -0

index a1d5d920302b..0ba15a9e126f 100644
@@ -496,6 +496,7 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	kvm_recalculate_apic_map(vcpu->kvm);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(kvm_set_apic_base);
 
 /*
  * Handle a fault on a hardware virtualization (VMX or SVM) instruction.

@@ -12170,6 +12171,7 @@ bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu)
 {
 	return vcpu->kvm->arch.bsp_vcpu_id == vcpu->vcpu_id;
 }
+EXPORT_SYMBOL_GPL(kvm_vcpu_is_reset_bsp);
 
 bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu)
 {

Signature · 1 line

2.25.1

Zhi Wang

2023-04-04 19:44 UTC #85 lore raw

Re: [PATCH v13 024/113] KVM: TDX: allocate/free TDX vcpu structure

Quoted 131 lines · On Sun, 12 Mar 2023 10:55:48 -0700 [email protected] wrote:

From: Isaku Yamahata <[email protected]>

---
Changes v11 -> v12:
- add more comments in tdx_vcpu_reset().
- use KVM_BUG_ON()

Changes v10 -> v11:
- NULL check of kvmalloc_array() in tdx_vcpu_reset. Move it to tdx_vcpu_create()

---
 arch/x86/kvm/vmx/main.c    | 44 ++++++++++++++++++++++++++++++----
 arch/x86/kvm/vmx/tdx.c     | 49 ++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h | 10 ++++++++
 arch/x86/kvm/x86.c         |  2 ++
 4 files changed, 101 insertions(+), 4 deletions(-)

arch/x86/kvm/vmx/main.c +40 -4

index e57b36902313..e9fd4e80b67d 100644
@@ -97,6 +97,42 @@ static void vt_vm_free(struct kvm *kvm)
 		tdx_vm_free(kvm);
 }
 
+static int vt_vcpu_precreate(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		return 0;
+
+	return vmx_vcpu_precreate(kvm);
+}
+
+static int vt_vcpu_create(struct kvm_vcpu *vcpu)
+{
+	if (is_td_vcpu(vcpu))
+		return tdx_vcpu_create(vcpu);
+
+	return vmx_vcpu_create(vcpu);
+}
+
+static void vt_vcpu_free(struct kvm_vcpu *vcpu)
+{
+	if (is_td_vcpu(vcpu)) {
+		tdx_vcpu_free(vcpu);
+		return;
+	}
+
+	vmx_vcpu_free(vcpu);
+}
+
+static void vt_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
+{
+	if (is_td_vcpu(vcpu)) {
+		tdx_vcpu_reset(vcpu, init_event);
+		return;
+	}
+
+	vmx_vcpu_reset(vcpu, init_event);
+}
+
 static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
 	if (!is_td(kvm))

@@ -137,10 +173,10 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.vm_destroy = vt_vm_destroy,
 	.vm_free = vt_vm_free,
 
-	.vcpu_precreate = vmx_vcpu_precreate,
-	.vcpu_create = vmx_vcpu_create,
-	.vcpu_free = vmx_vcpu_free,
-	.vcpu_reset = vmx_vcpu_reset,
+	.vcpu_precreate = vt_vcpu_precreate,
+	.vcpu_create = vt_vcpu_create,
+	.vcpu_free = vt_vcpu_free,
+	.vcpu_reset = vt_vcpu_reset,
 
 	.prepare_switch_to_guest = vmx_prepare_switch_to_guest,
 	.vcpu_load = vmx_vcpu_load,

arch/x86/kvm/vmx/tdx.c +37 -0

index aeec8bcf8921..4fec35f4f5b2 100644
@@ -321,6 +321,55 @@ int tdx_vm_init(struct kvm *kvm)
 	return 0;
 }
 
+int tdx_vcpu_create(struct kvm_vcpu *vcpu)
+{
+	/*
+	 * On cpu creation, cpuid entry is blank.  Forcibly enable
+	 * X2APIC feature to allow X2APIC.
+	 * Because vcpu_reset() can't return error, allocation is done here.
+	 */
+	WARN_ON_ONCE(vcpu->arch.cpuid_entries);
+	WARN_ON_ONCE(vcpu->arch.cpuid_nent);
+
+	/* TDX only supports x2APIC, which requires an in-kernel local APIC. */
+	if (!vcpu->arch.apic)
+		return -EINVAL;
+
+	fpstate_set_confidential(&vcpu->arch.guest_fpu);
+
+	vcpu->arch.efer = EFER_SCE | EFER_LME | EFER_LMA | EFER_NX;
+
+	vcpu->arch.cr0_guest_owned_bits = -1ul;
+	vcpu->arch.cr4_guest_owned_bits = -1ul;
+
+	vcpu->arch.tsc_offset = to_kvm_tdx(vcpu->kvm)->tsc_offset;
+	vcpu->arch.l1_tsc_offset = vcpu->arch.tsc_offset;
+	vcpu->arch.guest_state_protected =
+		!(to_kvm_tdx(vcpu->kvm)->attributes & TDX_TD_ATTRIBUTE_DEBUG);
+
+	return 0;
+}
+
+void tdx_vcpu_free(struct kvm_vcpu *vcpu)
+{
+	/* This is stub for now.  More logic will come. */
+}
+
+void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
+{
+

arch/x86/kvm/vmx/tdx.c int tdx_vm_init(struct kvm *kvm) · line 370

Quoted 10 lines

arch/x86/kvm/vmx/tdx.c +10 -0

+	/* Ignore INIT silently because TDX doesn't support INIT event. */
+	if (init_event)
+		return;
+
+	/* This is stub for now. More logic will come here. */
+
+	/*
+	 * Don't update mp_state to runnable because more initialization
+	 * is needed by TDX_VCPU_INIT.
+	 */

^
guess this comment belongs to the later patch.

Quoted 55 lines

arch/x86/kvm/vmx/tdx.c +2 -0

+}
+
 int tdx_dev_ioctl(void __user *argp)
 {
 	struct kvm_tdx_capabilities __user *user_caps;

arch/x86/kvm/vmx/x86_ops.h +10 -0

index 544f99141f8f..1669c95d8249 100644
@@ -149,7 +149,12 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);
 int tdx_vm_init(struct kvm *kvm);
 void tdx_mmu_release_hkid(struct kvm *kvm);
 void tdx_vm_free(struct kvm *kvm);
+
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
+
+int tdx_vcpu_create(struct kvm_vcpu *vcpu);
+void tdx_vcpu_free(struct kvm_vcpu *vcpu);
+void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
 static inline void tdx_hardware_unsetup(void) {}

@@ -163,7 +168,12 @@ static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; }
 static inline void tdx_mmu_release_hkid(struct kvm *kvm) {}
 static inline void tdx_flush_shadow_all_private(struct kvm *kvm) {}
 static inline void tdx_vm_free(struct kvm *kvm) {}
+
 static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
+
+static inline int tdx_vcpu_create(struct kvm_vcpu *vcpu) { return -EOPNOTSUPP; }
+static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {}
+static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {}
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

arch/x86/kvm/x86.c +2 -0

index a1d5d920302b..0ba15a9e126f 100644
@@ -496,6 +496,7 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	kvm_recalculate_apic_map(vcpu->kvm);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(kvm_set_apic_base);
 
 /*
  * Handle a fault on a hardware virtualization (VMX or SVM) instruction.

@@ -12170,6 +12171,7 @@ bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu)
 {
 	return vcpu->kvm->arch.bsp_vcpu_id == vcpu->vcpu_id;
 }
+EXPORT_SYMBOL_GPL(kvm_vcpu_is_reset_bsp);
 
 bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu)
 {

Isaku Yamahata

2023-04-05 19:07 UTC #86 lore raw

Re: [PATCH v13 024/113] KVM: TDX: allocate/free TDX vcpu structure

Quoted 7 lines · Zhi Wang

Quoted 7 lines

arch/x86/kvm/vmx/tdx.c +7 -0

+void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
+{
+
+	/* Ignore INIT silently because TDX doesn't support INIT event. */
+	if (init_event)
+		return;
+

Quoted 8 lines

arch/x86/kvm/vmx/tdx.c int tdx_vm_init(struct kvm *kvm) · line 370
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6 -0
+	/* This is stub for now. More logic will come here. */
+
+	/*
+	 * Don't update mp_state to runnable because more initialization
+	 * is needed by TDX_VCPU_INIT.
+	 */
^
guess this comment belongs to the later patch.

Ah, right. I'll move the comment to the next patch.

Signature · 1 line

Isaku Yamahata <[email protected]>

[email protected]

author

2023-03-12 18:00 UTC #87 lore raw

PATCH v13 25/113 KVM: TDX: Do TDX specific vcpu initialization

From: Isaku Yamahata <[email protected]>

TD guest vcpu needs TDX specific initialization before running. Repurpose KVM_MEMORY_ENCRYPT_OP to vcpu-scope, add a new sub-command KVM_TDX_INIT_VCPU, and implement the callback for it.

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/include/asm/kvm-x86-ops.h    |   1 +
 arch/x86/include/asm/kvm_host.h       |   1 +
 arch/x86/include/uapi/asm/kvm.h       |   1 +
 arch/x86/kvm/vmx/main.c               |   9 ++
 arch/x86/kvm/vmx/tdx.c                | 171 +++++++++++++++++++++++++-
 arch/x86/kvm/vmx/tdx.h                |   7 ++
 arch/x86/kvm/vmx/x86_ops.h            |   4 +
 arch/x86/kvm/x86.c                    |   6 +
 tools/arch/x86/include/uapi/asm/kvm.h |   1 +
 9 files changed, 200 insertions(+), 1 deletion(-)

arch/x86/include/asm/kvm-x86-ops.h +1 -0

index f763981b7dbc..d29e16098c30 100644
@@ -124,6 +124,7 @@ KVM_X86_OP(enable_smi_window)
 #endif
 KVM_X86_OP(dev_mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_ioctl)
+KVM_X86_OP_OPTIONAL(vcpu_mem_enc_ioctl)
 KVM_X86_OP_OPTIONAL(mem_enc_register_region)
 KVM_X86_OP_OPTIONAL(mem_enc_unregister_region)
 KVM_X86_OP_OPTIONAL(vm_copy_enc_context_from)

arch/x86/include/asm/kvm_host.h +1 -0

index f30543caddbf..42fa696a04ea 100644
@@ -1726,6 +1726,7 @@ struct kvm_x86_ops {
 
 	int (*dev_mem_enc_ioctl)(void __user *argp);
 	int (*mem_enc_ioctl)(struct kvm *kvm, void __user *argp);
+	int (*vcpu_mem_enc_ioctl)(struct kvm_vcpu *vcpu, void __user *argp);
 	int (*mem_enc_register_region)(struct kvm *kvm, struct kvm_enc_region *argp);
 	int (*mem_enc_unregister_region)(struct kvm *kvm, struct kvm_enc_region *argp);
 	int (*vm_copy_enc_context_from)(struct kvm *kvm, unsigned int source_fd);

arch/x86/include/uapi/asm/kvm.h +1 -0

index 68e8d544afe5..da7078cd7d7d 100644
@@ -566,6 +566,7 @@ struct kvm_pmu_event_filter {
 enum kvm_tdx_cmd_id {
 	KVM_TDX_CAPABILITIES = 0,
 	KVM_TDX_INIT_VM,
+	KVM_TDX_INIT_VCPU,
 
 	KVM_TDX_CMD_NR_MAX,
 };

arch/x86/kvm/vmx/main.c +9 -0

index e9fd4e80b67d..57240d6ab97a 100644
@@ -141,6 +141,14 @@ static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 	return tdx_vm_ioctl(kvm, argp);
 }
 
+static int vt_vcpu_mem_enc_ioctl(struct kvm_vcpu *vcpu, void __user *argp)
+{
+	if (!is_td_vcpu(vcpu))
+		return -EINVAL;
+
+	return tdx_vcpu_ioctl(vcpu, argp);
+}
+
 #define VMX_REQUIRED_APICV_INHIBITS		       \
 (						       \
        BIT(APICV_INHIBIT_REASON_DISABLE)|	       \

@@ -300,6 +308,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 
 	.dev_mem_enc_ioctl = tdx_dev_ioctl,
 	.mem_enc_ioctl = vt_mem_enc_ioctl,
+	.vcpu_mem_enc_ioctl = vt_vcpu_mem_enc_ioctl,
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {

arch/x86/kvm/vmx/tdx.c +170 -1

213 lines not on this page · raw message

arch/x86/kvm/vmx/tdx.h +7 -0

index 5728820fed5e..5fa4d3198873 100644
@@ -17,12 +17,19 @@ struct kvm_tdx {
 	u64 xfam;
 	int hkid;
 
+	bool finalized;
+
 	u64 tsc_offset;
 };
 
 struct vcpu_tdx {
 	struct kvm_vcpu	vcpu;
 
+	unsigned long tdvpr_pa;
+	unsigned long *tdvpx_pa;
+
+	bool initialized;
+
 	/*
 	 * Dummy to make pmu_intel not corrupt memory.
 	 * TODO: Support PMU for TDX.  Future work.

arch/x86/kvm/vmx/x86_ops.h +4 -0

index 1669c95d8249..92af5d2d5db7 100644
@@ -155,6 +155,8 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 int tdx_vcpu_create(struct kvm_vcpu *vcpu);
 void tdx_vcpu_free(struct kvm_vcpu *vcpu);
 void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event);
+
+int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -ENOSYS; }
 static inline void tdx_hardware_unsetup(void) {}

@@ -174,6 +176,8 @@ static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOP
 static inline int tdx_vcpu_create(struct kvm_vcpu *vcpu) { return -EOPNOTSUPP; }
 static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {}
 static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {}
+
+static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; }
 #endif
 
 #endif /* __KVM_X86_VMX_X86_OPS_H */

arch/x86/kvm/x86.c +6 -0

index 0ba15a9e126f..a0960b468c74 100644
@@ -6002,6 +6002,12 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 	case KVM_SET_DEVICE_ATTR:
 		r = kvm_vcpu_ioctl_device_attr(vcpu, ioctl, argp);
 		break;
+	case KVM_MEMORY_ENCRYPT_OP:
+		r = -ENOTTY;
+		if (!kvm_x86_ops.vcpu_mem_enc_ioctl)
+			goto out;
+		r = kvm_x86_ops.vcpu_mem_enc_ioctl(vcpu, argp);
+		break;
 	default:
 		r = -EINVAL;
 	}

tools/arch/x86/include/uapi/asm/kvm.h +1 -0

index c0f011384934..0f88e32d02a8 100644
@@ -566,6 +566,7 @@ struct kvm_pmu_event_filter {
 enum kvm_tdx_cmd_id {
 	KVM_TDX_CAPABILITIES = 0,
 	KVM_TDX_INIT_VM,
+	KVM_TDX_INIT_VCPU,
 
 	KVM_TDX_CMD_NR_MAX,
 };

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:01 UTC #88 lore raw

PATCH v13 26/113 The start of TDX KVM patch series: KVM MMU GPA shared bits

From: Isaku Yamahata <[email protected]>

This empty commit is to mark the start of patch series of KVM MMU GPA shared bits.

---
 Documentation/virt/kvm/intel-tdx-layer-status.rst | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

Documentation/virt/kvm/intel-tdx-layer-status.rst +3 -2

index 25082e9c0b20..8b8186e7bfeb 100644
@@ -10,6 +10,7 @@ What qemu can do
 ----------------
 - TDX VM TYPE is exposed to Qemu.
 - Qemu can create/destroy guest of TDX vm type.
+- Qemu can create/destroy vcpu of TDX vm type.
 
 Patch Layer status
 ------------------

@@ -18,12 +19,12 @@ Patch Layer status
 * TDX, VMX coexistence:                 Applied
 * TDX architectural definitions:        Applied
 * TD VM creation/destruction:           Applied
-* TD vcpu creation/destruction:         Applying
+* TD vcpu creation/destruction:         Applied
 * TDX EPT violation:                    Not yet
 * TD finalization:                      Not yet
 * TD vcpu enter/exit:                   Not yet
 * TD vcpu interrupts/exit/hypercall:    Not yet
 
-* KVM MMU GPA shared bits:              Not yet
+* KVM MMU GPA shared bits:              Applying
 * KVM TDP refactoring for TDX:          Not yet
 * KVM TDP MMU hooks:                    Not yet

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:01 UTC #89 lore raw

PATCH v13 27/113 KVM: x86/mmu: introduce config for PRIVATE KVM MMU

From: Isaku Yamahata <[email protected]>

To keep the case of non TDX intact, introduce a new config option for private KVM MMU support. At the moment, this is synonym for CONFIG_INTEL_TDX_HOST && CONFIG_KVM_INTEL. The config makes it clear that the config is only for x86 KVM MMU.

---
 arch/x86/kvm/Kconfig | 4 ++++
 1 file changed, 4 insertions(+)

arch/x86/kvm/Kconfig +4 -0

index 718010600956..45ac6b01af44 100644
@@ -153,4 +153,8 @@ config KVM_XEN
 config KVM_EXTERNAL_WRITE_TRACKING
 	bool
 
+config KVM_MMU_PRIVATE
+	def_bool y
+	depends on INTEL_TDX_HOST && KVM_INTEL
+
 endif # VIRTUALIZATION

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:01 UTC #90 lore raw

PATCH v13 28/113 KVM: x86/mmu: Add address conversion functions for TDX shared bit of GPA

From: Isaku Yamahata <[email protected]>

TDX repurposes one GPA bit (51 bit or 47 bit based on configuration) to indicate the GPA is private(if cleared) or shared (if set) with VMM. If GPA.shared is set, GPA is covered by the existing conventional EPT pointed by EPTP. If GPA.shared bit is cleared, GPA is covered by TDX module. VMM has to issue SEAMCALLs to operate.

Add a member to remember GPA shared bit for each guest TDs, add address conversion functions between private GPA and shared GPA and test if GPA is private.

Because struct kvm_arch (or struct kvm which includes struct kvm_arch. See kvm_arch_alloc_vm() that passes __GPF_ZERO) is zero-cleared when allocated, the new member to remember GPA shared bit is guaranteed to be zero with this patch unless it's initialized explicitly.

Co-developed-by Rick Edgecombe <[email protected]>

Signed-off-by Rick Edgecombe <[email protected]>

---
 arch/x86/include/asm/kvm_host.h |  4 ++++
 arch/x86/kvm/mmu.h              | 27 +++++++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.c          |  5 +++++
 3 files changed, 36 insertions(+)

arch/x86/include/asm/kvm_host.h +4 -0

index 42fa696a04ea..1802ee8bf1ad 100644
@@ -1463,6 +1463,10 @@ struct kvm_arch {
 	 */
 #define SPLIT_DESC_CACHE_MIN_NR_OBJECTS (SPTE_ENT_PER_PAGE + 1)
 	struct kvm_mmu_memory_cache split_desc_cache;
+
+#ifdef CONFIG_KVM_MMU_PRIVATE
+	gfn_t gfn_shared_mask;
+#endif
 };
 
 struct kvm_vm_stat {

arch/x86/kvm/mmu.h +27 -0

index 168c46fd8dd1..34a7266f145f 100644
@@ -278,4 +278,31 @@ static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu,
 		return gpa;
 	return translate_nested_gpa(vcpu, gpa, access, exception);
 }
+
+static inline gfn_t kvm_gfn_shared_mask(const struct kvm *kvm)
+{
+#ifdef CONFIG_KVM_MMU_PRIVATE
+	return kvm->arch.gfn_shared_mask;
+#else
+	return 0;
+#endif
+}
+
+static inline gfn_t kvm_gfn_to_shared(const struct kvm *kvm, gfn_t gfn)
+{
+	return gfn | kvm_gfn_shared_mask(kvm);
+}
+
+static inline gfn_t kvm_gfn_to_private(const struct kvm *kvm, gfn_t gfn)
+{
+	return gfn & ~kvm_gfn_shared_mask(kvm);
+}
+
+static inline bool kvm_is_private_gpa(const struct kvm *kvm, gpa_t gpa)
+{
+	gfn_t mask = kvm_gfn_shared_mask(kvm);
+
+	return mask && !(gpa_to_gfn(gpa) & mask);
+}
+
 #endif

arch/x86/kvm/vmx/tdx.c +5 -0

index 1415ab60ce2c..1994f4463fd8 100644
@@ -826,6 +826,11 @@ static int tdx_td_init(struct kvm *kvm, struct kvm_tdx_cmd *cmd)
 	kvm_tdx->attributes = td_params->attributes;
 	kvm_tdx->xfam = td_params->xfam;
 
+	if (td_params->exec_controls & TDX_EXEC_CONTROL_MAX_GPAW)
+		kvm->arch.gfn_shared_mask = gpa_to_gfn(BIT_ULL(51));
+	else
+		kvm->arch.gfn_shared_mask = gpa_to_gfn(BIT_ULL(47));
+
 out:
 	/* kfree() accepts NULL. */
 	kfree(init_vm);

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:01 UTC #91 lore raw

PATCH v13 29/113 The start of TDX KVM patch series: KVM TDP refactoring for TDX

From: Isaku Yamahata <[email protected]>

This empty commit is to mark the start of patch series of KVM TDP refactoring for TDX.

---
 Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Documentation/virt/kvm/intel-tdx-layer-status.rst +2 -2

index 8b8186e7bfeb..e893a3d714c7 100644
@@ -25,6 +25,6 @@ Patch Layer status
 * TD vcpu enter/exit:                   Not yet
 * TD vcpu interrupts/exit/hypercall:    Not yet
 
-* KVM MMU GPA shared bits:              Applying
-* KVM TDP refactoring for TDX:          Not yet
+* KVM MMU GPA shared bits:              Applied
+* KVM TDP refactoring for TDX:          Applying
 * KVM TDP MMU hooks:                    Not yet

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:02 UTC #92 lore raw

PATCH v13 30/113 KVM: Allow page-sized MMU caches to be initialized with custom 64-bit values

From: Sean Christopherson <[email protected]>

Add support to MMU caches for initializing a page with a custom 64-bit value, e.g. to pre-fill an entire page table with non-zero PTE values. The functionality will be used by x86 to support Intel's TDX, which needs to set bit 63 in all non-present PTEs in order to prevent !PRESENT page faults from getting reflected into the guest (Intel's EPT Violation #VE architecture made the less than brilliant decision of having the per-PTE behavior be opt-out instead of opt-in).

Signed-off-by Sean Christopherson <[email protected]>

---
 include/linux/kvm_types.h |  1 +
 virt/kvm/kvm_main.c       | 16 ++++++++++++++--
 2 files changed, 15 insertions(+), 2 deletions(-)

include/linux/kvm_types.h +1 -0

index 2728d49bbdf6..7c2b9332b7c5 100644
@@ -94,6 +94,7 @@ struct kvm_mmu_memory_cache {
 	int nobjs;
 	gfp_t gfp_zero;
 	gfp_t gfp_custom;
+	u64 init_value;
 	struct kmem_cache *kmem_cache;
 	int capacity;
 	void **objects;

virt/kvm/kvm_main.c +14 -2

index f8495e27d210..87400796df6e 100644
@@ -381,12 +381,17 @@ static void kvm_flush_shadow_all(struct kvm *kvm)
 static inline void *mmu_memory_cache_alloc_obj(struct kvm_mmu_memory_cache *mc,
 					       gfp_t gfp_flags)
 {
+	void *page;
+
 	gfp_flags |= mc->gfp_zero;
 
 	if (mc->kmem_cache)
 		return kmem_cache_alloc(mc->kmem_cache, gfp_flags);
-	else
-		return (void *)__get_free_page(gfp_flags);
+
+	page = (void *)__get_free_page(gfp_flags);
+	if (page && mc->init_value)
+		memset64(page, mc->init_value, PAGE_SIZE / sizeof(mc->init_value));
+	return page;
 }
 
 int __kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int capacity, int min)

@@ -401,6 +406,13 @@ int __kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int capacity,
 		if (WARN_ON_ONCE(!capacity))
 			return -EIO;
 
+		/*
+		 * Custom init values can be used only for page allocations,
+		 * and obviously conflict with __GFP_ZERO.
+		 */
+		if (WARN_ON_ONCE(mc->init_value && (mc->kmem_cache || mc->gfp_zero)))
+			return -EIO;
+
 		mc->objects = kvmalloc_array(sizeof(void *), capacity, gfp);
 		if (!mc->objects)
 			return -ENOMEM;

Signature · 1 line

2.25.1

Zhi Wang

2023-04-05 07:12 UTC #93 lore raw

Re: [PATCH v13 030/113] KVM: Allow page-sized MMU caches to be initialized with custom 64-bit values

Quoted 11 lines · On Sun, 12 Mar 2023 10:55:54 -0700 [email protected] wrote:

From: Sean Christopherson <[email protected]>
Add support to MMU caches for initializing a page with a custom 64-bit value, e.g. to pre-fill an entire page table with non-zero PTE values. The functionality will be used by x86 to support Intel's TDX, which needs to set bit 63 in all non-present PTEs in order to prevent !PRESENT page faults from getting reflected into the guest (Intel's EPT Violation #VE architecture made the less than brilliant decision of having the per-PTE behavior be opt-out instead of opt-in).
Signed-off-by Sean Christopherson <[email protected]>

Your Sighned-off-by is required as well.

Quoted 55 lines

---
 include/linux/kvm_types.h |  1 +
 virt/kvm/kvm_main.c       | 16 ++++++++++++++--
 2 files changed, 15 insertions(+), 2 deletions(-)

include/linux/kvm_types.h +1 -0

index 2728d49bbdf6..7c2b9332b7c5 100644
@@ -94,6 +94,7 @@ struct kvm_mmu_memory_cache {
 	int nobjs;
 	gfp_t gfp_zero;
 	gfp_t gfp_custom;
+	u64 init_value;
 	struct kmem_cache *kmem_cache;
 	int capacity;
 	void **objects;

virt/kvm/kvm_main.c +14 -2

index f8495e27d210..87400796df6e 100644
@@ -381,12 +381,17 @@ static void kvm_flush_shadow_all(struct kvm *kvm)
 static inline void *mmu_memory_cache_alloc_obj(struct kvm_mmu_memory_cache *mc,
 					       gfp_t gfp_flags)
 {
+	void *page;
+
 	gfp_flags |= mc->gfp_zero;
 
 	if (mc->kmem_cache)
 		return kmem_cache_alloc(mc->kmem_cache, gfp_flags);
-	else
-		return (void *)__get_free_page(gfp_flags);
+
+	page = (void *)__get_free_page(gfp_flags);
+	if (page && mc->init_value)
+		memset64(page, mc->init_value, PAGE_SIZE / sizeof(mc->init_value));
+	return page;
 }
 
 int __kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int capacity, int min)

@@ -401,6 +406,13 @@ int __kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int capacity,
 		if (WARN_ON_ONCE(!capacity))
 			return -EIO;
 
+		/*
+		 * Custom init values can be used only for page allocations,
+		 * and obviously conflict with __GFP_ZERO.
+		 */
+		if (WARN_ON_ONCE(mc->init_value && (mc->kmem_cache || mc->gfp_zero)))
+			return -EIO;
+
 		mc->objects = kvmalloc_array(sizeof(void *), capacity, gfp);
 		if (!mc->objects)
 			return -ENOMEM;

[email protected]

author

2023-03-12 18:02 UTC #94 lore raw

PATCH v13 31/113 KVM: x86/mmu: Replace hardcoded value 0 for the initial value for SPTE

From: Isaku Yamahata <[email protected]>

The TDX support will need the "suppress #VE" bit (bit 63) set as the initial value for SPTE. To reduce code change size, introduce a new macro SHADOW_NONPRESENT_VALUE for the initial value for the shadow page table entry (SPTE) and replace hard-coded value 0 for it. Initialize shadow page tables with their value.

The plan is to unconditionally set the "suppress #VE" bit for both AMD and Intel as: 1) AMD hardware uses the bit 63 as NX for present SPTE and ignored for non-present SPTE; 2) for conventional VMX guests, KVM never enables the "EPT-violation #VE" in VMCS control and "suppress #VE" bit is ignored by hardware.

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/kvm/mmu/mmu.c         | 17 +++++++++++++----
 arch/x86/kvm/mmu/paging_tmpl.h |  3 ++-
 arch/x86/kvm/mmu/spte.h        |  2 ++
 arch/x86/kvm/mmu/tdp_mmu.c     | 15 ++++++++-------
 4 files changed, 25 insertions(+), 12 deletions(-)

arch/x86/kvm/mmu/mmu.c +13 -4

index a11183dbaaa0..6026c5c130d4 100644
@@ -559,9 +559,9 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep)
 
 	if (!is_shadow_present_pte(old_spte) ||
 	    !spte_has_volatile_bits(old_spte))
-		__update_clear_spte_fast(sptep, 0ull);
+		__update_clear_spte_fast(sptep, SHADOW_NONPRESENT_VALUE);
 	else
-		old_spte = __update_clear_spte_slow(sptep, 0ull);
+		old_spte = __update_clear_spte_slow(sptep, SHADOW_NONPRESENT_VALUE);
 
 	if (!is_shadow_present_pte(old_spte))
 		return old_spte;

@@ -595,7 +595,7 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep)
  */
 static void mmu_spte_clear_no_track(u64 *sptep)
 {
-	__update_clear_spte_fast(sptep, 0ull);
+	__update_clear_spte_fast(sptep, SHADOW_NONPRESENT_VALUE);
 }
 
 static u64 mmu_spte_get_lockless(u64 *sptep)

@@ -6013,7 +6013,16 @@ int kvm_mmu_create(struct kvm_vcpu *vcpu)
 	vcpu->arch.mmu_page_header_cache.kmem_cache = mmu_page_header_cache;
 	vcpu->arch.mmu_page_header_cache.gfp_zero = __GFP_ZERO;
 
-	vcpu->arch.mmu_shadow_page_cache.gfp_zero = __GFP_ZERO;
+	/*
+	 * When X86_64, initial SEPT entries are initialized with
+	 * SHADOW_NONPRESENT_VALUE.  Otherwise zeroed.  See
+	 * mmu_memory_cache_alloc_obj().
+	 */
+	if (IS_ENABLED(CONFIG_X86_64))
+		vcpu->arch.mmu_shadow_page_cache.init_value =
+			SHADOW_NONPRESENT_VALUE;
+	if (!vcpu->arch.mmu_shadow_page_cache.init_value)
+		vcpu->arch.mmu_shadow_page_cache.gfp_zero = __GFP_ZERO;
 
 	vcpu->arch.mmu = &vcpu->arch.root_mmu;
 	vcpu->arch.walk_mmu = &vcpu->arch.root_mmu;

arch/x86/kvm/mmu/paging_tmpl.h +2 -1

index 57f0b75c80f9..f5b0e7c3877e 100644
@@ -1025,7 +1025,8 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)
 		gpa_t pte_gpa;
 		gfn_t gfn;
 
-		if (!sp->spt[i])
+		/* spt[i] has initial value of shadow page table allocation */
+		if (sp->spt[i] == SHADOW_NONPRESENT_VALUE)
 			continue;
 
 		pte_gpa = first_pte_gpa + i * sizeof(pt_element_t);

arch/x86/kvm/mmu/spte.h +2 -0

index 1279db2eab44..a99eb7d4ae5d 100644
@@ -148,6 +148,8 @@ static_assert(MMIO_SPTE_GEN_LOW_BITS == 8 && MMIO_SPTE_GEN_HIGH_BITS == 11);
 
 #define MMIO_SPTE_GEN_MASK		GENMASK_ULL(MMIO_SPTE_GEN_LOW_BITS + MMIO_SPTE_GEN_HIGH_BITS - 1, 0)
 
+#define SHADOW_NONPRESENT_VALUE	0ULL
+
 extern u64 __read_mostly shadow_host_writable_mask;
 extern u64 __read_mostly shadow_mmu_writable_mask;
 extern u64 __read_mostly shadow_nx_mask;

arch/x86/kvm/mmu/tdp_mmu.c +8 -7

index f2b56584089b..76e3751c8e9d 100644
@@ -689,7 +689,7 @@ static inline int tdp_mmu_zap_spte_atomic(struct kvm *kvm,
 	 * here since the SPTE is going from non-present to non-present.  Use
 	 * the raw write helper to avoid an unnecessary check on volatile bits.
 	 */
-	__kvm_tdp_mmu_write_spte(iter->sptep, 0);
+	__kvm_tdp_mmu_write_spte(iter->sptep, SHADOW_NONPRESENT_VALUE);
 
 	return 0;
 }

@@ -866,8 +866,8 @@ static void __tdp_mmu_zap_root(struct kvm *kvm, struct kvm_mmu_page *root,
 			continue;
 
 		if (!shared)
-			tdp_mmu_set_spte(kvm, &iter, 0);
-		else if (tdp_mmu_set_spte_atomic(kvm, &iter, 0))
+			tdp_mmu_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE);
+		else if (tdp_mmu_set_spte_atomic(kvm, &iter, SHADOW_NONPRESENT_VALUE))
 			goto retry;
 	}
 }

@@ -923,8 +923,9 @@ bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp)
 	if (WARN_ON_ONCE(!is_shadow_present_pte(old_spte)))
 		return false;
 
-	__tdp_mmu_set_spte(kvm, kvm_mmu_page_as_id(sp), sp->ptep, old_spte, 0,
-			   sp->gfn, sp->role.level + 1, true, true);
+	__tdp_mmu_set_spte(kvm, kvm_mmu_page_as_id(sp), sp->ptep, old_spte,
+			   SHADOW_NONPRESENT_VALUE, sp->gfn, sp->role.level + 1,
+			   true, true);
 
 	return true;
 }

@@ -958,7 +959,7 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root,
 		    !is_last_spte(iter.old_spte, iter.level))
 			continue;
 
-		tdp_mmu_set_spte(kvm, &iter, 0);
+		tdp_mmu_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE);
 		flush = true;
 	}

@@ -1326,7 +1327,7 @@ static bool set_spte_gfn(struct kvm *kvm, struct tdp_iter *iter,
 	 * invariant that the PFN of a present * leaf SPTE can never change.
 	 * See __handle_changed_spte().
 	 */
-	tdp_mmu_set_spte(kvm, iter, 0);
+	tdp_mmu_set_spte(kvm, iter, SHADOW_NONPRESENT_VALUE);
 
 	if (!pte_write(range->pte)) {
 		new_spte = kvm_mmu_changed_pte_notifier_make_spte(iter->old_spte,

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:02 UTC #95 lore raw

PATCH v13 32/113 KVM: x86/mmu: Allow non-zero value for non-present SPTE and removed SPTE

From: Sean Christopherson <[email protected]>

For TD guest, the current way to emulate MMIO doesn't work any more, as KVM is not able to access the private memory of TD guest and do the emulation. Instead, TD guest expects to receive #VE when it accesses the MMIO and then it can explicitly make hypercall to KVM to get the expected information.

To achieve this, the TDX module always enables "EPT-violation #VE" in the VMCS control. And accordingly, for the MMIO spte for the shared GPA,
1. KVM needs to set "suppress #VE" bit for the non-present SPTE so that EPT violation happens on TD accessing MMIO range. 2. On EPT violation, KVM sets the MMIO spte to clear "suppress #VE" bit so the TD guest can receive the #VE instead of EPT misconfigration unlike VMX case. For the shared GPA that is not populated yet, EPT violation need to be triggered when TD guest accesses such shared GPA. The non-present SPTE value for shared GPA should set "suppress #VE" bit.

Add "suppress #VE" bit (bit 63) to SHADOW_NONPRESENT_VALUE and REMOVED_SPTE. Unconditionally set the "suppress #VE" bit (which is bit 63) for both AMD and Intel as: 1) AMD hardware doesn't use this bit when present bit is off; 2) for normal VMX guest, KVM never enables the "EPT-violation #VE" in VMCS control and "suppress #VE" bit is ignored by hardware.

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/kvm/mmu/spte.h | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

arch/x86/kvm/mmu/spte.h +14 -1

index a99eb7d4ae5d..a57667810344 100644
@@ -148,7 +148,20 @@ static_assert(MMIO_SPTE_GEN_LOW_BITS == 8 && MMIO_SPTE_GEN_HIGH_BITS == 11);
 
 #define MMIO_SPTE_GEN_MASK		GENMASK_ULL(MMIO_SPTE_GEN_LOW_BITS + MMIO_SPTE_GEN_HIGH_BITS - 1, 0)
 
+/*
+ * Non-present SPTE value for both VMX and SVM for TDP MMU.
+ * For SVM NPT, for non-present spte (bit 0 = 0), other bits are ignored.
+ * For VMX EPT, bit 63 is ignored if #VE is disabled. (EPT_VIOLATION_VE=0)
+ *              bit 63 is #VE suppress if #VE is enabled. (EPT_VIOLATION_VE=1)
+ * For TDX:
+ *   TDX module sets EPT_VIOLATION_VE for Secure-EPT and conventional EPT
+ */
+#ifdef CONFIG_X86_64
+#define SHADOW_NONPRESENT_VALUE	BIT_ULL(63)
+static_assert(!(SHADOW_NONPRESENT_VALUE & SPTE_MMU_PRESENT_MASK));
+#else
 #define SHADOW_NONPRESENT_VALUE	0ULL
+#endif
 
 extern u64 __read_mostly shadow_host_writable_mask;
 extern u64 __read_mostly shadow_mmu_writable_mask;

@@ -195,7 +208,7 @@ extern u64 __read_mostly shadow_nonpresent_or_rsvd_mask;
  *
  * Only used by the TDP MMU.
  */
-#define REMOVED_SPTE	0x5a0ULL
+#define REMOVED_SPTE	(SHADOW_NONPRESENT_VALUE | 0x5a0ULL)
 
 /* Removed SPTEs must not be misconstrued as shadow present PTEs. */
 static_assert(!(REMOVED_SPTE & SPTE_MMU_PRESENT_MASK));

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:02 UTC #96 lore raw

PATCH v13 33/113 KVM: x86/mmu: Add Suppress VE bit to shadow_mmio_mask/shadow_present_mask

From: Isaku Yamahata <[email protected]>

To make use of the same value of shadow_mmio_mask and shadow_present_mask for TDX and VMX, add Suppress-VE bit to shadow_mmio_mask and shadow_present_mask so that they can be common for both VMX and TDX.

TDX will require shadow_mmio_mask and shadow_present_mask to include VMX_SUPPRESS_VE for shared GPA so that EPT violation is triggered for shared GPA. For VMX, VMX_SUPPRESS_VE doesn't matter for MMIO because the spte value is required to cause EPT misconfig. the additional bit doesn't affect VMX logic to add the bit to shadow_mmio_{value, mask}.

---
 arch/x86/include/asm/vmx.h | 1 +
 arch/x86/kvm/mmu/spte.c    | 6 ++++--
 2 files changed, 5 insertions(+), 2 deletions(-)

arch/x86/include/asm/vmx.h +1 -0

index 498dc600bd5c..cdbf12c1a83c 100644
@@ -511,6 +511,7 @@ enum vmcs_field {
 #define VMX_EPT_IPAT_BIT    			(1ull << 6)
 #define VMX_EPT_ACCESS_BIT			(1ull << 8)
 #define VMX_EPT_DIRTY_BIT			(1ull << 9)
+#define VMX_EPT_SUPPRESS_VE_BIT			(1ull << 63)
 #define VMX_EPT_RWX_MASK                        (VMX_EPT_READABLE_MASK |       \
 						 VMX_EPT_WRITABLE_MASK |       \
 						 VMX_EPT_EXECUTABLE_MASK)

arch/x86/kvm/mmu/spte.c +4 -2

index c15bfca3ed15..41802e6d6f57 100644
@@ -431,7 +431,9 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only)
 	shadow_dirty_mask	= has_ad_bits ? VMX_EPT_DIRTY_BIT : 0ull;
 	shadow_nx_mask		= 0ull;
 	shadow_x_mask		= VMX_EPT_EXECUTABLE_MASK;
-	shadow_present_mask	= has_exec_only ? 0ull : VMX_EPT_READABLE_MASK;
+	/* VMX_EPT_SUPPRESS_VE_BIT is needed for W or X violation. */
+	shadow_present_mask	=
+		(has_exec_only ? 0ull : VMX_EPT_READABLE_MASK) | VMX_EPT_SUPPRESS_VE_BIT;
 	/*
 	 * EPT overrides the host MTRRs, and so KVM must program the desired
 	 * memtype directly into the SPTEs.  Note, this mask is just the mask

@@ -448,7 +450,7 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only)
 	 * of an EPT paging-structure entry is 110b (write/execute).
 	 */
 	kvm_mmu_set_mmio_spte_mask(VMX_EPT_MISCONFIG_WX_VALUE,
-				   VMX_EPT_RWX_MASK, 0);
+				   VMX_EPT_RWX_MASK | VMX_EPT_SUPPRESS_VE_BIT, 0);
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_set_ept_masks);

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:02 UTC #97 lore raw

PATCH v13 34/113 KVM: x86/mmu: Track shadow MMIO value on a per-VM basis

From: Isaku Yamahata <[email protected]>

TDX will use a different shadow PTE entry value for MMIO from VMX. Add members to kvm_arch and track value for MMIO per-VM instead of global variables. By using the per-VM EPT entry value for MMIO, the existing VMX logic is kept working. Introduce a separate setter function so that guest TD can override later.

Also require mmio spte cachcing for TDX. Actually this is true case because TDX require EPT and KVM EPT allows mmio spte caching.

Signed-off-by Sean Christopherson <[email protected]>

---
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/mmu.h              |  1 +
 arch/x86/kvm/mmu/mmu.c          |  7 ++++---
 arch/x86/kvm/mmu/spte.c         | 10 ++++++++--
 arch/x86/kvm/mmu/spte.h         |  4 ++--
 arch/x86/kvm/mmu/tdp_mmu.c      |  6 +++---
 6 files changed, 20 insertions(+), 10 deletions(-)

arch/x86/include/asm/kvm_host.h +2 -0

index 1802ee8bf1ad..0fdd432e62e2 100644
@@ -1276,6 +1276,8 @@ struct kvm_arch {
 	 */
 	spinlock_t mmu_unsync_pages_lock;
 
+	u64 shadow_mmio_value;
+
 	struct list_head assigned_dev_head;
 	struct iommu_domain *iommu_domain;
 	bool iommu_noncoherent;

arch/x86/kvm/mmu.h +1 -0

index 34a7266f145f..0ecdad63ad8c 100644
@@ -101,6 +101,7 @@ static inline u8 kvm_get_shadow_phys_bits(void)
 }
 
 void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask);
+void kvm_mmu_set_mmio_spte_value(struct kvm *kvm, u64 mmio_value);
 void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask);
 void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only);

arch/x86/kvm/mmu/mmu.c +4 -3

index 6026c5c130d4..9bc640e0d821 100644
@@ -2455,7 +2455,7 @@ static int mmu_page_zap_pte(struct kvm *kvm, struct kvm_mmu_page *sp,
 				return kvm_mmu_prepare_zap_page(kvm, child,
 								invalid_list);
 		}
-	} else if (is_mmio_spte(pte)) {
+	} else if (is_mmio_spte(kvm, pte)) {
 		mmu_spte_clear_no_track(spte);
 	}
 	return 0;

@@ -4135,7 +4135,7 @@ static int handle_mmio_page_fault(struct kvm_vcpu *vcpu, u64 addr, bool direct)
 	if (WARN_ON(reserved))
 		return -EINVAL;
 
-	if (is_mmio_spte(spte)) {
+	if (is_mmio_spte(vcpu->kvm, spte)) {
 		gfn_t gfn = get_mmio_spte_gfn(spte);
 		unsigned int access = get_mmio_spte_access(spte);

@@ -4686,7 +4686,7 @@ static unsigned long get_cr3(struct kvm_vcpu *vcpu)
 static bool sync_mmio_spte(struct kvm_vcpu *vcpu, u64 *sptep, gfn_t gfn,
 			   unsigned int access)
 {
-	if (unlikely(is_mmio_spte(*sptep))) {
+	if (unlikely(is_mmio_spte(vcpu->kvm, *sptep))) {
 		if (gfn != get_mmio_spte_gfn(*sptep)) {
 			mmu_spte_clear_no_track(sptep);
 			return true;

@@ -6175,6 +6175,7 @@ int kvm_mmu_init_vm(struct kvm *kvm)
 	struct kvm_page_track_notifier_node *node = &kvm->arch.mmu_sp_tracker;
 	int r;
 
+	kvm->arch.shadow_mmio_value = shadow_mmio_value;
 	INIT_LIST_HEAD(&kvm->arch.active_mmu_pages);
 	INIT_LIST_HEAD(&kvm->arch.zapped_obsolete_pages);
 	INIT_LIST_HEAD(&kvm->arch.possible_nx_huge_pages);

arch/x86/kvm/mmu/spte.c +8 -2

index 41802e6d6f57..72c7b8d27efb 100644
@@ -74,10 +74,10 @@ u64 make_mmio_spte(struct kvm_vcpu *vcpu, u64 gfn, unsigned int access)
 	u64 spte = generation_mmio_spte_mask(gen);
 	u64 gpa = gfn << PAGE_SHIFT;
 
-	WARN_ON_ONCE(!shadow_mmio_value);
+	WARN_ON_ONCE(!vcpu->kvm->arch.shadow_mmio_value);
 
 	access &= shadow_mmio_access_mask;
-	spte |= shadow_mmio_value | access;
+	spte |= vcpu->kvm->arch.shadow_mmio_value | access;
 	spte |= gpa | shadow_nonpresent_or_rsvd_mask;
 	spte |= (gpa & shadow_nonpresent_or_rsvd_mask)
 		<< SHADOW_NONPRESENT_OR_RSVD_MASK_LEN;

@@ -413,6 +413,12 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask)
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_set_mmio_spte_mask);
 
+void kvm_mmu_set_mmio_spte_value(struct kvm *kvm, u64 mmio_value)
+{
+	kvm->arch.shadow_mmio_value = mmio_value;
+}
+EXPORT_SYMBOL_GPL(kvm_mmu_set_mmio_spte_value);
+
 void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask)
 {
 	/* shadow_me_value must be a subset of shadow_me_mask */

arch/x86/kvm/mmu/spte.h +2 -2

index a57667810344..a8418fd8ae9e 100644
@@ -251,9 +251,9 @@ static inline struct kvm_mmu_page *sptep_to_sp(u64 *sptep)
 	return to_shadow_page(__pa(sptep));
 }
 
-static inline bool is_mmio_spte(u64 spte)
+static inline bool is_mmio_spte(struct kvm *kvm, u64 spte)
 {
-	return (spte & shadow_mmio_mask) == shadow_mmio_value &&
+	return (spte & shadow_mmio_mask) == kvm->arch.shadow_mmio_value &&
 	       likely(enable_mmio_caching);
 }

arch/x86/kvm/mmu/tdp_mmu.c +3 -3

index 76e3751c8e9d..b342ce3a6e83 100644
@@ -576,8 +576,8 @@ static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn,
 		 * impact the guest since both the former and current SPTEs
 		 * are nonpresent.
 		 */
-		if (WARN_ON(!is_mmio_spte(old_spte) &&
-			    !is_mmio_spte(new_spte) &&
+		if (WARN_ON(!is_mmio_spte(kvm, old_spte) &&
+			    !is_mmio_spte(kvm, new_spte) &&
 			    !is_removed_spte(new_spte)))
 			pr_err("Unexpected SPTE change! Nonpresent SPTEs\n"
 			       "should not be replaced with another,\n"

@@ -1093,7 +1093,7 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu,
 	}
 
 	/* If a MMIO SPTE is installed, the MMIO will need to be emulated. */
-	if (unlikely(is_mmio_spte(new_spte))) {
+	if (unlikely(is_mmio_spte(vcpu->kvm, new_spte))) {
 		vcpu->stat.pf_mmio_spte_created++;
 		trace_mark_mmio_spte(rcu_dereference(iter->sptep), iter->gfn,
 				     new_spte);

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:03 UTC #98 lore raw

PATCH v13 35/113 KVM: x86/mmu: Disallow fast page fault on private GPA

From: Isaku Yamahata <[email protected]>

TDX requires TDX SEAMCALL to operate Secure EPT instead of direct memory access and TDX SEAMCALL is heavy operation. Fast page fault on private GPA doesn't make sense. Disallow fast page fault on private GPA.

---
 arch/x86/kvm/mmu/mmu.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

arch/x86/kvm/mmu/mmu.c +10 -2

index 9bc640e0d821..5abf2c9c3402 100644
@@ -3289,8 +3289,16 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu,
 	return RET_PF_CONTINUE;
 }
 
-static bool page_fault_can_be_fast(struct kvm_page_fault *fault)
+static bool page_fault_can_be_fast(struct kvm *kvm, struct kvm_page_fault *fault)
 {
+	/*
+	 * TDX private mapping doesn't support fast page fault because the EPT
+	 * entry is read/written with TDX SEAMCALLs instead of direct memory
+	 * access.
+	 */
+	if (kvm_is_private_gpa(kvm, fault->addr))
+		return false;
+
 	/*
 	 * Page faults with reserved bits set, i.e. faults on MMIO SPTEs, only
 	 * reach the common page fault handler if the SPTE has an invalid MMIO

@@ -3400,7 +3408,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)
 	u64 *sptep = NULL;
 	uint retry_count = 0;
 
-	if (!page_fault_can_be_fast(fault))
+	if (!page_fault_can_be_fast(vcpu->kvm, fault))
 		return ret;
 
 	walk_shadow_page_lockless_begin(vcpu);

Signature · 1 line

2.25.1

[email protected]

author

2023-03-12 18:03 UTC #99 lore raw

PATCH v13 36/113 KVM: x86/mmu: Allow per-VM override of the TDP max page level

From: Sean Christopherson <[email protected]>

TDX requires special handling to support large private page. For simplicity, only support 4K page for TD guest for now. Add per-VM maximum page level support to support different maximum page sizes for TD guest and conventional VMX guest.

Signed-off-by Sean Christopherson <[email protected]>