Skip to contentSkip to conversation

TDX KVM selftests

RFC PATCH In review Sagi Shahar 3 participants 23 messages Aug 30, 2022 – Sep 1, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v2 cover TDX KVM selftests

Hello,

This is v2 of the patch series for TDX selftests.

It is based on v5.19-rc8 and Intel's V8 of the TDX host patches which was proposed in https://lkml.org/lkml/2022/8/8/877

The tree can be found at
https://github.com/googleprodkernel/linux-cc/tree/selftests

Major changes vrom v1:
- rebased to v5.19
- added helpers for success and failure reporting
- added additional test cases

---
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform.

Intel has recently submitted a set of RFC patches for KVM support for TDX and more information can be found on the latest TDX Support
Patches: https://lkml.org/lkml/2022/8/8/877

Due to the nature of the confidential computing environment that TDX provides, it is very difficult to verify/test the KVM support. TDX requires UEFI and the guest kernel to be enlightened which are all under development.

We are working on a set of selftests to close this gap and be able to verify the KVM functionality to support TDX lifecycle and GHCI [1] interface.

We are looking for any feedback on:
- Patch series itself
- Any suggestion on how we should approach testing TDX functionality. Does selftests seems reasonable or should we switch to using KVM unit tests. I would be happy to get some perspective on how KVM unit tests can help us more.
- Any test case or scenario that we should add.
- Anything else I have not thought of yet.

Current patch series provide the following capabilities:

- Provide helper functions to create a TD (Trusted Domain) using the KVM ioctls
- Provide helper functions to create a guest image that can include any testing code
- Provide helper functions and wrapper functions to write testing code using GHCI interface
- Add a test case that verifies TDX life cycle
- Add a test case that verifies TDX GHCI port IO

TODOs:
- Use existing function to create page tables dynamically (ie __virt_pg_map())
- Remove arbitrary defined magic numbers for data structure offsets
- Add TDVMCALL for error reporting
- Add additional test cases as some listed below
- Add #VE handlers to help testing more complicated test cases

---
Erdem Aktas (4):
KVM: selftests: Add support for creating non-default type VMs
KVM: selftest: Add helper functions to create TDX VMs
KVM: selftest: Adding TDX life cycle test.
KVM: selftest: Adding test case for TDX port IO

Roger Wang (1):
KVM: selftest: TDX: Add TDG.VP.INFO test

Ryan Afranji (2):
KVM: selftest: TDX: Verify the behavior when host consumes a TD
private memory
KVM: selftest: TDX: Add shared memory test

Sagi Shahar (10):
KVM: selftest: TDX: Add report_fatal_error test
KVM: selftest: TDX: Add basic TDX CPUID test
KVM: selftest: TDX: Add basic get_td_vmcall_info test
KVM: selftest: TDX: Add TDX IO writes test
KVM: selftest: TDX: Add TDX IO reads test
KVM: selftest: TDX: Add TDX MSR read/write tests
KVM: selftest: TDX: Add TDX HLT exit test
KVM: selftest: TDX: Add TDX MMIO reads test
KVM: selftest: TDX: Add TDX MMIO writes test
KVM: selftest: TDX: Add TDX CPUID TDVMCALL test

 tools/testing/selftests/kvm/Makefile          |    2 +
 .../selftests/kvm/include/kvm_util_base.h     |   12 +-
 .../selftests/kvm/include/x86_64/processor.h  |    1 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |    6 +-
 .../selftests/kvm/lib/x86_64/processor.c      |   27 +
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  495 +++++
 .../selftests/kvm/lib/x86_64/tdx_lib.c        |  373 ++++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 1666 +++++++++++++++++
 8 files changed, 2577 insertions(+), 5 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/lib/x86_64/tdx.h
 create mode 100644 tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c
 create mode 100644 tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c
Signature · 1 line
2.37.2.789.g6183377224-goog
#1 lore raw

RFC PATCH v2 1/17 KVM: selftests: Add support for creating non-default type VMs

From: Erdem Aktas <[email protected]>

Currently vm_create function only creates KVM_VM_TYPE_DEFAULT type VMs. Adding type parameter to ____vm_create to create new VM types.

---
 tools/testing/selftests/kvm/include/kvm_util_base.h | 6 ++++--
 tools/testing/selftests/kvm/lib/kvm_util.c          | 6 +++---
 2 files changed, 7 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/include/kvm_util_base.h +4 -2
index 24fde97f6121..2de7a7a2e56b 100644
@@ -26,6 +26,8 @@
 
 #define NSEC_PER_SEC 1000000000L
 
+#define KVM_VM_TYPE_DEFAULT	0
+
 typedef uint64_t vm_paddr_t; /* Virtual Machine (Guest) physical address */
 typedef uint64_t vm_vaddr_t; /* Virtual Machine (Guest) virtual address */
 
@@ -642,13 +644,13 @@ vm_paddr_t vm_alloc_page_table(struct kvm_vm *vm);
  * __vm_create() does NOT create vCPUs, @nr_runnable_vcpus is used purely to
  * calculate the amount of memory needed for per-vCPU data, e.g. stacks.
  */
-struct kvm_vm *____vm_create(enum vm_guest_mode mode, uint64_t nr_pages);
+struct kvm_vm *____vm_create(enum vm_guest_mode mode, uint64_t nr_pages, int type);
 struct kvm_vm *__vm_create(enum vm_guest_mode mode, uint32_t nr_runnable_vcpus,
 			   uint64_t nr_extra_pages);
 
 static inline struct kvm_vm *vm_create_barebones(void)
 {
-	return ____vm_create(VM_MODE_DEFAULT, 0);
+	return ____vm_create(VM_MODE_DEFAULT, 0, KVM_VM_TYPE_DEFAULT);
 }
 
 static inline struct kvm_vm *vm_create(uint32_t nr_runnable_vcpus)
tools/testing/selftests/kvm/lib/kvm_util.c +3 -3
index 9889fe0d8919..ac10ad8919a6 100644
@@ -143,7 +143,7 @@ const struct vm_guest_mode_params vm_guest_mode_params[] = {
 _Static_assert(sizeof(vm_guest_mode_params)/sizeof(struct vm_guest_mode_params) == NUM_VM_MODES,
 	       "Missing new mode params?");
 
-struct kvm_vm *____vm_create(enum vm_guest_mode mode, uint64_t nr_pages)
+struct kvm_vm *____vm_create(enum vm_guest_mode mode, uint64_t nr_pages, int type)
 {
 	struct kvm_vm *vm;
 
@@ -159,7 +159,7 @@ struct kvm_vm *____vm_create(enum vm_guest_mode mode, uint64_t nr_pages)
 	hash_init(vm->regions.slot_hash);
 
 	vm->mode = mode;
-	vm->type = 0;
+	vm->type = type;
 
 	vm->pa_bits = vm_guest_mode_params[mode].pa_bits;
 	vm->va_bits = vm_guest_mode_params[mode].va_bits;
@@ -294,7 +294,7 @@ struct kvm_vm *__vm_create(enum vm_guest_mode mode, uint32_t nr_runnable_vcpus,
 						 nr_extra_pages);
 	struct kvm_vm *vm;
 
-	vm = ____vm_create(mode, nr_pages);
+	vm = ____vm_create(mode, nr_pages, KVM_VM_TYPE_DEFAULT);
 
 	kvm_vm_elf_load(vm, program_invocation_name);
 
Signature · 1 line
2.37.2.789.g6183377224-goog
#2 lore raw

RFC PATCH v2 2/17 KVM: selftest: Add helper functions to create TDX VMs

In reply to Sagi Shahar

From: Erdem Aktas <[email protected]>

TDX requires additional IOCTLs to initialize VM and vCPUs, to add private memory and to finalize the VM memory. Also additional utility functions are provided to create a guest image that will include test code.

TDX enabled VM's memory is encrypted and cannot be modified or observed by the VMM. We need to create a guest image that includes the testing code.

When TDX is enabled, vCPUs will enter guest mode with 32 bit mode with paging disabled. TDX requires the CPU to run on long mode with paging enabled. The guest image should have transition code from 32 bit to 64 bit, enable paging and run the testing code. There has to be predifined offset values for each data structure that will be used by the guest code. The guest image layout is as following:

| Page Tables | GDTR | GDT | Stack | Testing Code | Transition Boot Code |

Guest image will be loaded to the bottom of the first 4GB of the memory.

---
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/include/kvm_util_base.h     |   6 +
 .../selftests/kvm/include/x86_64/processor.h  |   1 +
 .../selftests/kvm/lib/x86_64/processor.c      |  27 ++
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  86 +++++
 .../selftests/kvm/lib/x86_64/tdx_lib.c        | 338 ++++++++++++++++++
 6 files changed, 459 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/lib/x86_64/tdx.h
 create mode 100644 tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c
tools/testing/selftests/kvm/Makefile +1 -0
index 690b499c3471..ad4d60dadc06 100644
@@ -52,6 +52,7 @@ LIBKVM_x86_64 += lib/x86_64/handlers.S
 LIBKVM_x86_64 += lib/x86_64/perf_test_util.c
 LIBKVM_x86_64 += lib/x86_64/processor.c
 LIBKVM_x86_64 += lib/x86_64/svm.c
+LIBKVM_x86_64 += lib/x86_64/tdx_lib.c
 LIBKVM_x86_64 += lib/x86_64/ucall.c
 LIBKVM_x86_64 += lib/x86_64/vmx.c
 
tools/testing/selftests/kvm/include/kvm_util_base.h +6 -0
index 2de7a7a2e56b..8d3bcf1719c6 100644
@@ -653,6 +653,12 @@ static inline struct kvm_vm *vm_create_barebones(void)
 	return ____vm_create(VM_MODE_DEFAULT, 0, KVM_VM_TYPE_DEFAULT);
 }
 
+/* TDX VMs are always created with no memory and memory is added later */
+static inline struct kvm_vm *vm_create_tdx(void)
+{
+	return ____vm_create(VM_MODE_DEFAULT, 0, KVM_X86_TDX_VM);
+}
+
 static inline struct kvm_vm *vm_create(uint32_t nr_runnable_vcpus)
 {
 	return __vm_create(VM_MODE_DEFAULT, nr_runnable_vcpus, 0);
tools/testing/selftests/kvm/include/x86_64/processor.h +1 -0
index 45edf45821d0..57bbab7d025c 100644
@@ -855,6 +855,7 @@ enum pg_level {
 #define PG_SIZE_1G PG_LEVEL_SIZE(PG_LEVEL_1G)
 
 void __virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr, int level);
+struct kvm_vcpu *vm_vcpu_add_tdx(struct kvm_vm *vm, uint32_t vcpu_id);
 
 /*
  * Basic CPU control in CR0
tools/testing/selftests/kvm/lib/x86_64/processor.c +27 -0
index f35626df1dea..2a6e28c769f2 100644
@@ -8,6 +8,7 @@
 #include "test_util.h"
 #include "kvm_util.h"
 #include "processor.h"
+#include "tdx.h"
 
 #ifndef NUM_INTERRUPTS
 #define NUM_INTERRUPTS 256
@@ -641,6 +642,32 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id,
 	return vcpu;
 }
 
+/*
+ * Adds a vCPU to a TD (Trusted Domain) with minimum  defaults. It will not set
+ * up any general purpose registers as they will be initialized by the TDX. In
+ * TDX, vCPUs RIP is set to 0xFFFFFFF0. See Intel TDX EAS Section "Initial State
+ * of Guest GPRs" for more information on vCPUs initial register values when
+ * entering the TD first time.
+ *
+ * Input Args:
+ *   vm - Virtual Machine
+ *   vcpuid - The id of the VCPU to add to the VM.
+ */
+struct kvm_vcpu *vm_vcpu_add_tdx(struct kvm_vm *vm, uint32_t vcpu_id)
+{
+	struct kvm_mp_state mp_state;
+	struct kvm_vcpu *vcpu;
+
+	vcpu = __vm_vcpu_add(vm, vcpu_id);
+	initialize_td_vcpu(vcpu);
+
+	/* Setup the MP state */
+	mp_state.mp_state = 0;
+	vcpu_mp_state_set(vcpu, &mp_state);
+
+	return vcpu;
+}
+
 struct kvm_vcpu *vm_arch_vcpu_recreate(struct kvm_vm *vm, uint32_t vcpu_id)
 {
 	struct kvm_vcpu *vcpu = __vm_vcpu_add(vm, vcpu_id);
tools/testing/selftests/kvm/lib/x86_64/tdx.h new file +86 -0

86 lines not on this page · raw message

tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c new file +338 -0

338 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#5 lore raw

RFC PATCH v2 3/17 KVM: selftest: Adding TDX life cycle test.

In reply to Sagi Shahar

From: Erdem Aktas <[email protected]>

Adding a test to verify TDX lifecycle by creating a TD and running a dummy TDVMCALL<INSTRUCTION.IO> inside it.

---
 tools/testing/selftests/kvm/Makefile          |   1 +
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  | 149 ++++++++++++++++++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 104 ++++++++++++
 3 files changed, 254 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c
tools/testing/selftests/kvm/Makefile +1 -0
index ad4d60dadc06..208e0cc30048 100644
@@ -140,6 +140,7 @@ TEST_GEN_PROGS_x86_64 += set_memory_region_test
 TEST_GEN_PROGS_x86_64 += steal_time
 TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
 TEST_GEN_PROGS_x86_64 += system_counter_offset_test
+TEST_GEN_PROGS_x86_64 += x86_64/tdx_vm_tests
 
 # Compiled outputs used by test targets
 TEST_GEN_PROGS_EXTENDED_x86_64 += x86_64/nx_huge_pages_test
tools/testing/selftests/kvm/lib/x86_64/tdx.h +149 -0

159 lines not on this page · raw message

tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c new file +104 -0

104 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#8 lore raw

RFC PATCH v2 4/17 KVM: selftest: TDX: Add report_fatal_error test

In reply to Sagi Shahar

The test checks report_fatal_error functionality.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  | 17 ++++++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 55 +++++++++++++++++++
 2 files changed, 72 insertions(+)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +17 -0
index d5de52657112..351ece3e80e2 100644
@@ -51,6 +51,7 @@
 #define _PAGE_RW            (1UL<<1)       /* writeable */
 #define _PAGE_PS            (1UL<<7)       /* page size bit*/
 
+#define TDX_REPORT_FATAL_ERROR 0x10003
 #define TDX_INSTRUCTION_IO 30
 
 #define TDX_SUCCESS_PORT 0x30
@@ -212,6 +213,22 @@ static inline void tdvmcall_success(void)
 	tdvmcall_io(TDX_SUCCESS_PORT, /*size=*/4, TDX_IO_WRITE, &code);
 }
 
+/*
+ * Report an error to user space.
+ * data_gpa may point to an optional shared guest memory holding the error string.
+ * Return value from tdvmcall is ignored since execution is not expected to
+ * continue beyond this point.
+ */
+static inline void tdvmcall_fatal(uint64_t error_code)
+{
+	struct kvm_regs regs;
+
+	memset(&regs, 0, sizeof(regs));
+	regs.r11 = TDX_REPORT_FATAL_ERROR;
+	regs.r12 = error_code;
+	regs.rcx = 0x1C00;
+	tdcall(&regs);
+}
 
 #define TDX_FUNCTION_SIZE(name) ((uint64_t)&__stop_sec_ ## name -\
 			   (uint64_t)&__start_sec_ ## name) \
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +55 -0

67 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#9 lore raw

RFC PATCH v2 5/17 KVM: selftest: Adding test case for TDX port IO

In reply to Sagi Shahar

From: Erdem Aktas <[email protected]>

Verifies TDVMCALL<INSTRUCTION.IO> READ and WRITE operations.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |   1 +
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 108 ++++++++++++++++++
 2 files changed, 109 insertions(+)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +1 -0
index 351ece3e80e2..c78dba1af14f 100644
@@ -55,6 +55,7 @@
 #define TDX_INSTRUCTION_IO 30
 
 #define TDX_SUCCESS_PORT 0x30
+#define TDX_TEST_PORT 0x31
 #define TDX_IO_READ 0
 #define TDX_IO_WRITE 1
 
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +108 -0

126 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#11 lore raw

RFC PATCH v2 7/17 KVM: selftest: TDX: Add basic get_td_vmcall_info test

In reply to Sagi Shahar

The test calls get_td_vmcall_info from the guest and verifies the expected returned values.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  43 +++++++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 107 ++++++++++++++++++
 2 files changed, 150 insertions(+)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +43 -0
index a28d15417d3e..39b000118e26 100644
@@ -51,6 +51,7 @@
 #define _PAGE_RW            (1UL<<1)       /* writeable */
 #define _PAGE_PS            (1UL<<7)       /* page size bit*/
 
+#define TDX_GET_TD_VM_CALL_INFO 0x10000
 #define TDX_REPORT_FATAL_ERROR 0x10003
 #define TDX_INSTRUCTION_IO 30
 
@@ -232,6 +233,28 @@ static inline void tdvmcall_fatal(uint64_t error_code)
 	tdcall(&regs);
 }
 
+/*
+ * Get td vmcall info.
+ * Used to help request the host VMM enumerate which TDG.VP.VMCALLs are supported.
+ * Returns return in r10 code and leaf-specific output in r11-r14.
+ */
+static inline uint64_t tdvmcall_get_td_vmcall_info(uint64_t *r11, uint64_t *r12,
+						   uint64_t *r13, uint64_t *r14)
+{
+	struct kvm_regs regs;
+
+	memset(&regs, 0, sizeof(regs));
+	regs.r11 = TDX_GET_TD_VM_CALL_INFO;
+	regs.r12 = 0;
+	regs.rcx = 0x1C00;
+	tdcall(&regs);
+	*r11 = regs.r11;
+	*r12 = regs.r12;
+	*r13 = regs.r13;
+	*r14 = regs.r14;
+	return regs.r10;
+}
+
 /*
  * Reports a 32 bit value from the guest to user space using a TDVM IO call.
  * Data is reported on port TDX_DATA_REPORT_PORT.
@@ -244,6 +267,26 @@ static inline uint64_t tdvm_report_to_user_space(uint32_t data)
 	return tdvmcall_io(TDX_DATA_REPORT_PORT, /*size=*/4, TDX_IO_WRITE, &data_64);
 }
 
+/*
+ * Reports a 64 bit value from the guest to user space using a TDVM IO call.
+ * Data is reported on port TDX_DATA_REPORT_PORT.
+ * Data is sent to host in 2 calls. LSB is sent (and needs to be read) first.
+ */
+static inline uint64_t tdvm_report_64bit_to_user_space(uint64_t data)
+{
+	uint64_t err;
+	uint64_t data_lo = data & 0xFFFFFFFF;
+	uint64_t data_hi = (data >> 32) & 0xFFFFFFFF;
+
+	err = tdvmcall_io(TDX_DATA_REPORT_PORT, /*size=*/4, TDX_IO_WRITE,
+			  &data_lo);
+	if (err)
+		return err;
+
+	return tdvmcall_io(TDX_DATA_REPORT_PORT, /*size=*/4, TDX_IO_WRITE,
+			   &data_hi);
+}
+
 #define TDX_FUNCTION_SIZE(name) ((uint64_t)&__stop_sec_ ## name -\
 			   (uint64_t)&__start_sec_ ## name) \
 
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +107 -0

125 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#10 lore raw

RFC PATCH v2 6/17 KVM: selftest: TDX: Add basic TDX CPUID test

In reply to Sagi Shahar

The test reads CPUID values from inside a TD VM and compare them to expected values.

The test targets CPUID values which are virtualized as "As Configured", "As Configured (if Native)", "Calculated", "Fixed" and "Native" according to the TDX spec.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  13 ++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 132 ++++++++++++++++++
 2 files changed, 145 insertions(+)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +13 -0
index c78dba1af14f..a28d15417d3e 100644
@@ -56,6 +56,7 @@
 
 #define TDX_SUCCESS_PORT 0x30
 #define TDX_TEST_PORT 0x31
+#define TDX_DATA_REPORT_PORT 0x32
 #define TDX_IO_READ 0
 #define TDX_IO_WRITE 1
 
@@ -231,6 +232,18 @@ static inline void tdvmcall_fatal(uint64_t error_code)
 	tdcall(&regs);
 }
 
+/*
+ * Reports a 32 bit value from the guest to user space using a TDVM IO call.
+ * Data is reported on port TDX_DATA_REPORT_PORT.
+ */
+static inline uint64_t tdvm_report_to_user_space(uint32_t data)
+{
+	// Need to upcast data to match tdvmcall_io signature.
+	uint64_t data_64 = data;
+
+	return tdvmcall_io(TDX_DATA_REPORT_PORT, /*size=*/4, TDX_IO_WRITE, &data_64);
+}
+
 #define TDX_FUNCTION_SIZE(name) ((uint64_t)&__stop_sec_ ## name -\
 			   (uint64_t)&__start_sec_ ## name) \
 
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +132 -0

150 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#12 lore raw

RFC PATCH v2 8/17 KVM: selftest: TDX: Add TDX IO writes test

In reply to Sagi Shahar

The test verifies IO writes of various sizes from the guest to the host.

Google-Bug-Id: 235407183

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  3 +
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 85 +++++++++++++++++++
 2 files changed, 88 insertions(+)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +3 -0
index 39b000118e26..f1f44c2ad40e 100644
@@ -51,6 +51,9 @@
 #define _PAGE_RW            (1UL<<1)       /* writeable */
 #define _PAGE_PS            (1UL<<7)       /* page size bit*/
 
+#define TDX_VMCALL_SUCCESS 0x0000000000000000
+#define TDX_VMCALL_INVALID_OPERAND 0x8000000000000000
+
 #define TDX_GET_TD_VM_CALL_INFO 0x10000
 #define TDX_REPORT_FATAL_ERROR 0x10003
 #define TDX_INSTRUCTION_IO 30
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +85 -0

97 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#13 lore raw

RFC PATCH v2 9/17 KVM: selftest: TDX: Add TDX IO reads test

In reply to Sagi Shahar

The test verifies IO reads of various sizes from the host to the guest.

---
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 84 +++++++++++++++++++
 1 file changed, 84 insertions(+)
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +84 -0
index ee60f77fe38e..85b5ab99424e 100644
@@ -1,5 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0-only
 
+#include "asm/kvm.h"
+#include <bits/stdint-uintn.h>
 #include <fcntl.h>
 #include <limits.h>
 #include <kvm_util.h>
@@ -573,6 +575,87 @@ void verify_guest_writes(void)
 	printf("\t ... PASSED\n");
 }
 
+/*
+ * Verifies IO functionality by reading values of different sizes
+ * from the host.
+ */
+TDX_GUEST_FUNCTION(guest_io_reads)
+{
+	uint64_t data;
+	uint64_t ret;
+
+	ret = tdvmcall_io(TDX_TEST_PORT, 1, TDX_IO_READ, &data);
+	if (ret)
+		tdvmcall_fatal(ret);
+	if (data != 0xAB)
+		tdvmcall_fatal(1);
+
+	ret = tdvmcall_io(TDX_TEST_PORT, 2, TDX_IO_READ, &data);
+	if (ret)
+		tdvmcall_fatal(ret);
+	if (data != 0xABCD)
+		tdvmcall_fatal(2);
+
+	ret = tdvmcall_io(TDX_TEST_PORT, 4, TDX_IO_READ, &data);
+	if (ret)
+		tdvmcall_fatal(ret);
+	if (data != 0xFFABCDEF)
+		tdvmcall_fatal(4);
+
+	// Read an invalid number of bytes.
+	ret = tdvmcall_io(TDX_TEST_PORT, 5, TDX_IO_READ, &data);
+	if (ret)
+		tdvmcall_fatal(ret);
+
+	tdvmcall_success();
+}
+
+void verify_guest_reads(void)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	printf("Verifying guest reads:\n");
+	/* Create a TD VM with no memory.*/
+	vm = vm_create_tdx();
+
+	/* Allocate TD guest memory and initialize the TD.*/
+	initialize_td(vm);
+
+	/* Initialize the TD vcpu and copy the test code to the guest memory.*/
+	vcpu = vm_vcpu_add_tdx(vm, 0);
+
+	/* Setup and initialize VM memory */
+	prepare_source_image(vm, guest_io_reads,
+			     TDX_FUNCTION_SIZE(guest_io_reads), 0);
+	finalize_td_memory(vm);
+
+	vcpu_run(vcpu);
+	CHECK_GUEST_FAILURE(vcpu);
+	CHECK_IO(vcpu, TDX_TEST_PORT, 1, TDX_IO_READ);
+	*(uint8_t *)((void *)vcpu->run + vcpu->run->io.data_offset) = 0xAB;
+
+	vcpu_run(vcpu);
+	CHECK_GUEST_FAILURE(vcpu);
+	CHECK_IO(vcpu, TDX_TEST_PORT, 2, TDX_IO_READ);
+	*(uint16_t *)((void *)vcpu->run + vcpu->run->io.data_offset) = 0xABCD;
+
+	vcpu_run(vcpu);
+	CHECK_GUEST_FAILURE(vcpu);
+	CHECK_IO(vcpu, TDX_TEST_PORT, 4, TDX_IO_READ);
+	*(uint32_t *)((void *)vcpu->run + vcpu->run->io.data_offset) = 0xFFABCDEF;
+
+	vcpu_run(vcpu);
+	ASSERT_EQ(vcpu->run->exit_reason, KVM_EXIT_SYSTEM_EVENT);
+	ASSERT_EQ(vcpu->run->system_event.data[1], TDX_VMCALL_INVALID_OPERAND);
+
+	vcpu_run(vcpu);
+	CHECK_GUEST_COMPLETION(vcpu);
+
+	kvm_vm_free(vm);
+	printf("\t ... PASSED\n");
+}
+
 int main(int argc, char **argv)
 {
 	if (!is_tdx_enabled()) {
@@ -586,6 +669,7 @@ int main(int argc, char **argv)
 	run_in_new_process(&verify_td_cpuid);
 	run_in_new_process(&verify_get_td_vmcall_info);
 	run_in_new_process(&verify_guest_writes);
+	run_in_new_process(&verify_guest_reads);
 
 	return 0;
 }
Signature · 1 line
2.37.2.789.g6183377224-goog
#14 lore raw

RFC PATCH v2 10/17 KVM: selftest: TDX: Add TDX MSR read/write tests

In reply to Sagi Shahar

The test verifies reads and writes for MSR registers with different access level.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  34 +++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 229 ++++++++++++++++++
 2 files changed, 263 insertions(+)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +34 -0
index f1f44c2ad40e..263834979727 100644
@@ -57,6 +57,8 @@
 #define TDX_GET_TD_VM_CALL_INFO 0x10000
 #define TDX_REPORT_FATAL_ERROR 0x10003
 #define TDX_INSTRUCTION_IO 30
+#define TDX_INSTRUCTION_RDMSR 31
+#define TDX_INSTRUCTION_WRMSR 32
 
 #define TDX_SUCCESS_PORT 0x30
 #define TDX_TEST_PORT 0x31
@@ -258,6 +260,38 @@ static inline uint64_t tdvmcall_get_td_vmcall_info(uint64_t *r11, uint64_t *r12,
 	return regs.r10;
 }
 
+/*
+ * Read MSR register.
+ */
+static inline uint64_t tdvmcall_rdmsr(uint64_t index, uint64_t *ret_value)
+{
+	struct kvm_regs regs;
+
+	memset(&regs, 0, sizeof(regs));
+	regs.r11 = TDX_INSTRUCTION_RDMSR;
+	regs.r12 = index;
+	regs.rcx = 0x1C00;
+	tdcall(&regs);
+	*ret_value = regs.r11;
+	return regs.r10;
+}
+
+/*
+ * Write MSR register.
+ */
+static inline uint64_t tdvmcall_wrmsr(uint64_t index, uint64_t value)
+{
+	struct kvm_regs regs;
+
+	memset(&regs, 0, sizeof(regs));
+	regs.r11 = TDX_INSTRUCTION_WRMSR;
+	regs.r12 = index;
+	regs.r13 = value;
+	regs.rcx = 0x3C00;
+	tdcall(&regs);
+	return regs.r10;
+}
+
 /*
  * Reports a 32 bit value from the guest to user space using a TDVM IO call.
  * Data is reported on port TDX_DATA_REPORT_PORT.
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +229 -0

247 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#15 lore raw

RFC PATCH v2 11/17 KVM: selftest: TDX: Add TDX HLT exit test

In reply to Sagi Shahar

The test verifies that the guest runs TDVMCALL<INSTRUCTION.HLT> and the guest vCPU enters to the halted state.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  | 16 ++++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 88 +++++++++++++++++++
 2 files changed, 104 insertions(+)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +16 -0
index 263834979727..b11200028546 100644
@@ -56,6 +56,7 @@
 
 #define TDX_GET_TD_VM_CALL_INFO 0x10000
 #define TDX_REPORT_FATAL_ERROR 0x10003
+#define TDX_INSTRUCTION_HLT 12
 #define TDX_INSTRUCTION_IO 30
 #define TDX_INSTRUCTION_RDMSR 31
 #define TDX_INSTRUCTION_WRMSR 32
@@ -292,6 +293,21 @@ static inline uint64_t tdvmcall_wrmsr(uint64_t index, uint64_t value)
 	return regs.r10;
 }
 
+/*
+ * Execute HLT instruction.
+ */
+static inline uint64_t tdvmcall_hlt(uint64_t interrupt_blocked_flag)
+{
+	struct kvm_regs regs;
+
+	memset(&regs, 0, sizeof(regs));
+	regs.r11 = TDX_INSTRUCTION_HLT;
+	regs.r12 = interrupt_blocked_flag;
+	regs.rcx = 0x1C00;
+	tdcall(&regs);
+	return regs.r10;
+}
+
 /*
  * Reports a 32 bit value from the guest to user space using a TDVM IO call.
  * Data is reported on port TDX_DATA_REPORT_PORT.
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +88 -0

100 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#16 lore raw

RFC PATCH v2 12/17 KVM: selftest: TDX: Add TDX MMIO reads test

In reply to Sagi Shahar

The test verifies MMIO reads of various sizes from the host to the guest.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  21 ++++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 113 ++++++++++++++++++
 2 files changed, 134 insertions(+)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +21 -0
index b11200028546..7045d617dd78 100644
@@ -60,12 +60,15 @@
 #define TDX_INSTRUCTION_IO 30
 #define TDX_INSTRUCTION_RDMSR 31
 #define TDX_INSTRUCTION_WRMSR 32
+#define TDX_INSTRUCTION_VE_REQUEST_MMIO 48
 
 #define TDX_SUCCESS_PORT 0x30
 #define TDX_TEST_PORT 0x31
 #define TDX_DATA_REPORT_PORT 0x32
 #define TDX_IO_READ 0
 #define TDX_IO_WRITE 1
+#define TDX_MMIO_READ 0
+#define TDX_MMIO_WRITE 1
 
 #define GDT_ENTRY(flags, base, limit)				\
 		((((base)  & 0xff000000ULL) << (56-24)) |	\
@@ -308,6 +311,24 @@ static inline uint64_t tdvmcall_hlt(uint64_t interrupt_blocked_flag)
 	return regs.r10;
 }
 
+/*
+ * Execute MMIO request instruction for read.
+ */
+static inline uint64_t tdvmcall_mmio_read(uint64_t address, uint64_t size, uint64_t *data_out)
+{
+	struct kvm_regs regs;
+
+	memset(&regs, 0, sizeof(regs));
+	regs.r11 = TDX_INSTRUCTION_VE_REQUEST_MMIO;
+	regs.r12 = size;
+	regs.r13 = TDX_MMIO_READ;
+	regs.r14 = address;
+	regs.rcx = 0x7C00;
+	tdcall(&regs);
+	*data_out = regs.r11;
+	return regs.r10;
+}
+
 /*
  * Reports a 32 bit value from the guest to user space using a TDVM IO call.
  * Data is reported on port TDX_DATA_REPORT_PORT.
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +113 -0

143 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#18 lore raw

RFC PATCH v2 14/17 KVM: selftest: TDX: Add TDX CPUID TDVMCALL test

In reply to Sagi Shahar

This test issues a CPUID TDVMCALL from inside the guest to get the CPUID values as seen by KVM.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  23 ++++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 102 ++++++++++++++++++
 2 files changed, 125 insertions(+)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +23 -0
index 17e3649e5729..3729543a05a3 100644
@@ -56,6 +56,7 @@
 
 #define TDX_GET_TD_VM_CALL_INFO 0x10000
 #define TDX_REPORT_FATAL_ERROR 0x10003
+#define TDX_INSTRUCTION_CPUID 10
 #define TDX_INSTRUCTION_HLT 12
 #define TDX_INSTRUCTION_IO 30
 #define TDX_INSTRUCTION_RDMSR 31
@@ -347,6 +348,28 @@ static inline uint64_t tdvmcall_mmio_write(uint64_t address, uint64_t size, uint
 	return regs.r10;
 }
 
+/*
+ * Execute CPUID instruction.
+ */
+static inline uint64_t tdvmcall_cpuid(uint32_t eax, uint32_t ecx,
+				      uint32_t *ret_eax, uint32_t *ret_ebx,
+				      uint32_t *ret_ecx, uint32_t *ret_edx)
+{
+	struct kvm_regs regs;
+
+	memset(&regs, 0, sizeof(regs));
+	regs.r11 = TDX_INSTRUCTION_CPUID;
+	regs.r12 = eax;
+	regs.r13 = ecx;
+	regs.rcx = 0xFC00;
+	tdcall(&regs);
+	*ret_eax = regs.r12;
+	*ret_ebx = regs.r13;
+	*ret_ecx = regs.r14;
+	*ret_edx = regs.r15;
+	return regs.r10;
+}
+
 /*
  * Reports a 32 bit value from the guest to user space using a TDVM IO call.
  * Data is reported on port TDX_DATA_REPORT_PORT.
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +102 -0

114 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#17 lore raw

RFC PATCH v2 13/17 KVM: selftest: TDX: Add TDX MMIO writes test

In reply to Sagi Shahar

The test verifies MMIO writes of various sizes from the guest to the host.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  | 18 ++++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 92 +++++++++++++++++++
 2 files changed, 110 insertions(+)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +18 -0
index 7045d617dd78..17e3649e5729 100644
@@ -329,6 +329,24 @@ static inline uint64_t tdvmcall_mmio_read(uint64_t address, uint64_t size, uint6
 	return regs.r10;
 }
 
+/*
+ * Execute MMIO request instruction for write.
+ */
+static inline uint64_t tdvmcall_mmio_write(uint64_t address, uint64_t size, uint64_t data_in)
+{
+	struct kvm_regs regs;
+
+	memset(&regs, 0, sizeof(regs));
+	regs.r11 = TDX_INSTRUCTION_VE_REQUEST_MMIO;
+	regs.r12 = size;
+	regs.r13 = TDX_MMIO_WRITE;
+	regs.r14 = address;
+	regs.r15 = data_in;
+	regs.rcx = 0xFC00;
+	tdcall(&regs);
+	return regs.r10;
+}
+
 /*
  * Reports a 32 bit value from the guest to user space using a TDVM IO call.
  * Data is reported on port TDX_DATA_REPORT_PORT.
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +92 -0

104 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#21 lore raw

RFC PATCH v2 17/17 KVM: selftest: TDX: Add shared memory test

In reply to Sagi Shahar

From: Ryan Afranji <[email protected]>

Adds a test that sets up shared memory between the host and guest.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  38 +++++-
 .../selftests/kvm/lib/x86_64/tdx_lib.c        |  41 +++++-
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 124 ++++++++++++++++++
 3 files changed, 192 insertions(+), 11 deletions(-)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +34 -4
index 7af2d189043f..be8564f4672d 100644
@@ -23,10 +23,21 @@
 
 /*
  * Max Page Table Size
- * To map 4GB memory region with 2MB pages, there needs to be 1 page for PML4,
- * 1 Page for PDPT, 4 pages for PD. Reserving 6 pages for PT.
+ * To map 4GB memory regions for each private and shared memory with 2MB pages,
+ * there needs to be 1 page for PML4, 1 Page for PDPT, 8 pages for PD. Reserving
+ * 10 pages for PT.
  */
-#define TDX_GUEST_NR_PT_PAGES (1 + 1 + 4)
+#define TDX_GUEST_NR_PT_PAGES (1 + 1 + 8)
+
+/*
+ * Guest Virtual Address Shared Bit
+ * TDX's shared bit is defined as the highest order bit in the GPA. Since the
+ * highest order bit allowed in the GPA may exceed the GVA's, a 1:1 mapping
+ * cannot be applied for shared memory. This value is a bit within the range
+ * [32 - 38] (0-indexed) that will designate a 4 GB region of GVAs that map the
+ * shared GPAs. This approach does not increase number of PML4 and PDPT pages.
+ */
+#define TDX_GUEST_VIRT_SHARED_BIT 32
 
 /*
  * Predefined GDTR values.
@@ -60,6 +71,7 @@
 #define TDX_VMCALL_INVALID_OPERAND 0x8000000000000000
 
 #define TDX_GET_TD_VM_CALL_INFO 0x10000
+#define TDX_MAP_GPA 0x10001
 #define TDX_REPORT_FATAL_ERROR 0x10003
 #define TDX_INSTRUCTION_CPUID 10
 #define TDX_INSTRUCTION_HLT 12
@@ -101,7 +113,7 @@ struct __packed tdx_gdtr {
 struct page_table {
 	uint64_t  pml4[512];
 	uint64_t  pdpt[512];
-	uint64_t  pd[4][512];
+	uint64_t  pd[8][512];
 };
 
 void add_td_memory(struct kvm_vm *vm, void *source_page,
@@ -411,6 +423,24 @@ static inline uint64_t tdcall_vp_info(uint64_t *rcx, uint64_t *rdx,
 	return regs.rax;
 }
 
+/*
+ * Execute MapGPA instruction.
+ */
+static inline uint64_t tdvmcall_map_gpa(uint64_t address, uint64_t size,
+					uint64_t *data_out)
+{
+	struct kvm_regs regs;
+
+	memset(&regs, 0, sizeof(regs));
+	regs.r11 = TDX_MAP_GPA;
+	regs.r12 = address;
+	regs.r13 = size;
+	regs.rcx = 0x3C00;
+	tdcall(&regs);
+	*data_out = regs.r11;
+	return regs.r10;
+}
+
 /*
  * Reports a 32 bit value from the guest to user space using a TDVM IO call.
  * Data is reported on port TDX_DATA_REPORT_PORT.
tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c +34 -7

70 lines not on this page · raw message

tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +124 -0

136 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#19 lore raw

RFC PATCH v2 15/17 KVM: selftest: TDX: Verify the behavior when host consumes a TD private memory

In reply to Sagi Shahar

From: Ryan Afranji <[email protected]>

The test checks that host can only read fixed values when trying to access the guest's private memory.

---
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 93 +++++++++++++++++++
 1 file changed, 93 insertions(+)
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +93 -0
index 934f2f7a5df9..1776b39b7d9e 100644
@@ -1274,6 +1274,98 @@ void verify_mmio_writes(void)
 	printf("\t ... PASSED\n");
 }
 
+TDX_GUEST_FUNCTION(guest_host_read_priv_mem)
+{
+	uint64_t guest_var = 0xABCD;
+	uint64_t ret;
+
+	/* Sends address to host. */
+	ret = tdvm_report_64bit_to_user_space((uint64_t)&guest_var);
+	if (ret)
+		tdvmcall_fatal(ret);
+
+	/* Update guest_var's value and have host reread it. */
+	guest_var = 0xFEDC;
+
+	tdvmcall_success();
+}
+
+void verify_host_reading_private_mem(void)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	struct userspace_mem_region *region;
+	uint64_t guest_var_addr;
+	uint64_t host_virt;
+	uint64_t first_host_read;
+	uint64_t second_host_read;
+	int ctr;
+
+	printf("Verifying host's behavior when reading TD private memory:\n");
+	/* Create a TD VM with no memory. */
+	vm = vm_create_tdx();
+
+	/* Allocate TD guest memory and initialize the TD. */
+	initialize_td(vm);
+
+	/* Initialize the TD vcpu and copy the test code to the guest memory. */
+	vcpu = vm_vcpu_add_tdx(vm, 0);
+
+	/* Setup and initialize VM memory. */
+	prepare_source_image(vm, guest_host_read_priv_mem,
+			     TDX_FUNCTION_SIZE(guest_host_read_priv_mem), 0);
+	finalize_td_memory(vm);
+
+	/* Get the address of the guest's variable. */
+	vcpu_run(vcpu);
+	CHECK_GUEST_FAILURE(vcpu);
+	printf("\t ... Guest's variable contains 0xABCD\n");
+
+	/* Guest virtual and guest physical addresses have 1:1 mapping. */
+	guest_var_addr = read_64bit_from_guest(vcpu, TDX_DATA_REPORT_PORT);
+
+	/* Search for the guest's address in guest's memory regions. */
+	host_virt = 0;
+	hash_for_each(vm->regions.slot_hash, ctr, region, slot_node) {
+		uint64_t offset;
+		uint64_t host_virt_base;
+		uint64_t guest_base;
+
+		guest_base = (uint64_t)region->region.guest_phys_addr;
+		offset = guest_var_addr - guest_base;
+
+		if (guest_base <= guest_var_addr &&
+		    offset <= region->region.memory_size) {
+			host_virt_base = (uint64_t)region->host_mem;
+			host_virt = host_virt_base + offset;
+			break;
+		}
+	}
+	TEST_ASSERT(host_virt != 0,
+		    "Guest address not found in guest memory regions\n");
+
+	/* Host reads guest's variable. */
+	first_host_read = *(uint64_t *)host_virt;
+	printf("\t ... Host's read attempt value: %lu\n", first_host_read);
+
+	/* Guest updates variable and host rereads it. */
+	vcpu_run(vcpu);
+	CHECK_GUEST_FAILURE(vcpu);
+	printf("\t ... Guest's variable updated to 0xFEDC\n");
+
+	second_host_read = *(uint64_t *)host_virt;
+	printf("\t ... Host's second read attempt value: %lu\n",
+	       second_host_read);
+
+	TEST_ASSERT(first_host_read == second_host_read,
+		    "Host did not read a fixed pattern\n");
+
+	printf("\t ... Fixed pattern was returned to the host\n");
+
+	kvm_vm_free(vm);
+	printf("\t ... PASSED\n");
+}
+
 int main(int argc, char **argv)
 {
 	if (!is_tdx_enabled()) {
@@ -1294,6 +1386,7 @@ int main(int argc, char **argv)
 	run_in_new_process(&verify_guest_hlt);
 	run_in_new_process(&verify_mmio_reads);
 	run_in_new_process(&verify_mmio_writes);
+	run_in_new_process(&verify_host_reading_private_mem);
 
 	return 0;
 }
Signature · 1 line
2.37.2.789.g6183377224-goog
#20 lore raw

RFC PATCH v2 16/17 KVM: selftest: TDX: Add TDG.VP.INFO test

In reply to Sagi Shahar

From: Roger Wang <[email protected]>

Adds a test for TDG.VP.INFO

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  | 103 ++++++++----
 .../selftests/kvm/lib/x86_64/tdx_lib.c        |  18 ++-
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 150 ++++++++++++++++++
 3 files changed, 235 insertions(+), 36 deletions(-)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +72 -31
index 3729543a05a3..7af2d189043f 100644
@@ -10,6 +10,11 @@
  */
 #define TDX_GUEST_MAX_NR_PAGES 10000
 
+/*
+ * Max number of vCPUs for the guest VM
+ */
+ #define TDX_GUEST_MAX_NUM_VCPUS 3
+
 /*
  * Page Table Address used when paging is enabled.
  */
@@ -71,6 +76,11 @@
 #define TDX_MMIO_READ 0
 #define TDX_MMIO_WRITE 1
 
+#define TDX_TDCALL_INFO   1
+
+#define TDX_TDPARAM_ATTR_SEPT_VE_DISABLE_BIT	(1UL << 28)
+#define TDX_TDPARAM_ATTR_PKS_BIT		(1UL << 30)
+
 #define GDT_ENTRY(flags, base, limit)				\
 		((((base)  & 0xff000000ULL) << (56-24)) |	\
 		 (((flags) & 0x0000f0ffULL) << 40) |		\
@@ -98,6 +108,7 @@ void add_td_memory(struct kvm_vm *vm, void *source_page,
 		   uint64_t gpa, int size);
 void finalize_td_memory(struct kvm_vm *vm);
 void initialize_td(struct kvm_vm *vm);
+void initialize_td_with_attributes(struct kvm_vm *vm, uint64_t attributes);
 void initialize_td_vcpu(struct kvm_vcpu *vcpu);
 void prepare_source_image(struct kvm_vm *vm, void *guest_code,
 			  size_t guest_code_size,
@@ -116,40 +127,41 @@ void prepare_source_image(struct kvm_vm *vm, void *guest_code,
 static inline void tdcall(struct kvm_regs *regs)
 {
 	asm volatile (
-			"mov %13, %%rax;\n\t"
-			"mov %14, %%rbx;\n\t"
-			"mov %15, %%rcx;\n\t"
-			"mov %16, %%rdx;\n\t"
-			"mov %17, %%r8;\n\t"
-			"mov %18, %%r9;\n\t"
-			"mov %19, %%r10;\n\t"
-			"mov %20, %%r11;\n\t"
-			"mov %21, %%r12;\n\t"
-			"mov %22, %%r13;\n\t"
-			"mov %23, %%r14;\n\t"
-			"mov %24, %%r15;\n\t"
-			"mov %25, %%rbp;\n\t"
-			"mov %26, %%rsi;\n\t"
-			"mov %27, %%rdi;\n\t"
+			"mov %14, %%rax;\n\t"
+			"mov %15, %%rbx;\n\t"
+			"mov %16, %%rcx;\n\t"
+			"mov %17, %%rdx;\n\t"
+			"mov %18, %%r8;\n\t"
+			"mov %19, %%r9;\n\t"
+			"mov %20, %%r10;\n\t"
+			"mov %21, %%r11;\n\t"
+			"mov %22, %%r12;\n\t"
+			"mov %23, %%r13;\n\t"
+			"mov %24, %%r14;\n\t"
+			"mov %25, %%r15;\n\t"
+			"mov %26, %%rbp;\n\t"
+			"mov %27, %%rsi;\n\t"
+			"mov %28, %%rdi;\n\t"
 			".byte 0x66, 0x0F, 0x01, 0xCC;\n\t"
 			"mov %%rax, %0;\n\t"
 			"mov %%rbx, %1;\n\t"
-			"mov %%rdx, %2;\n\t"
-			"mov %%r8, %3;\n\t"
-			"mov %%r9, %4;\n\t"
-			"mov %%r10, %5;\n\t"
-			"mov %%r11, %6;\n\t"
-			"mov %%r12, %7;\n\t"
-			"mov %%r13, %8;\n\t"
-			"mov %%r14, %9;\n\t"
-			"mov %%r15, %10;\n\t"
-			"mov %%rsi, %11;\n\t"
-			"mov %%rdi, %12;\n\t"
-			: "=m" (regs->rax), "=m" (regs->rbx), "=m" (regs->rdx),
-			"=m" (regs->r8), "=m" (regs->r9), "=m" (regs->r10),
-			"=m" (regs->r11), "=m" (regs->r12), "=m" (regs->r13),
-			"=m" (regs->r14), "=m" (regs->r15), "=m" (regs->rsi),
-			"=m" (regs->rdi)
+			"mov %%rcx, %2;\n\t"
+			"mov %%rdx, %3;\n\t"
+			"mov %%r8, %4;\n\t"
+			"mov %%r9, %5;\n\t"
+			"mov %%r10, %6;\n\t"
+			"mov %%r11, %7;\n\t"
+			"mov %%r12, %8;\n\t"
+			"mov %%r13, %9;\n\t"
+			"mov %%r14, %10;\n\t"
+			"mov %%r15, %11;\n\t"
+			"mov %%rsi, %12;\n\t"
+			"mov %%rdi, %13;\n\t"
+			: "=m" (regs->rax), "=m" (regs->rbx), "=m" (regs->rcx),
+			"=m" (regs->rdx), "=m" (regs->r8), "=m" (regs->r9),
+			"=m" (regs->r10), "=m" (regs->r11), "=m" (regs->r12),
+			"=m" (regs->r13), "=m" (regs->r14), "=m" (regs->r15),
+			"=m" (regs->rsi), "=m" (regs->rdi)
 			: "m" (regs->rax), "m" (regs->rbx), "m" (regs->rcx),
 			"m" (regs->rdx), "m" (regs->r8), "m" (regs->r9),
 			"m" (regs->r10), "m" (regs->r11), "m" (regs->r12),
@@ -370,6 +382,35 @@ static inline uint64_t tdvmcall_cpuid(uint32_t eax, uint32_t ecx,
 	return regs.r10;
 }
 
+/*
+ * Execute TDG.VP.INFO instruction.
+ */
+static inline uint64_t tdcall_vp_info(uint64_t *rcx, uint64_t *rdx,
+				      uint64_t *r8, uint64_t *r9,
+				      uint64_t *r10, uint64_t *r11)
+{
+	struct kvm_regs regs;
+
+	memset(&regs, 0, sizeof(regs));
+	regs.rax = TDX_TDCALL_INFO;
+	tdcall(&regs);
+
+	if (rcx)
+		*rcx = regs.rcx;
+	if (rdx)
+		*rdx = regs.rdx;
+	if (r8)
+		*r8 = regs.r8;
+	if (r9)
+		*r9 = regs.r9;
+	if (r10)
+		*r10 = regs.r10;
+	if (r11)
+		*r11 = regs.r11;
+
+	return regs.rax;
+}
+
 /*
  * Reports a 32 bit value from the guest to user space using a TDVM IO call.
  * Data is reported on port TDX_DATA_REPORT_PORT.
tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c +13 -5
index 72bf2ff24a29..dc9a44ae4064 100644
@@ -78,10 +78,10 @@ static struct tdx_cpuid_data get_tdx_cpuid_data(struct kvm_vm *vm)
 }
 
 /*
- * Initialize a VM as a TD.
+ * Initialize a VM as a TD with attributes.
  *
  */
-void initialize_td(struct kvm_vm *vm)
+void initialize_td_with_attributes(struct kvm_vm *vm, uint64_t attributes)
 {
 	struct tdx_cpuid_data cpuid_data;
 	int rc;
@@ -99,7 +99,7 @@ void initialize_td(struct kvm_vm *vm)
 			      KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK);
 	vm_enable_cap(vm, KVM_CAP_SPLIT_IRQCHIP, 24);
 
-	/* Allocate and setup memoryfor the td guest. */
+	/* Allocate and setup memory for the td guest. */
 	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,
 				    TDX_GUEST_PT_FIXED_ADDR,
 				    0, TDX_GUEST_MAX_NR_PAGES, 0);
@@ -108,12 +108,20 @@ void initialize_td(struct kvm_vm *vm)
 
 	cpuid_data = get_tdx_cpuid_data(vm);
 
-	init_vm.max_vcpus = 1;
-	init_vm.attributes = 0;
+	init_vm.max_vcpus = TDX_GUEST_MAX_NUM_VCPUS;
+	init_vm.attributes = attributes;
 	memcpy(&init_vm.cpuid, &cpuid_data, sizeof(cpuid_data));
 	tdx_ioctl(vm->fd, KVM_TDX_INIT_VM, 0, &init_vm);
 }
 
+/*
+ * Initialize a VM as a TD with no attributes.
+ *
+ */
+void initialize_td(struct kvm_vm *vm)
+{
+	initialize_td_with_attributes(vm, 0);
+}
 
 void initialize_td_vcpu(struct kvm_vcpu *vcpu)
 {
tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +150 -0

168 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#6 lore raw

Re: [RFC PATCH v2 03/17] KVM: selftest: Adding TDX life cycle test.

In reply to Sagi Shahar
Quoted 66 lines · Sagi Shahar
tools/testing/selftests/kvm/lib/x86_64/tdx.h +51 -0
index 61b997dfc420..d5de52657112 100644
@@ -51,6 +51,12 @@
 #define _PAGE_RW            (1UL<<1)       /* writeable */
 #define _PAGE_PS            (1UL<<7)       /* page size bit*/
 
+#define TDX_INSTRUCTION_IO 30
+
+#define TDX_SUCCESS_PORT 0x30
+#define TDX_IO_READ 0
+#define TDX_IO_WRITE 1
+
 #define GDT_ENTRY(flags, base, limit)				\
 		((((base)  & 0xff000000ULL) << (56-24)) |	\
 		 (((flags) & 0x0000f0ffULL) << 40) |		\
@@ -83,4 +89,147 @@ void prepare_source_image(struct kvm_vm *vm, void *guest_code,
 			  size_t guest_code_size,
 			  uint64_t guest_code_signature);
 
+/*
+ * Generic TDCALL function that can be used to communicate with TDX module or
+ * VMM.
+ * Input operands: rax, rbx, rcx, rdx, r8-r15, rbp, rsi, rdi
+ * Output operands: rax, r8-r15, rbx, rdx, rdi, rsi
+ * rcx is actually a bitmap to tell TDX module which register values will be
+ * exposed to the VMM.
+ * XMM0-XMM15 registers can be used as input operands but the current
+ * implementation does not support it yet.
+ */
+static inline void tdcall(struct kvm_regs *regs)
+{
+	asm volatile (
+			"mov %13, %%rax;\n\t"
+			"mov %14, %%rbx;\n\t"
+			"mov %15, %%rcx;\n\t"
+			"mov %16, %%rdx;\n\t"
+			"mov %17, %%r8;\n\t"
+			"mov %18, %%r9;\n\t"
+			"mov %19, %%r10;\n\t"
+			"mov %20, %%r11;\n\t"
+			"mov %21, %%r12;\n\t"
+			"mov %22, %%r13;\n\t"
+			"mov %23, %%r14;\n\t"
+			"mov %24, %%r15;\n\t"
+			"mov %25, %%rbp;\n\t"
+			"mov %26, %%rsi;\n\t"
+			"mov %27, %%rdi;\n\t"
+			".byte 0x66, 0x0F, 0x01, 0xCC;\n\t"
+			"mov %%rax, %0;\n\t"
+			"mov %%rbx, %1;\n\t"
+			"mov %%rdx, %2;\n\t"
+			"mov %%r8, %3;\n\t"
+			"mov %%r9, %4;\n\t"
+			"mov %%r10, %5;\n\t"
+			"mov %%r11, %6;\n\t"
+			"mov %%r12, %7;\n\t"
+			"mov %%r13, %8;\n\t"
+			"mov %%r14, %9;\n\t"
+			"mov %%r15, %10;\n\t"
+			"mov %%rsi, %11;\n\t"
+			"mov %%rdi, %12;\n\t"
+			: "=m" (regs->rax), "=m" (regs->rbx), "=m" (regs->rdx),
+			"=m" (regs->r8), "=m" (regs->r9), "=m" (regs->r10),
+			"=m" (regs->r11), "=m" (regs->r12), "=m" (regs->r13),

tools/testing/selftests/kvm/lib/x86_64/tdx.h void prepare_source_image(struct kvm_vm *vm, void *guest_code, · line 146

Quoted 10 lines
tools/testing/selftests/kvm/lib/x86_64/tdx.h +10 -0
+			"=m" (regs->r14), "=m" (regs->r15), "=m" (regs->rsi),
+			"=m" (regs->rdi)
+			: "m" (regs->rax), "m" (regs->rbx), "m" (regs->rcx),
+			"m" (regs->rdx), "m" (regs->r8), "m" (regs->r9),
+			"m" (regs->r10), "m" (regs->r11), "m" (regs->r12),
+			"m" (regs->r13), "m" (regs->r14), "m" (regs->r15),
+			"m" (regs->rbp), "m" (regs->rsi), "m" (regs->rdi)
+			: "rax", "rbx", "rcx", "rdx", "r8", "r9", "r10", "r11",
+			"r12", "r13", "r14", "r15", "rbp", "rsi", "rdi");
+}

Sometimes compiler (my gcc is (Ubuntu 11.1.0-1ubuntu1~20.04) 11.1.0) doesn't like clobering the frame pointer as follows. (I edited the caller site for other test.)

x86_64/tdx_vm_tests.c:343:1: error: bp cannot be used in ‘asm’ here

I ended up the following workaround. I didn't use pushq/popq pair because I didn't want to play with offset in the stack of the caller.

tools/testing/selftests/kvm/lib/x86_64/tdx.h +9 -3
index aa6961c6f304..8ddf3b64f003 100644
@@ -122,7 +122,11 @@ void prepare_source_image(struct kvm_vm *vm, void *guest_code,
  */
 static inline void tdcall(struct kvm_regs *regs)
 {
+       unsigned long saved_rbp = 0;
+
        asm volatile (
+               /* gcc complains that frame pointer %rbp can't be clobbered. */
+                       "movq %%rbp, %28;\n\t"
                        "mov %13, %%rax;\n\t"
                        "mov %14, %%rbx;\n\t"
                        "mov %15, %%rcx;\n\t"
@@ -152,6 +156,8 @@ static inline void tdcall(struct kvm_regs *regs)
                        "mov %%r15, %10;\n\t"
                        "mov %%rsi, %11;\n\t"
                        "mov %%rdi, %12;\n\t"
+                       "movq %28, %%rbp\n\t"
+                       "movq $0, %28\n\t"
                        : "=m" (regs->rax), "=m" (regs->rbx), "=m" (regs->rdx),
                        "=m" (regs->r8), "=m" (regs->r9), "=m" (regs->r10),
                        "=m" (regs->r11), "=m" (regs->r12), "=m" (regs->r13),
@@ -161,9 +167,10 @@ static inline void tdcall(struct kvm_regs *regs)
                        "m" (regs->rdx), "m" (regs->r8), "m" (regs->r9),
                        "m" (regs->r10), "m" (regs->r11), "m" (regs->r12),
                        "m" (regs->r13), "m" (regs->r14), "m" (regs->r15),
-                       "m" (regs->rbp), "m" (regs->rsi), "m" (regs->rdi)
+                       "m" (regs->rbp), "m" (regs->rsi), "m" (regs->rdi),
+                       "m" (saved_rbp)
                        : "rax", "rbx", "rcx", "rdx", "r8", "r9", "r10", "r11",
-                       "r12", "r13", "r14", "r15", "rbp", "rsi", "rdi");
+                         "r12", "r13", "r14", "r15", "rsi", "rdi");
 }
 
-- 

Isaku Yamahata <[email protected]>

#3 lore raw

Re: [RFC PATCH v2 02/17] KVM: selftest: Add helper functions to create TDX VMs

In reply to Sagi Shahar
Quoted 32 lines · Sagi Shahar
tools/testing/selftests/kvm/lib/x86_64/processor.c +17 -0
index f35626df1dea..2a6e28c769f2 100644
@@ -8,6 +8,7 @@
 #include "test_util.h"
 #include "kvm_util.h"
 #include "processor.h"
+#include "tdx.h"
 
 #ifndef NUM_INTERRUPTS
 #define NUM_INTERRUPTS 256
@@ -641,6 +642,32 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id,
 	return vcpu;
 }
 
+/*
+ * Adds a vCPU to a TD (Trusted Domain) with minimum  defaults. It will not set
+ * up any general purpose registers as they will be initialized by the TDX. In
+ * TDX, vCPUs RIP is set to 0xFFFFFFF0. See Intel TDX EAS Section "Initial State
+ * of Guest GPRs" for more information on vCPUs initial register values when
+ * entering the TD first time.
+ *
+ * Input Args:
+ *   vm - Virtual Machine
+ *   vcpuid - The id of the VCPU to add to the VM.
+ */
+struct kvm_vcpu *vm_vcpu_add_tdx(struct kvm_vm *vm, uint32_t vcpu_id)
+{
+	struct kvm_mp_state mp_state;
+	struct kvm_vcpu *vcpu;
+

tools/testing/selftests/kvm/lib/x86_64/processor.c struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id, · line 670

Quoted 10 lines
tools/testing/selftests/kvm/lib/x86_64/processor.c +10 -0
+	vcpu = __vm_vcpu_add(vm, vcpu_id);
+	initialize_td_vcpu(vcpu);
+
+	/* Setup the MP state */
+	mp_state.mp_state = 0;
+	vcpu_mp_state_set(vcpu, &mp_state);
+
+	return vcpu;
+}
+

It's better to use symbolic value. I know this is copied from vmx version, though.

tools/testing/selftests/kvm/lib/x86_64/processor.c +2 -2
index 3bb7dc5a55ea..4009bc926e33 100644
@@ -636,7 +636,7 @@ struct kvm_vcpu *vm_arch_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id,
        vcpu_regs_set(vcpu, &regs);
 
        /* Setup the MP state */
-       mp_state.mp_state = 0;
+       mp_state.mp_state = KVM_MP_STATE_RUNNABLE;
        vcpu_mp_state_set(vcpu, &mp_state);
 
        return vcpu;
@@ -662,7 +662,7 @@ struct kvm_vcpu *vm_vcpu_add_tdx(struct kvm_vm *vm, uint32_t vcpu_id)
        initialize_td_vcpu(vcpu);
 
        /* Setup the MP state */
-       mp_state.mp_state = 0;
+       mp_state.mp_state = KVM_MP_STATE_RUNNABLE;
        vcpu_mp_state_set(vcpu, &mp_state);
 
        return vcpu;
Signature · 1 line
Isaku Yamahata <[email protected]>
#4 lore raw

Re: [RFC PATCH v2 02/17] KVM: selftest: Add helper functions to create TDX VMs

In reply to Sagi Shahar

On Tue, Aug 30, 2022 at 10:19:45PM +0000,
Sagi Shahar <[email protected]> wrote:
...

Quoted 74 lines
tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c new file +68 -0
new file mode 100644
index 000000000000..72bf2ff24a29
@@ -0,0 +1,338 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/stringify.h>
+#include "asm/kvm.h"
+#include "tdx.h"
+#include <stdlib.h>
+#include <malloc.h>
+#include "processor.h"
+#include <string.h>
+
+char *tdx_cmd_str[] = {
+	"KVM_TDX_CAPABILITIES",
+	"KVM_TDX_INIT_VM",
+	"KVM_TDX_INIT_VCPU",
+	"KVM_TDX_INIT_MEM_REGION",
+	"KVM_TDX_FINALIZE_VM"
+};
+
+#define TDX_MAX_CMD_STR (ARRAY_SIZE(tdx_cmd_str))
+#define EIGHT_INT3_INSTRUCTIONS 0xCCCCCCCCCCCCCCCC
+
+#define XFEATURE_LBR		15
+#define XFEATURE_XTILECFG	17
+#define XFEATURE_XTILEDATA	18
+#define XFEATURE_MASK_LBR	(1 << XFEATURE_LBR)
+#define XFEATURE_MASK_XTILECFG	(1 << XFEATURE_XTILECFG)
+#define XFEATURE_MASK_XTILEDATA	(1 << XFEATURE_XTILEDATA)
+#define XFEATURE_MASK_XTILE	(XFEATURE_MASK_XTILECFG | XFEATURE_MASK_XTILEDATA)
+
+
+static void tdx_ioctl(int fd, int ioctl_no, uint32_t flags, void *data)
+{
+	struct kvm_tdx_cmd tdx_cmd;
+	int r;
+
+	TEST_ASSERT(ioctl_no < TDX_MAX_CMD_STR, "Unknown TDX CMD : %d\n",
+		    ioctl_no);
+
+	memset(&tdx_cmd, 0x0, sizeof(tdx_cmd));
+	tdx_cmd.id = ioctl_no;
+	tdx_cmd.flags = flags;
+	tdx_cmd.data = (uint64_t)data;
+	r = ioctl(fd, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+	TEST_ASSERT(r == 0, "%s failed: %d  %d", tdx_cmd_str[ioctl_no], r,
+		    errno);
+}
+
+static struct tdx_cpuid_data get_tdx_cpuid_data(struct kvm_vm *vm)
+{
+	static struct tdx_cpuid_data cpuid_data;
+	int ret, i;
+
+	if (cpuid_data.cpuid.nent)
+		return cpuid_data;
+
+	memset(&cpuid_data, 0, sizeof(cpuid_data));
+	cpuid_data.cpuid.nent = KVM_MAX_CPUID_ENTRIES;
+	ret = ioctl(vm->kvm_fd, KVM_GET_SUPPORTED_CPUID, &cpuid_data);
+	if (ret) {
+		TEST_FAIL("KVM_GET_SUPPORTED_CPUID failed %d %d\n",
+		    ret, errno);
+		cpuid_data.cpuid.nent = 0;
+		return cpuid_data;
+	}
+
+	for (i = 0; i < KVM_MAX_CPUID_ENTRIES; i++) {
+		struct kvm_cpuid_entry2 *e = &cpuid_data.entries[i];
+
+		/* TDX doesn't support LBR and AMX features yet.

tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c line 78

Quoted 10 lines
tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c new file +10 -0
+		 * Disable those bits from the XCR0 register.
+		 */
+		if (e->function == 0xd && (e->index == 0)) {
+			e->eax &= ~XFEATURE_MASK_LBR;
+			e->eax &= ~XFEATURE_MASK_XTILE;
+		}
+	}
+
+	return cpuid_data;
+}

CET also needs adjust. How about the followings?

tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c +17 -3
index 1c3e47006cd2..123db9b76f82 100644
@@ -25,7 +25,7 @@ char *tdx_cmd_str[] = {
 #define XFEATURE_MASK_XTILECFG (1 << XFEATURE_XTILECFG)
 #define XFEATURE_MASK_XTILEDATA        (1 << XFEATURE_XTILEDATA)
 #define XFEATURE_MASK_XTILE    (XFEATURE_MASK_XTILECFG | XFEATURE_MASK_XTILEDATA)
-
+#define XFEATURE_MASK_CET      ((1 << 11) | (1 << 12))
 
 static int __tdx_ioctl(int fd, int ioctl_no, uint32_t flags, void *data)
 {
@@ -72,12 +72,26 @@ static struct tdx_cpuid_data get_tdx_cpuid_data(struct kvm_vm *vm)
        for (i = 0; i < KVM_MAX_CPUID_ENTRIES; i++) {
                struct kvm_cpuid_entry2 *e = &cpuid_data.entries[i];
 
-               /* TDX doesn't support LBR and AMX features yet.
+               /* TDX doesn't support LBR yet.
                 * Disable those bits from the XCR0 register.
                 */
                if (e->function == 0xd && (e->index == 0)) {
                        e->eax &= ~XFEATURE_MASK_LBR;
-                       e->eax &= ~XFEATURE_MASK_XTILE;
+
+                       /*
+                        * TDX modules requires both CET_{U, S} to be set even
+                        * if only one is supported.
+                        */
+                       if (e->eax & XFEATURE_MASK_CET) {
+                               e->eax |= XFEATURE_MASK_CET;
+                       }
+                       /*
+                        * TDX module requires both XTILE_{CFG, DATA} to be set.
+                        * Both bits are required for AMX to be functional.
+                        */
+                       if ((e->eax & XFEATURE_MASK_XTILE) != XFEATURE_MASK_XTILE) {
+                               e->eax &= ~XFEATURE_MASK_XTILE;
+                       }
                }
        }
 
Signature · 1 line
Isaku Yamahata <[email protected]>
#22 lore raw
In reply to Sagi Shahar

Here is one more test to exercise KVM_TDX_CAPABILITIES on top of this patch series.

From f9c4c9013040ce7dee84e1d3370875e5158900bf Mon Sep 17 00:00:00 2001
Message-Id: <f9c4c9013040ce7dee84e1d3370875e5158900bf.1661995648.git.isaku.yamahata@intel.com>
In-Reply-To: <6ce32225079b83991b9f170730a8810005a079b0.1661995647.git.isaku.yamahata@intel.com>
References: <6ce32225079b83991b9f170730a8810005a079b0.1661995647.git.isaku.yamahata@intel.com>
From: Isaku Yamahata <[email protected]>
Date: Wed, 16 Mar 2022 09:15:40 -0700
Subject: [PATCH] KVM: selftest: tdx: call KVM_TDX_CAPABILITIES for
test

Add exercise of KVM_TDX_CAPABILITIES. The result isn't used.

---
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  1 +
 .../selftests/kvm/lib/x86_64/tdx_lib.c        | 52 +++++++++++++++++--
 .../selftests/kvm/x86_64/tdx_vm_tests.c       |  3 ++
 3 files changed, 53 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/lib/x86_64/tdx.h +1 -0
index be8564f4672d..bfa3709a76e5 100644
@@ -119,6 +119,7 @@ struct page_table {
 void add_td_memory(struct kvm_vm *vm, void *source_page,
 		   uint64_t gpa, int size);
 void finalize_td_memory(struct kvm_vm *vm);
+void get_tdx_capabilities(struct kvm_vm *vm);
 void initialize_td(struct kvm_vm *vm);
 void initialize_td_with_attributes(struct kvm_vm *vm, uint64_t attributes);
 void initialize_td_vcpu(struct kvm_vcpu *vcpu);
tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c +49 -3

72 lines not on this page · raw message

tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c +3 -0
index a96abada54b6..b3f9e3fa41f4 100644
@@ -197,6 +197,9 @@ void verify_td_lifecycle(void)
 	/* Create a TD VM with no memory.*/
 	vm = vm_create_tdx();
 
+	/* Get TDX capabilities */
+	get_tdx_capabilities(vm);
+
 	/* Allocate TD guest memory and initialize the TD.*/
 	initialize_td(vm);
 
Signature · 3 lines
2.25.1

On Tue, Aug 30, 2022 at 10:19:43PM +0000,
Quoted 104 lines · Sagi Shahar

Hello,

This is v2 of the patch series for TDX selftests.

It is based on v5.19-rc8 and Intel's V8 of the TDX host patches which was proposed in https://lkml.org/lkml/2022/8/8/877

The tree can be found at
https://github.com/googleprodkernel/linux-cc/tree/selftests

Major changes vrom v1:
- rebased to v5.19
- added helpers for success and failure reporting
- added additional test cases

---
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform.

Intel has recently submitted a set of RFC patches for KVM support for TDX and more information can be found on the latest TDX Support
Patches: https://lkml.org/lkml/2022/8/8/877

Due to the nature of the confidential computing environment that TDX provides, it is very difficult to verify/test the KVM support. TDX requires UEFI and the guest kernel to be enlightened which are all under development.

We are working on a set of selftests to close this gap and be able to verify the KVM functionality to support TDX lifecycle and GHCI [1] interface.

We are looking for any feedback on:
- Patch series itself
- Any suggestion on how we should approach testing TDX functionality. Does selftests seems reasonable or should we switch to using KVM unit tests. I would be happy to get some perspective on how KVM unit tests can help us more.
- Any test case or scenario that we should add.
- Anything else I have not thought of yet.

Current patch series provide the following capabilities:

- Provide helper functions to create a TD (Trusted Domain) using the KVM ioctls
- Provide helper functions to create a guest image that can include any testing code
- Provide helper functions and wrapper functions to write testing code using GHCI interface
- Add a test case that verifies TDX life cycle
- Add a test case that verifies TDX GHCI port IO

TODOs:
- Use existing function to create page tables dynamically (ie __virt_pg_map())
- Remove arbitrary defined magic numbers for data structure offsets
- Add TDVMCALL for error reporting
- Add additional test cases as some listed below
- Add #VE handlers to help testing more complicated test cases

---
Erdem Aktas (4):
KVM: selftests: Add support for creating non-default type VMs
KVM: selftest: Add helper functions to create TDX VMs
KVM: selftest: Adding TDX life cycle test.
KVM: selftest: Adding test case for TDX port IO

Roger Wang (1):
KVM: selftest: TDX: Add TDG.VP.INFO test

Ryan Afranji (2):
KVM: selftest: TDX: Verify the behavior when host consumes a TD
private memory
KVM: selftest: TDX: Add shared memory test

Sagi Shahar (10):
KVM: selftest: TDX: Add report_fatal_error test
KVM: selftest: TDX: Add basic TDX CPUID test
KVM: selftest: TDX: Add basic get_td_vmcall_info test
KVM: selftest: TDX: Add TDX IO writes test
KVM: selftest: TDX: Add TDX IO reads test
KVM: selftest: TDX: Add TDX MSR read/write tests
KVM: selftest: TDX: Add TDX HLT exit test
KVM: selftest: TDX: Add TDX MMIO reads test
KVM: selftest: TDX: Add TDX MMIO writes test
KVM: selftest: TDX: Add TDX CPUID TDVMCALL test

 tools/testing/selftests/kvm/Makefile          |    2 +
 .../selftests/kvm/include/kvm_util_base.h     |   12 +-
 .../selftests/kvm/include/x86_64/processor.h  |    1 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |    6 +-
 .../selftests/kvm/lib/x86_64/processor.c      |   27 +
 tools/testing/selftests/kvm/lib/x86_64/tdx.h  |  495 +++++
 .../selftests/kvm/lib/x86_64/tdx_lib.c        |  373 ++++
 .../selftests/kvm/x86_64/tdx_vm_tests.c       | 1666 +++++++++++++++++
 8 files changed, 2577 insertions(+), 5 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/lib/x86_64/tdx.h
 create mode 100644 tools/testing/selftests/kvm/lib/x86_64/tdx_lib.c
 create mode 100644 tools/testing/selftests/kvm/x86_64/tdx_vm_tests.c
Signature · 1 line
2.37.2.789.g6183377224-goog
Signature · 1 line
Isaku Yamahata <[email protected]>
#7 lore raw

Re: [RFC PATCH v2 03/17] KVM: selftest: Adding TDX life cycle test.

In reply to Isaku Yamahata
Quoted 35 lines · Isaku Yamahata

Sometimes compiler (my gcc is (Ubuntu 11.1.0-1ubuntu1~20.04) 11.1.0) doesn't like clobering the frame pointer as follows. (I edited the caller site for other test.)

x86_64/tdx_vm_tests.c:343:1: error: bp cannot be used in ‘asm’ here

I ended up the following workaround. I didn't use pushq/popq pair because I didn't want to play with offset in the stack of the caller.

tools/testing/selftests/kvm/lib/x86_64/tdx.h +6 -0
index aa6961c6f304..8ddf3b64f003 100644
@@ -122,7 +122,11 @@ void prepare_source_image(struct kvm_vm *vm, void *guest_code,
  */
 static inline void tdcall(struct kvm_regs *regs)
 {
+       unsigned long saved_rbp = 0;
+
        asm volatile (
+               /* gcc complains that frame pointer %rbp can't be clobbered. */
+                       "movq %%rbp, %28;\n\t"
                        "mov %13, %%rax;\n\t"
                        "mov %14, %%rbx;\n\t"
                        "mov %15, %%rcx;\n\t"
@@ -152,6 +156,8 @@ static inline void tdcall(struct kvm_regs *regs)
                        "mov %%r15, %10;\n\t"
                        "mov %%rsi, %11;\n\t"
                        "mov %%rdi, %12;\n\t"
+                       "movq %28, %%rbp\n\t"
+                       "movq $0, %28\n\t"
                        : "=m" (regs->rax), "=m" (regs->rbx), "=m" (regs->rdx),
                        "=m" (regs->r8), "=m" (regs->r9), "=m" (regs->r10),
                        "=m" (regs->r11), "=m" (regs->r12), "=m" (regs->r13),

tools/testing/selftests/kvm/lib/x86_64/tdx.h static inline void tdcall(struct kvm_regs *regs) · line 174

Quoted 10 lines
tools/testing/selftests/kvm/lib/x86_64/tdx.h +3 -2
@@ -161,9 +167,10 @@ static inline void tdcall(struct kvm_regs *regs)
                        "m" (regs->rdx), "m" (regs->r8), "m" (regs->r9),
                        "m" (regs->r10), "m" (regs->r11), "m" (regs->r12),
                        "m" (regs->r13), "m" (regs->r14), "m" (regs->r15),
-                       "m" (regs->rbp), "m" (regs->rsi), "m" (regs->rdi)
+                       "m" (regs->rbp), "m" (regs->rsi), "m" (regs->rdi),
+                       "m" (saved_rbp)
                        : "rax", "rbx", "rcx", "rdx", "r8", "r9", "r10", "r11",
-                       "r12", "r13", "r14", "r15", "rbp", "rsi", "rdi");
+                         "r12", "r13", "r14", "r15", "rsi", "rdi");
 }

Inline assembly for TDCALL is going to be a mess. Assuming proper assembly doesn't Just Work for selftests, we should solve that problem and build this on top.