Skip to contentSkip to conversation

Execute hypercalls from guests according to cpu type

PATCH Superseded Vishal Annapurve 4 participants 33 messages Sep 15, 2022 – Oct 13, 2022

Subsystems (4)

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v2 cover Execute hypercalls from guests according to cpu type

This series is posted in context of the discussion at:
https://lore.kernel.org/lkml/Ywa9T+jKUpaHLu%[email protected]/

Changes in v2:
* Addressed comments from Andrew and David
* Common function with constructor attribute used to setup initial state
* Changes are split in more logical granules as per feedback

Major changes:
1) Move common startup logic to a single function in kvm_util.c
2) Introduce following APIs:
kvm_selftest_arch_init: to perform arch specific common startup.
kvm_arch_post_vm_elf_load: to update the guest memory state to convey
common information to guests.
3) For x86, capture cpu type at startup and pass on the cpu type to guest after guest elf is loaded.
4) Execute hypercall instruction from within guest VMs according to the cpu type. This will help prevent an extra kvm exit during hypercall execution.

Vishal Annapurve (8):
KVM: selftests: move common startup logic to kvm_util.c
KVM: selftests: Add arch specific initialization
KVM: selftests: Add arch specific post vm load setup
KVM: selftests: x86: Precompute the result for is_{intel,amd}_cpu()
KVM: selftests: x86: delete svm_vmcall_test
KVM: selftests: x86: Execute cpu specific hypercall from nested guests
Kvm: selftests: x86: Execute cpu specific vmcall instruction
KVM: selftests: x86: xen: Execute cpu specific vmcall instruction

 tools/testing/selftests/kvm/.gitignore        |  1 -
 .../selftests/kvm/aarch64/arch_timer.c        |  3 -
 .../selftests/kvm/aarch64/hypercalls.c        |  2 -
 .../testing/selftests/kvm/aarch64/vgic_irq.c  |  3 -
 .../selftests/kvm/include/kvm_util_base.h     |  9 +++
 .../selftests/kvm/include/x86_64/processor.h  | 10 +++
 .../selftests/kvm/include/x86_64/vmx.h        |  9 ---
 .../selftests/kvm/lib/aarch64/processor.c     | 22 +++---
 tools/testing/selftests/kvm/lib/elf.c         |  2 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |  8 ++
 .../selftests/kvm/lib/riscv/processor.c       |  8 ++
 .../selftests/kvm/lib/s390x/processor.c       |  8 ++
 .../selftests/kvm/lib/x86_64/perf_test_util.c |  2 +-
 .../selftests/kvm/lib/x86_64/processor.c      | 38 +++++++++-
 .../testing/selftests/kvm/memslot_perf_test.c |  3 -
 tools/testing/selftests/kvm/rseq_test.c       |  3 -
 tools/testing/selftests/kvm/s390x/memop.c     |  2 -
 tools/testing/selftests/kvm/s390x/resets.c    |  2 -
 .../selftests/kvm/s390x/sync_regs_test.c      |  3 -
 .../selftests/kvm/set_memory_region_test.c    |  3 -
 .../kvm/x86_64/cr4_cpuid_sync_test.c          |  3 -
 .../kvm/x86_64/emulator_error_test.c          |  3 -
 .../selftests/kvm/x86_64/hyperv_cpuid.c       |  3 -
 .../selftests/kvm/x86_64/platform_info_test.c |  3 -
 .../kvm/x86_64/pmu_event_filter_test.c        |  3 -
 .../selftests/kvm/x86_64/set_sregs_test.c     |  3 -
 tools/testing/selftests/kvm/x86_64/smm_test.c |  2 +-
 .../testing/selftests/kvm/x86_64/state_test.c |  8 +-
 .../kvm/x86_64/svm_nested_soft_inject_test.c  |  3 -
 .../selftests/kvm/x86_64/svm_vmcall_test.c    | 74 -------------------
 .../selftests/kvm/x86_64/sync_regs_test.c     |  3 -
 .../selftests/kvm/x86_64/userspace_io_test.c  |  3 -
 .../kvm/x86_64/userspace_msr_exit_test.c      |  3 -
 .../kvm/x86_64/vmx_apic_access_test.c         |  2 +-
 .../selftests/kvm/x86_64/vmx_dirty_log_test.c |  2 +-
 .../kvm/x86_64/vmx_nested_tsc_scaling_test.c  |  2 +-
 .../kvm/x86_64/vmx_preemption_timer_test.c    |  2 +-
 .../kvm/x86_64/vmx_tsc_adjust_test.c          |  2 +-
 .../selftests/kvm/x86_64/xen_shinfo_test.c    | 64 ++++++----------
 .../selftests/kvm/x86_64/xen_vmcall_test.c    | 14 +++-
 40 files changed, 138 insertions(+), 205 deletions(-)
 delete mode 100644 tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
Signature · 1 line
2.37.2.789.g6183377224-goog
#1 lore raw

PATCH v2 1/8 KVM: selftests: move common startup logic to kvm_util.c

Consolidate common startup logic in one place by implementing a single setup function with __attribute((constructor)) for all selftests within kvm_util.c.

This allows moving logic like:

        /* Tell stdout not to buffer its content */
        setbuf(stdout, NULL);

to a single file for all selftests.

This will also allow any required setup at entry in future to be done in common main function.

More context is discussed at:
https://lore.kernel.org/lkml/Ywa9T+jKUpaHLu%[email protected]/

---
 tools/testing/selftests/kvm/aarch64/arch_timer.c            | 3 ---
 tools/testing/selftests/kvm/aarch64/hypercalls.c            | 2 --
 tools/testing/selftests/kvm/aarch64/vgic_irq.c              | 3 ---
 tools/testing/selftests/kvm/lib/kvm_util.c                  | 6 ++++++
 tools/testing/selftests/kvm/memslot_perf_test.c             | 3 ---
 tools/testing/selftests/kvm/rseq_test.c                     | 3 ---
 tools/testing/selftests/kvm/s390x/memop.c                   | 2 --
 tools/testing/selftests/kvm/s390x/resets.c                  | 2 --
 tools/testing/selftests/kvm/s390x/sync_regs_test.c          | 3 ---
 tools/testing/selftests/kvm/set_memory_region_test.c        | 3 ---
 tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c    | 3 ---
 tools/testing/selftests/kvm/x86_64/emulator_error_test.c    | 3 ---
 tools/testing/selftests/kvm/x86_64/hyperv_cpuid.c           | 3 ---
 tools/testing/selftests/kvm/x86_64/platform_info_test.c     | 3 ---
 tools/testing/selftests/kvm/x86_64/pmu_event_filter_test.c  | 3 ---
 tools/testing/selftests/kvm/x86_64/set_sregs_test.c         | 3 ---
 .../selftests/kvm/x86_64/svm_nested_soft_inject_test.c      | 3 ---
 tools/testing/selftests/kvm/x86_64/sync_regs_test.c         | 3 ---
 tools/testing/selftests/kvm/x86_64/userspace_io_test.c      | 3 ---
 .../testing/selftests/kvm/x86_64/userspace_msr_exit_test.c  | 3 ---
 20 files changed, 6 insertions(+), 54 deletions(-)
tools/testing/selftests/kvm/aarch64/arch_timer.c +0 -3
index 574eb73f0e90..07836bd2672b 100644
@@ -462,9 +462,6 @@ int main(int argc, char *argv[])
 {
 	struct kvm_vm *vm;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	if (!parse_args(argc, argv))
 		exit(KSFT_SKIP);
 
tools/testing/selftests/kvm/aarch64/hypercalls.c +0 -2
index a39da3fe4952..6463fd118429 100644
@@ -306,8 +306,6 @@ static void test_run(void)
 
 int main(void)
 {
-	setbuf(stdout, NULL);
-
 	test_run();
 	return 0;
 }
tools/testing/selftests/kvm/aarch64/vgic_irq.c +0 -3
index 17417220a083..3f204f2e93bf 100644
@@ -818,9 +818,6 @@ int main(int argc, char **argv)
 	int opt;
 	bool eoi_split = false;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	while ((opt = getopt(argc, argv, "hn:e:l:")) != -1) {
 		switch (opt) {
 		case 'n':
tools/testing/selftests/kvm/lib/kvm_util.c +6 -0
index 9889fe0d8919..3c83838999f5 100644
@@ -1979,3 +1979,9 @@ void __vm_get_stat(struct kvm_vm *vm, const char *stat_name, uint64_t *data,
 		break;
 	}
 }
+
+void __attribute((constructor)) kvm_selftest_init(void)
+{
+	/* Tell stdout not to buffer its content. */
+	setbuf(stdout, NULL);
+}
tools/testing/selftests/kvm/memslot_perf_test.c +0 -3
index 44995446d942..f7ba77ff45c9 100644
@@ -1007,9 +1007,6 @@ int main(int argc, char *argv[])
 	struct test_result rbestslottime;
 	int tctr;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	if (!parse_args(argc, argv, &targs))
 		return -1;
 
tools/testing/selftests/kvm/rseq_test.c +0 -3
index fac248a43666..1cc459822c27 100644
@@ -205,9 +205,6 @@ int main(int argc, char *argv[])
 	struct kvm_vcpu *vcpu;
 	u32 cpu, rseq_cpu;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	r = sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
 	TEST_ASSERT(!r, "sched_getaffinity failed, errno = %d (%s)", errno,
 		    strerror(errno));
tools/testing/selftests/kvm/s390x/memop.c +0 -2
index 9113696d5178..3fd81e58f40c 100644
@@ -760,8 +760,6 @@ int main(int argc, char *argv[])
 
 	TEST_REQUIRE(kvm_has_cap(KVM_CAP_S390_MEM_OP));
 
-	setbuf(stdout, NULL);	/* Tell stdout not to buffer its content */
-
 	ksft_print_header();
 
 	ksft_set_plan(ARRAY_SIZE(testlist));
tools/testing/selftests/kvm/s390x/resets.c +0 -2
index 19486084eb30..e41e2cb8ffa9 100644
@@ -296,8 +296,6 @@ int main(int argc, char *argv[])
 	bool has_s390_vcpu_resets = kvm_check_cap(KVM_CAP_S390_VCPU_RESETS);
 	int idx;
 
-	setbuf(stdout, NULL);	/* Tell stdout not to buffer its content */
-
 	ksft_print_header();
 	ksft_set_plan(ARRAY_SIZE(testlist));
 
tools/testing/selftests/kvm/s390x/sync_regs_test.c +0 -3
index 3fdb6e2598eb..2ddde41c44ba 100644
@@ -231,9 +231,6 @@ int main(int argc, char *argv[])
 
 	TEST_REQUIRE(kvm_has_cap(KVM_CAP_SYNC_REGS));
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	ksft_print_header();
 
 	ksft_set_plan(ARRAY_SIZE(testlist));
tools/testing/selftests/kvm/set_memory_region_test.c +0 -3
index 0d55f508d595..614141d6e53d 100644
@@ -392,9 +392,6 @@ int main(int argc, char *argv[])
 	int i, loops;
 #endif
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 #ifdef __x86_64__
 	/*
 	 * FIXME: the zero-memslot test fails on aarch64 and s390x because
tools/testing/selftests/kvm/x86_64/cr4_cpuid_sync_test.c +0 -3
index 4208487652f8..1027a671c7d3 100644
@@ -57,9 +57,6 @@ int main(int argc, char *argv[])
 
 	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_XSAVE));
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
 	run = vcpu->run;
 
tools/testing/selftests/kvm/x86_64/emulator_error_test.c +0 -3
index 236e11755ba6..3334adcfd591 100644
@@ -156,9 +156,6 @@ int main(int argc, char *argv[])
 	uint64_t *hva;
 	int rc;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	TEST_REQUIRE(kvm_has_cap(KVM_CAP_SMALLER_MAXPHYADDR));
 
 	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
tools/testing/selftests/kvm/x86_64/hyperv_cpuid.c +0 -3
index e804eb08dff9..5c27efbf405e 100644
@@ -134,9 +134,6 @@ int main(int argc, char *argv[])
 	const struct kvm_cpuid2 *hv_cpuid_entries;
 	struct kvm_vcpu *vcpu;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	TEST_REQUIRE(kvm_has_cap(KVM_CAP_HYPERV_CPUID));
 
 	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
tools/testing/selftests/kvm/x86_64/platform_info_test.c +0 -3
index 76417c7d687b..310a104d94f0 100644
@@ -72,9 +72,6 @@ int main(int argc, char *argv[])
 	struct kvm_vm *vm;
 	uint64_t msr_platform_info;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	TEST_REQUIRE(kvm_has_cap(KVM_CAP_MSR_PLATFORM_INFO));
 
 	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
tools/testing/selftests/kvm/x86_64/pmu_event_filter_test.c +0 -3
index ea4e259a1e2e..a6ffa245c897 100644
@@ -447,9 +447,6 @@ int main(int argc, char *argv[])
 	struct kvm_vcpu *vcpu;
 	struct kvm_vm *vm;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	TEST_REQUIRE(kvm_has_cap(KVM_CAP_PMU_EVENT_FILTER));
 
 	TEST_REQUIRE(use_intel_pmu() || use_amd_pmu());
tools/testing/selftests/kvm/x86_64/set_sregs_test.c +0 -3
index 2bb08bf2125d..a284fcef6ed7 100644
@@ -82,9 +82,6 @@ int main(int argc, char *argv[])
 	uint64_t cr4;
 	int rc;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	/*
 	 * Create a dummy VM, specifically to avoid doing KVM_SET_CPUID2, and
 	 * use it to verify all supported CR4 bits can be set prior to defining
tools/testing/selftests/kvm/x86_64/svm_nested_soft_inject_test.c +0 -3
index e637d7736012..e497ace629c1 100644
@@ -194,9 +194,6 @@ static void run_test(bool is_nmi)
 
 int main(int argc, char *argv[])
 {
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SVM));
 
 	TEST_ASSERT(kvm_cpu_has(X86_FEATURE_NRIPS),
tools/testing/selftests/kvm/x86_64/sync_regs_test.c +0 -3
index 9b6db0b0b13e..d2f9b5bdfab2 100644
@@ -90,9 +90,6 @@ int main(int argc, char *argv[])
 	struct kvm_vcpu_events events;
 	int rv, cap;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	cap = kvm_check_cap(KVM_CAP_SYNC_REGS);
 	TEST_REQUIRE((cap & TEST_SYNC_FIELDS) == TEST_SYNC_FIELDS);
 	TEST_REQUIRE(!(cap & INVALID_SYNC_FIELD));
tools/testing/selftests/kvm/x86_64/userspace_io_test.c +0 -3
index 7316521428f8..91076c9787b4 100644
@@ -56,9 +56,6 @@ int main(int argc, char *argv[])
 	struct kvm_vm *vm;
 	struct ucall uc;
 
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
 	run = vcpu->run;
 
tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c +0 -3
index a4f06370a245..8ef5c8b25e95 100644
@@ -735,9 +735,6 @@ static void test_msr_permission_bitmap(void)
 
 int main(int argc, char *argv[])
 {
-	/* Tell stdout not to buffer its content */
-	setbuf(stdout, NULL);
-
 	test_msr_filter_allow();
 
 	test_msr_filter_deny();
Signature · 1 line
2.37.2.789.g6183377224-goog
Reviewed-by on v2 1/8
#2 lore raw

Re: [V2 PATCH 1/8] KVM: selftests: move common startup logic to kvm_util.c

Quoted 12 lines · Vishal Annapurve

Consolidate common startup logic in one place by implementing a single setup function with __attribute((constructor)) for all selftests within kvm_util.c.

This allows moving logic like:

        /* Tell stdout not to buffer its content */
        setbuf(stdout, NULL);

to a single file for all selftests.

This will also allow any required setup at entry in future to be done in common main function.

Quoted 6 lines

More context is discussed at:
https://lore.kernel.org/lkml/Ywa9T+jKUpaHLu%[email protected]/

---

#3 lore raw

PATCH v2 2/8 KVM: selftests: Add arch specific initialization

In reply to Vishal Annapurve

Introduce arch specific API: kvm_selftest_arch_init to allow each arch to handle initialization before running any selftest logic.

---
 .../selftests/kvm/include/kvm_util_base.h      |  5 +++++
 .../selftests/kvm/lib/aarch64/processor.c      | 18 +++++++++---------
 tools/testing/selftests/kvm/lib/kvm_util.c     |  2 ++
 .../selftests/kvm/lib/riscv/processor.c        |  4 ++++
 .../selftests/kvm/lib/s390x/processor.c        |  4 ++++
 .../selftests/kvm/lib/x86_64/processor.c       |  4 ++++
 6 files changed, 28 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/include/kvm_util_base.h +5 -0
index 24fde97f6121..98edbbda9f97 100644
@@ -834,4 +834,9 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm)
 	return __vm_enable_cap(vm, KVM_CAP_VM_DISABLE_NX_HUGE_PAGES, 0);
 }
 
+/*
+ * API to execute architecture specific setup before executing selftest logic.
+ */
+void kvm_selftest_arch_init(void);
+
 #endif /* SELFTEST_KVM_UTIL_BASE_H */
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -9
index 6f5551368944..2281d6c5d02f 100644
@@ -495,15 +495,6 @@ void aarch64_get_supported_page_sizes(uint32_t ipa,
 	close(kvm_fd);
 }
 
-/*
- * arm64 doesn't have a true default mode, so start by computing the
- * available IPA space and page sizes early.
- */
-void __attribute__((constructor)) init_guest_modes(void)
-{
-       guest_modes_append_default();
-}
-
 void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1,
 	       uint64_t arg2, uint64_t arg3, uint64_t arg4, uint64_t arg5,
 	       uint64_t arg6, struct arm_smccc_res *res)
@@ -528,3 +519,12 @@ void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1,
 		       [arg4] "r"(arg4), [arg5] "r"(arg5), [arg6] "r"(arg6)
 		     : "x0", "x1", "x2", "x3", "x4", "x5", "x6", "x7");
 }
+
+/*
+ * arm64 doesn't have a true default mode, so start by computing the
+ * available IPA space and page sizes early.
+ */
+void kvm_selftest_arch_init(void)
+{
+	guest_modes_append_default();
+}
tools/testing/selftests/kvm/lib/kvm_util.c +2 -0
index 3c83838999f5..dafe4471a6c7 100644
@@ -1984,4 +1984,6 @@ void __attribute((constructor)) kvm_selftest_init(void)
 {
 	/* Tell stdout not to buffer its content. */
 	setbuf(stdout, NULL);
+
+	kvm_selftest_arch_init();
 }
tools/testing/selftests/kvm/lib/riscv/processor.c +4 -0
index 604478151212..26660dd2ba78 100644
@@ -362,3 +362,7 @@ void vcpu_args_set(struct kvm_vcpu *vcpu, unsigned int num, ...)
 void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 {
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
tools/testing/selftests/kvm/lib/s390x/processor.c +4 -0
index 89d7340d9cbd..8654ec74009a 100644
@@ -218,3 +218,7 @@ void vcpu_arch_dump(FILE *stream, struct kvm_vcpu *vcpu, uint8_t indent)
 void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 {
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
tools/testing/selftests/kvm/lib/x86_64/processor.c +4 -0
index 2e6e61bbe81b..20bf125f9363 100644
@@ -1311,3 +1311,7 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 
 	return val == 'Y';
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
Signature · 1 line
2.37.2.789.g6183377224-goog
Reviewed-by on v2 2/8
#4 lore raw

Re: [V2 PATCH 2/8] KVM: selftests: Add arch specific initialization

Quoted 49 lines · Vishal Annapurve

Introduce arch specific API: kvm_selftest_arch_init to allow each arch to handle initialization before running any selftest logic.

---
 .../selftests/kvm/include/kvm_util_base.h      |  5 +++++
 .../selftests/kvm/lib/aarch64/processor.c      | 18 +++++++++---------
 tools/testing/selftests/kvm/lib/kvm_util.c     |  2 ++
 .../selftests/kvm/lib/riscv/processor.c        |  4 ++++
 .../selftests/kvm/lib/s390x/processor.c        |  4 ++++
 .../selftests/kvm/lib/x86_64/processor.c       |  4 ++++
 6 files changed, 28 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/include/kvm_util_base.h +5 -0
index 24fde97f6121..98edbbda9f97 100644
@@ -834,4 +834,9 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm)
 	return __vm_enable_cap(vm, KVM_CAP_VM_DISABLE_NX_HUGE_PAGES, 0);
 }
 
+/*
+ * API to execute architecture specific setup before executing selftest logic.
+ */
+void kvm_selftest_arch_init(void);
+
 #endif /* SELFTEST_KVM_UTIL_BASE_H */
tools/testing/selftests/kvm/lib/aarch64/processor.c +0 -9
index 6f5551368944..2281d6c5d02f 100644
@@ -495,15 +495,6 @@ void aarch64_get_supported_page_sizes(uint32_t ipa,
 	close(kvm_fd);
 }
 
-/*
- * arm64 doesn't have a true default mode, so start by computing the
- * available IPA space and page sizes early.
- */
-void __attribute__((constructor)) init_guest_modes(void)
-{
-       guest_modes_append_default();
-}
-
 void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1,
 	       uint64_t arg2, uint64_t arg3, uint64_t arg4, uint64_t arg5,
 	       uint64_t arg6, struct arm_smccc_res *res)

tools/testing/selftests/kvm/lib/aarch64/processor.c void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1, · line 526

Quoted 8 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +5 -0
@@ -528,3 +519,12 @@ void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1,
 		       [arg4] "r"(arg4), [arg5] "r"(arg5), [arg6] "r"(arg6)
 		     : "x0", "x1", "x2", "x3", "x4", "x5", "x6", "x7");
 }
+
+/*
+ * arm64 doesn't have a true default mode, so start by computing the
+ * available IPA space and page sizes early.
+ */

It'd be better to move this comment inside the function above the guest_modes_append_default call.

Quoted 54 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +4 -0
+void kvm_selftest_arch_init(void)
+{
+	guest_modes_append_default();
+}
tools/testing/selftests/kvm/lib/kvm_util.c +2 -0
index 3c83838999f5..dafe4471a6c7 100644
@@ -1984,4 +1984,6 @@ void __attribute((constructor)) kvm_selftest_init(void)
 {
 	/* Tell stdout not to buffer its content. */
 	setbuf(stdout, NULL);
+
+	kvm_selftest_arch_init();
 }
tools/testing/selftests/kvm/lib/riscv/processor.c +4 -0
index 604478151212..26660dd2ba78 100644
@@ -362,3 +362,7 @@ void vcpu_args_set(struct kvm_vcpu *vcpu, unsigned int num, ...)
 void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 {
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
tools/testing/selftests/kvm/lib/s390x/processor.c +4 -0
index 89d7340d9cbd..8654ec74009a 100644
@@ -218,3 +218,7 @@ void vcpu_arch_dump(FILE *stream, struct kvm_vcpu *vcpu, uint8_t indent)
 void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 {
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
tools/testing/selftests/kvm/lib/x86_64/processor.c +4 -0
index 2e6e61bbe81b..20bf125f9363 100644
@@ -1311,3 +1311,7 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 
 	return val == 'Y';
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
Signature · 1 line
2.37.2.789.g6183377224-goog

Otherwise,

#5 lore raw

Re: [V2 PATCH 2/8] KVM: selftests: Add arch specific initialization

Quoted 54 lines · Andrew Jones
Quoted 52 lines · Vishal Annapurve

Introduce arch specific API: kvm_selftest_arch_init to allow each arch to handle initialization before running any selftest logic.

---
 .../selftests/kvm/include/kvm_util_base.h      |  5 +++++
 .../selftests/kvm/lib/aarch64/processor.c      | 18 +++++++++---------
 tools/testing/selftests/kvm/lib/kvm_util.c     |  2 ++
 .../selftests/kvm/lib/riscv/processor.c        |  4 ++++
 .../selftests/kvm/lib/s390x/processor.c        |  4 ++++
 .../selftests/kvm/lib/x86_64/processor.c       |  4 ++++
 6 files changed, 28 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/include/kvm_util_base.h +5 -0
index 24fde97f6121..98edbbda9f97 100644
@@ -834,4 +834,9 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm)
      return __vm_enable_cap(vm, KVM_CAP_VM_DISABLE_NX_HUGE_PAGES, 0);
 }
 
+/*
+ * API to execute architecture specific setup before executing selftest logic.
+ */
+void kvm_selftest_arch_init(void);
+
 #endif /* SELFTEST_KVM_UTIL_BASE_H */
tools/testing/selftests/kvm/lib/aarch64/processor.c +0 -9
index 6f5551368944..2281d6c5d02f 100644
@@ -495,15 +495,6 @@ void aarch64_get_supported_page_sizes(uint32_t ipa,
      close(kvm_fd);
 }
 
-/*
- * arm64 doesn't have a true default mode, so start by computing the
- * available IPA space and page sizes early.
- */
-void __attribute__((constructor)) init_guest_modes(void)
-{
-       guest_modes_append_default();
-}
-
 void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1,
             uint64_t arg2, uint64_t arg3, uint64_t arg4, uint64_t arg5,
             uint64_t arg6, struct arm_smccc_res *res)
@@ -528,3 +519,12 @@ void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1,
                     [arg4] "r"(arg4), [arg5] "r"(arg5), [arg6] "r"(arg6)
                   : "x0", "x1", "x2", "x3", "x4", "x5", "x6", "x7");
 }
Quoted 9 lines

tools/testing/selftests/kvm/lib/aarch64/processor.c void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1, · line 526

Quoted 5 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +5 -0
+
+/*
+ * arm64 doesn't have a true default mode, so start by computing the
+ * available IPA space and page sizes early.
+ */

It'd be better to move this comment inside the function above the guest_modes_append_default call.

Ack, will fix this in the next series.

Quoted 58 lines
Quoted 52 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +4 -0
+void kvm_selftest_arch_init(void)
+{
+     guest_modes_append_default();
+}
tools/testing/selftests/kvm/lib/kvm_util.c +2 -0
index 3c83838999f5..dafe4471a6c7 100644
@@ -1984,4 +1984,6 @@ void __attribute((constructor)) kvm_selftest_init(void)
 {
      /* Tell stdout not to buffer its content. */
      setbuf(stdout, NULL);
+
+     kvm_selftest_arch_init();
 }
tools/testing/selftests/kvm/lib/riscv/processor.c +4 -0
index 604478151212..26660dd2ba78 100644
@@ -362,3 +362,7 @@ void vcpu_args_set(struct kvm_vcpu *vcpu, unsigned int num, ...)
 void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 {
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
tools/testing/selftests/kvm/lib/s390x/processor.c +4 -0
index 89d7340d9cbd..8654ec74009a 100644
@@ -218,3 +218,7 @@ void vcpu_arch_dump(FILE *stream, struct kvm_vcpu *vcpu, uint8_t indent)
 void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 {
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
tools/testing/selftests/kvm/lib/x86_64/processor.c +4 -1
index 2e6e61bbe81b..20bf125f9363 100644
@@ -1311,3 +1311,7 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 
      return val == 'Y';
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
--
Quoted 2 lines

2.37.2.789.g6183377224-goog

Otherwise,

#6 lore raw

Re: [V2 PATCH 2/8] KVM: selftests: Add arch specific initialization

In reply to Vishal Annapurve
Quoted 19 lines · Vishal Annapurve

Introduce arch specific API: kvm_selftest_arch_init to allow each arch to handle initialization before running any selftest logic.

---
 .../selftests/kvm/include/kvm_util_base.h      |  5 +++++
 .../selftests/kvm/lib/aarch64/processor.c      | 18 +++++++++---------
 tools/testing/selftests/kvm/lib/kvm_util.c     |  2 ++
 .../selftests/kvm/lib/riscv/processor.c        |  4 ++++
 .../selftests/kvm/lib/s390x/processor.c        |  4 ++++
 .../selftests/kvm/lib/x86_64/processor.c       |  4 ++++
 6 files changed, 28 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/include/kvm_util_base.h +0 -0
index 24fde97f6121..98edbbda9f97 100644

tools/testing/selftests/kvm/include/kvm_util_base.h static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm) · line 838

Quoted 5 lines
tools/testing/selftests/kvm/include/kvm_util_base.h +2 -0
@@ -834,4 +834,9 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm)
 	return __vm_enable_cap(vm, KVM_CAP_VM_DISABLE_NX_HUGE_PAGES, 0);
 }
 
+/*
+ * API to execute architecture specific setup before executing selftest logic.

nit: s/before executing selftest logic/before main()/

("selftest logic" is vague)

Quoted 42 lines
tools/testing/selftests/kvm/include/kvm_util_base.h +3 -0
+ */
+void kvm_selftest_arch_init(void);
+
 #endif /* SELFTEST_KVM_UTIL_BASE_H */
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -9
index 6f5551368944..2281d6c5d02f 100644
@@ -495,15 +495,6 @@ void aarch64_get_supported_page_sizes(uint32_t ipa,
 	close(kvm_fd);
 }
 
-/*
- * arm64 doesn't have a true default mode, so start by computing the
- * available IPA space and page sizes early.
- */
-void __attribute__((constructor)) init_guest_modes(void)
-{
-       guest_modes_append_default();
-}
-
 void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1,
 	       uint64_t arg2, uint64_t arg3, uint64_t arg4, uint64_t arg5,
 	       uint64_t arg6, struct arm_smccc_res *res)
@@ -528,3 +519,12 @@ void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1,
 		       [arg4] "r"(arg4), [arg5] "r"(arg5), [arg6] "r"(arg6)
 		     : "x0", "x1", "x2", "x3", "x4", "x5", "x6", "x7");
 }
+
+/*
+ * arm64 doesn't have a true default mode, so start by computing the
+ * available IPA space and page sizes early.
+ */
+void kvm_selftest_arch_init(void)
+{
+	guest_modes_append_default();
+}
tools/testing/selftests/kvm/lib/kvm_util.c +0 -0
index 3c83838999f5..dafe4471a6c7 100644

tools/testing/selftests/kvm/lib/kvm_util.c void __attribute((constructor)) kvm_selftest_init(void) · line 1989

Quoted 6 lines
tools/testing/selftests/kvm/lib/kvm_util.c +2 -0
@@ -1984,4 +1984,6 @@ void __attribute((constructor)) kvm_selftest_init(void)
 {
 	/* Tell stdout not to buffer its content. */
 	setbuf(stdout, NULL);
+
+	kvm_selftest_arch_init();
 }

Suggest defining a default no-op implementation of kvm_selftest_arch_init() using __weak since most architectures do not actually need an implementation.

Quoted 39 lines
tools/testing/selftests/kvm/lib/riscv/processor.c +4 -0
index 604478151212..26660dd2ba78 100644
@@ -362,3 +362,7 @@ void vcpu_args_set(struct kvm_vcpu *vcpu, unsigned int num, ...)
 void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 {
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
tools/testing/selftests/kvm/lib/s390x/processor.c +4 -0
index 89d7340d9cbd..8654ec74009a 100644
@@ -218,3 +218,7 @@ void vcpu_arch_dump(FILE *stream, struct kvm_vcpu *vcpu, uint8_t indent)
 void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 {
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
tools/testing/selftests/kvm/lib/x86_64/processor.c +4 -0
index 2e6e61bbe81b..20bf125f9363 100644
@@ -1311,3 +1311,7 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 
 	return val == 'Y';
 }
+
+void kvm_selftest_arch_init(void)
+{
+}
Signature · 1 line
2.37.2.789.g6183377224-goog
#7 lore raw

Re: [V2 PATCH 2/8] KVM: selftests: Add arch specific initialization

Quoted 24 lines · David Matlack
Quoted 22 lines · Vishal Annapurve

Introduce arch specific API: kvm_selftest_arch_init to allow each arch to handle initialization before running any selftest logic.

---
 .../selftests/kvm/include/kvm_util_base.h      |  5 +++++
 .../selftests/kvm/lib/aarch64/processor.c      | 18 +++++++++---------
 tools/testing/selftests/kvm/lib/kvm_util.c     |  2 ++
 .../selftests/kvm/lib/riscv/processor.c        |  4 ++++
 .../selftests/kvm/lib/s390x/processor.c        |  4 ++++
 .../selftests/kvm/lib/x86_64/processor.c       |  4 ++++
 6 files changed, 28 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/include/kvm_util_base.h +0 -0
index 24fde97f6121..98edbbda9f97 100644
@@ -834,4 +834,9 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm)
      return __vm_enable_cap(vm, KVM_CAP_VM_DISABLE_NX_HUGE_PAGES, 0);
 }
 
Quoted 5 lines

tools/testing/selftests/kvm/include/kvm_util_base.h static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm) · line 838

Quoted 2 lines
tools/testing/selftests/kvm/include/kvm_util_base.h +2 -0
+/*
+ * API to execute architecture specific setup before executing selftest logic.

nit: s/before executing selftest logic/before main()/

Ack, will fix this in the next series.

Quoted 46 lines

("selftest logic" is vague)

Quoted 44 lines
tools/testing/selftests/kvm/include/kvm_util_base.h +3 -0
+ */
+void kvm_selftest_arch_init(void);
+
 #endif /* SELFTEST_KVM_UTIL_BASE_H */
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -9
index 6f5551368944..2281d6c5d02f 100644
@@ -495,15 +495,6 @@ void aarch64_get_supported_page_sizes(uint32_t ipa,
      close(kvm_fd);
 }
 
-/*
- * arm64 doesn't have a true default mode, so start by computing the
- * available IPA space and page sizes early.
- */
-void __attribute__((constructor)) init_guest_modes(void)
-{
-       guest_modes_append_default();
-}
-
 void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1,
             uint64_t arg2, uint64_t arg3, uint64_t arg4, uint64_t arg5,
             uint64_t arg6, struct arm_smccc_res *res)
@@ -528,3 +519,12 @@ void smccc_hvc(uint32_t function_id, uint64_t arg0, uint64_t arg1,
                     [arg4] "r"(arg4), [arg5] "r"(arg5), [arg6] "r"(arg6)
                   : "x0", "x1", "x2", "x3", "x4", "x5", "x6", "x7");
 }
+
+/*
+ * arm64 doesn't have a true default mode, so start by computing the
+ * available IPA space and page sizes early.
+ */
+void kvm_selftest_arch_init(void)
+{
+     guest_modes_append_default();
+}
tools/testing/selftests/kvm/lib/kvm_util.c +0 -0
index 3c83838999f5..dafe4471a6c7 100644
@@ -1984,4 +1984,6 @@ void __attribute((constructor)) kvm_selftest_init(void)
 {
      /* Tell stdout not to buffer its content. */
Quoted 9 lines

tools/testing/selftests/kvm/lib/kvm_util.c void __attribute((constructor)) kvm_selftest_init(void) · line 1989

Quoted 4 lines
tools/testing/selftests/kvm/lib/kvm_util.c +2 -0
      setbuf(stdout, NULL);
+
+     kvm_selftest_arch_init();
 }

Suggest defining a default no-op implementation of kvm_selftest_arch_init() using __weak since most architectures do not actually need an implementation.

Ack, will update this in the next series.

Quoted 2 lines
Quoted 2 lines

...

#8 lore raw

PATCH v2 3/8 KVM: selftests: Add arch specific post vm load setup

In reply to Vishal Annapurve

Add arch specific API kvm_selftest_post_vm_elf_load to possibly communicate information to VM that is already known to selftest VMM logic.

This API will be used in followup commit to convey cpu vendor type to the guest vm.

---
 tools/testing/selftests/kvm/include/kvm_util_base.h | 4 ++++
 tools/testing/selftests/kvm/lib/aarch64/processor.c | 4 ++++
 tools/testing/selftests/kvm/lib/elf.c               | 2 ++
 tools/testing/selftests/kvm/lib/riscv/processor.c   | 4 ++++
 tools/testing/selftests/kvm/lib/s390x/processor.c   | 4 ++++
 tools/testing/selftests/kvm/lib/x86_64/processor.c  | 4 ++++
 6 files changed, 22 insertions(+)
tools/testing/selftests/kvm/include/kvm_util_base.h +4 -0
index 98edbbda9f97..73cfee3ebd76 100644
@@ -839,4 +839,8 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm)
  */
 void kvm_selftest_arch_init(void);
 
+/*
+ * API to execute architecture specific setup after loading the vm elf.
+ */
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm);
 #endif /* SELFTEST_KVM_UTIL_BASE_H */
tools/testing/selftests/kvm/lib/aarch64/processor.c +4 -0
index 2281d6c5d02f..12627c560f66 100644
@@ -528,3 +528,7 @@ void kvm_selftest_arch_init(void)
 {
 	guest_modes_append_default();
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
tools/testing/selftests/kvm/lib/elf.c +2 -0
index 9f54c098d9d0..b8963a7146ce 100644
@@ -189,4 +189,6 @@ void kvm_vm_elf_load(struct kvm_vm *vm, const char *filename)
 				phdr.p_filesz);
 		}
 	}
+
+	kvm_arch_post_vm_elf_load(vm);
 }
tools/testing/selftests/kvm/lib/riscv/processor.c +4 -0
index 26660dd2ba78..4491c0d4be45 100644
@@ -366,3 +366,7 @@ void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 void kvm_selftest_arch_init(void)
 {
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
tools/testing/selftests/kvm/lib/s390x/processor.c +4 -0
index 8654ec74009a..332501b3693f 100644
@@ -222,3 +222,7 @@ void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 void kvm_selftest_arch_init(void)
 {
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
tools/testing/selftests/kvm/lib/x86_64/processor.c +4 -0
index 20bf125f9363..25ae972f5c71 100644
@@ -1315,3 +1315,7 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 void kvm_selftest_arch_init(void)
 {
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
Signature · 1 line
2.37.2.789.g6183377224-goog
#9 lore raw

Re: [V2 PATCH 3/8] KVM: selftests: Add arch specific post vm load setup

Quoted 22 lines · Vishal Annapurve

Add arch specific API kvm_selftest_post_vm_elf_load to possibly communicate information to VM that is already known to selftest VMM logic.

This API will be used in followup commit to convey cpu vendor type to the guest vm.

---
 tools/testing/selftests/kvm/include/kvm_util_base.h | 4 ++++
 tools/testing/selftests/kvm/lib/aarch64/processor.c | 4 ++++
 tools/testing/selftests/kvm/lib/elf.c               | 2 ++
 tools/testing/selftests/kvm/lib/riscv/processor.c   | 4 ++++
 tools/testing/selftests/kvm/lib/s390x/processor.c   | 4 ++++
 tools/testing/selftests/kvm/lib/x86_64/processor.c  | 4 ++++
 6 files changed, 22 insertions(+)
tools/testing/selftests/kvm/include/kvm_util_base.h +0 -0
index 98edbbda9f97..73cfee3ebd76 100644

tools/testing/selftests/kvm/include/kvm_util_base.h static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm) · line 843

Quoted 5 lines
tools/testing/selftests/kvm/include/kvm_util_base.h +2 -0
@@ -839,4 +839,8 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm)
  */
 void kvm_selftest_arch_init(void);
 
+/*
+ * API to execute architecture specific setup after loading the vm elf.

It's not a "vm elf" per-se, it's "loading the elf into the VM". How about:

/*
* API to execute arch-specific logic after loading the selftest ELF image
* into the VM.
*/

Quoted 20 lines
tools/testing/selftests/kvm/include/kvm_util_base.h +2 -0
+ */
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm);
 #endif /* SELFTEST_KVM_UTIL_BASE_H */
tools/testing/selftests/kvm/lib/aarch64/processor.c +4 -0
index 2281d6c5d02f..12627c560f66 100644
@@ -528,3 +528,7 @@ void kvm_selftest_arch_init(void)
 {
 	guest_modes_append_default();
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
tools/testing/selftests/kvm/lib/elf.c +0 -0
index 9f54c098d9d0..b8963a7146ce 100644

tools/testing/selftests/kvm/lib/elf.c void kvm_vm_elf_load(struct kvm_vm *vm, const char *filename) · line 194

Quoted 6 lines
tools/testing/selftests/kvm/lib/elf.c +2 -0
@@ -189,4 +189,6 @@ void kvm_vm_elf_load(struct kvm_vm *vm, const char *filename)
 				phdr.p_filesz);
 		}
 	}
+
+	kvm_arch_post_vm_elf_load(vm);
 }

Same suggestion here as the previous patch: Use __weak to define a default no-op implementation of kvm_arch_post_vm_elf_load().

Quoted 39 lines
tools/testing/selftests/kvm/lib/riscv/processor.c +4 -0
index 26660dd2ba78..4491c0d4be45 100644
@@ -366,3 +366,7 @@ void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 void kvm_selftest_arch_init(void)
 {
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
tools/testing/selftests/kvm/lib/s390x/processor.c +4 -0
index 8654ec74009a..332501b3693f 100644
@@ -222,3 +222,7 @@ void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 void kvm_selftest_arch_init(void)
 {
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
tools/testing/selftests/kvm/lib/x86_64/processor.c +4 -0
index 20bf125f9363..25ae972f5c71 100644
@@ -1315,3 +1315,7 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 void kvm_selftest_arch_init(void)
 {
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
Signature · 1 line
2.37.2.789.g6183377224-goog
#10 lore raw

Re: [V2 PATCH 3/8] KVM: selftests: Add arch specific post vm load setup

Quoted 33 lines · David Matlack
Quoted 22 lines · Vishal Annapurve

Add arch specific API kvm_selftest_post_vm_elf_load to possibly communicate information to VM that is already known to selftest VMM logic.

This API will be used in followup commit to convey cpu vendor type to the guest vm.

---
 tools/testing/selftests/kvm/include/kvm_util_base.h | 4 ++++
 tools/testing/selftests/kvm/lib/aarch64/processor.c | 4 ++++
 tools/testing/selftests/kvm/lib/elf.c               | 2 ++
 tools/testing/selftests/kvm/lib/riscv/processor.c   | 4 ++++
 tools/testing/selftests/kvm/lib/s390x/processor.c   | 4 ++++
 tools/testing/selftests/kvm/lib/x86_64/processor.c  | 4 ++++
 6 files changed, 22 insertions(+)
tools/testing/selftests/kvm/include/kvm_util_base.h +0 -0
index 98edbbda9f97..73cfee3ebd76 100644

tools/testing/selftests/kvm/include/kvm_util_base.h static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm) · line 843

Quoted 5 lines
tools/testing/selftests/kvm/include/kvm_util_base.h +2 -0
@@ -839,4 +839,8 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm)
  */
 void kvm_selftest_arch_init(void);
 
+/*
+ * API to execute architecture specific setup after loading the vm elf.

It's not a "vm elf" per-se, it's "loading the elf into the VM". How about:

Quoted 5 lines

/*
* API to execute arch-specific logic after loading the selftest ELF image
* into the VM.
*/

Ack. Will update this in the next series.

Quoted 69 lines
Quoted 20 lines
tools/testing/selftests/kvm/include/kvm_util_base.h +2 -0
+ */
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm);
 #endif /* SELFTEST_KVM_UTIL_BASE_H */
tools/testing/selftests/kvm/lib/aarch64/processor.c +4 -0
index 2281d6c5d02f..12627c560f66 100644
@@ -528,3 +528,7 @@ void kvm_selftest_arch_init(void)
 {
      guest_modes_append_default();
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
tools/testing/selftests/kvm/lib/elf.c +0 -0
index 9f54c098d9d0..b8963a7146ce 100644

tools/testing/selftests/kvm/lib/elf.c void kvm_vm_elf_load(struct kvm_vm *vm, const char *filename) · line 194

Quoted 6 lines
tools/testing/selftests/kvm/lib/elf.c +2 -0
@@ -189,4 +189,6 @@ void kvm_vm_elf_load(struct kvm_vm *vm, const char *filename)
                              phdr.p_filesz);
              }
      }
+
+     kvm_arch_post_vm_elf_load(vm);
 }

Same suggestion here as the previous patch: Use __weak to define a default no-op implementation of kvm_arch_post_vm_elf_load().

Quoted 39 lines
tools/testing/selftests/kvm/lib/riscv/processor.c +4 -0
index 26660dd2ba78..4491c0d4be45 100644
@@ -366,3 +366,7 @@ void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 void kvm_selftest_arch_init(void)
 {
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
tools/testing/selftests/kvm/lib/s390x/processor.c +4 -0
index 8654ec74009a..332501b3693f 100644
@@ -222,3 +222,7 @@ void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 void kvm_selftest_arch_init(void)
 {
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
tools/testing/selftests/kvm/lib/x86_64/processor.c +4 -1
index 20bf125f9363..25ae972f5c71 100644
@@ -1315,3 +1315,7 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 void kvm_selftest_arch_init(void)
 {
 }
+
+void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
+{
+}
--

2.37.2.789.g6183377224-goog

#11 lore raw

Re: [V2 PATCH 3/8] KVM: selftests: Add arch specific post vm load setup

Quoted 17 lines · Vishal Annapurve
Quoted 16 lines · David Matlack

tools/testing/selftests/kvm/include/kvm_util_base.h static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm) · line 843

Quoted 10 lines · Vishal Annapurve
tools/testing/selftests/kvm/include/kvm_util_base.h +2 -0
index 98edbbda9f97..73cfee3ebd76 100644
@@ -839,4 +839,8 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm)
  */
 void kvm_selftest_arch_init(void);
 
+/*
+ * API to execute architecture specific setup after loading the vm elf.

It's not a "vm elf" per-se, it's "loading the elf into the VM". How about:

Quoted 7 lines
Quoted 5 lines

/*
* API to execute arch-specific logic after loading the selftest ELF image
* into the VM.
*/

Ack. Will update this in the next series.

Even better, call it from __vm_create() and name it something like kvm_arch_vm_post_create(). Like David said, while the hook has a dependency on being called after loading the ELF image, the action that arch code is expected to take doesn't have anything to do with loading the ELF image.

And then instead of introducing an arch hook with no implementation, the patch that adds the hook can instead use it to replace the x86-64 #ifdef in __vm_create().

tools/testing/selftests/kvm/lib/kvm_util.c +2 -3
index dafe4471a6c7..593dfadb662e 100644
@@ -298,9 +298,8 @@ struct kvm_vm *__vm_create(enum vm_guest_mode mode, uint32_t nr_runnable_vcpus,
 
        kvm_vm_elf_load(vm, program_invocation_name);
 
-#ifdef __x86_64__
-       vm_create_irqchip(vm);
-#endif
+       kvm_arch_vm_post_create(vm);
+
        return vm;
 }
 
#12 lore raw

Re: [V2 PATCH 3/8] KVM: selftests: Add arch specific post vm load setup

Quoted 27 lines · Sean Christopherson
Quoted 17 lines · Vishal Annapurve
Quoted 16 lines · David Matlack

tools/testing/selftests/kvm/include/kvm_util_base.h static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm) · line 843

Quoted 10 lines · Vishal Annapurve
tools/testing/selftests/kvm/include/kvm_util_base.h +2 -0
index 98edbbda9f97..73cfee3ebd76 100644
@@ -839,4 +839,8 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm)
  */
 void kvm_selftest_arch_init(void);
 
+/*
+ * API to execute architecture specific setup after loading the vm elf.

It's not a "vm elf" per-se, it's "loading the elf into the VM". How about:

Quoted 7 lines
Quoted 5 lines

/*
* API to execute arch-specific logic after loading the selftest ELF image
* into the VM.
*/

Ack. Will update this in the next series.

Quoted 8 lines

Even better, call it from __vm_create() and name it something like kvm_arch_vm_post_create(). Like David said, while the hook has a dependency on being called after loading the ELF image, the action that arch code is expected to take doesn't have anything to do with loading the ELF image.

And then instead of introducing an arch hook with no implementation, the patch that adds the hook can instead use it to replace the x86-64 #ifdef in __vm_create().

Today upstream kernel selftests don't have scenarios where kvm_vm_elf_load can get called directly outside ___vm_create but there are selftests that are up for review [1], [2] that may call kvm_vm_elf_load directly. Above suggestion will not work in this scenario, is it suitable to assume that all the callers of kvm_vm_elf_load will eventually execute kvm_arch_vm_post_create?

[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/lkml/[email protected]/T/#u

Quoted 17 lines
tools/testing/selftests/kvm/lib/kvm_util.c +2 -3
index dafe4471a6c7..593dfadb662e 100644
@@ -298,9 +298,8 @@ struct kvm_vm *__vm_create(enum vm_guest_mode mode, uint32_t nr_runnable_vcpus,
 
        kvm_vm_elf_load(vm, program_invocation_name);
 
-#ifdef __x86_64__
-       vm_create_irqchip(vm);
-#endif
+       kvm_arch_vm_post_create(vm);
+
        return vm;
 }
 
#13 lore raw

Re: [V2 PATCH 3/8] KVM: selftests: Add arch specific post vm load setup

Quoted 10 lines · Vishal Annapurve
Quoted 8 lines · Sean Christopherson

Even better, call it from __vm_create() and name it something like kvm_arch_vm_post_create(). Like David said, while the hook has a dependency on being called after loading the ELF image, the action that arch code is expected to take doesn't have anything to do with loading the ELF image.

And then instead of introducing an arch hook with no implementation, the patch that adds the hook can instead use it to replace the x86-64 #ifdef in __vm_create().

Quoted 6 lines

Today upstream kernel selftests don't have scenarios where kvm_vm_elf_load can get called directly outside ___vm_create but there are selftests that are up for review [1], [2] that may call kvm_vm_elf_load directly. Above suggestion will not work in this scenario, is it suitable to assume that all the callers of kvm_vm_elf_load will eventually execute kvm_arch_vm_post_create?

No, but that's irrelevant. And actually, in any reasonable hypothetical situation I can think of, it's actually undesirable to always call kvm_arch_vm_post_create() after kvm_vm_elf_load().

Hypothetically, if there were a use case where kvm_vm_elf_load() is called multiple times, then stuffing the "Intel vs. "AMD" flag should only be done for the binary that actually defines that flag. The flag is defined by the library's processor.c, and so the hook should be tied to the library's loading of its binary, i.e. to the creation of the VM.

If a test were loading multiple binaries, and the test wanted to tweak things specific to a binary after loading said binary, then the test can and should do that without needed a library arch hook.

If the arch hook was to take action specific to loading _any_ binary, than yes, a hook in kvm_vm_elf_load() would be in order, but this hook is about taking action when creating a VM, not to loading a binary.

But this is all very, very hypothetical. I can't think of a scenario where loading multiple binaries would be less complex than solving whatever hypothetical problem makes it difficult to link everything into a single binary.

And if a test manually loads a binary _and_ wants to actually run the guest after doing vm_create_barebones() or ____vm_create(), then the test is doing it wrong, as those low level APIs are provided _only_ for cases where a test doesn't need to run vCPUs.

#14 lore raw

Re: [V2 PATCH 3/8] KVM: selftests: Add arch specific post vm load setup

Quoted 37 lines · Sean Christopherson
Quoted 10 lines · Vishal Annapurve
Quoted 8 lines · Sean Christopherson

Even better, call it from __vm_create() and name it something like kvm_arch_vm_post_create(). Like David said, while the hook has a dependency on being called after loading the ELF image, the action that arch code is expected to take doesn't have anything to do with loading the ELF image.

And then instead of introducing an arch hook with no implementation, the patch that adds the hook can instead use it to replace the x86-64 #ifdef in __vm_create().

Quoted 6 lines

Today upstream kernel selftests don't have scenarios where kvm_vm_elf_load can get called directly outside ___vm_create but there are selftests that are up for review [1], [2] that may call kvm_vm_elf_load directly. Above suggestion will not work in this scenario, is it suitable to assume that all the callers of kvm_vm_elf_load will eventually execute kvm_arch_vm_post_create?

No, but that's irrelevant. And actually, in any reasonable hypothetical situation I can think of, it's actually undesirable to always call kvm_arch_vm_post_create() after kvm_vm_elf_load().

Hypothetically, if there were a use case where kvm_vm_elf_load() is called multiple times, then stuffing the "Intel vs. "AMD" flag should only be done for the binary that actually defines that flag. The flag is defined by the library's processor.c, and so the hook should be tied to the library's loading of its binary, i.e. to the creation of the VM.

If a test were loading multiple binaries, and the test wanted to tweak things specific to a binary after loading said binary, then the test can and should do that without needed a library arch hook.

If the arch hook was to take action specific to loading _any_ binary, than yes, a hook in kvm_vm_elf_load() would be in order, but this hook is about taking action when creating a VM, not to loading a binary.

Quoted 8 lines

But this is all very, very hypothetical. I can't think of a scenario where loading multiple binaries would be less complex than solving whatever hypothetical problem makes it difficult to link everything into a single binary.

And if a test manually loads a binary _and_ wants to actually run the guest after doing vm_create_barebones() or ____vm_create(), then the test is doing it wrong, as those low level APIs are provided _only_ for cases where a test doesn't need to run vCPUs.

Ack, will update this patch as per the feedback here.

#15 lore raw

PATCH v2 4/8 KVM: selftests: x86: Precompute the result for is_{intel,amd}_cpu()

In reply to Vishal Annapurve

Cache the vendor CPU type in a global variable so that multiple calls to is_intel_cpu() do not need to re-execute CPUID.

Add cpu vendor check in kvm_hypercall() so that it executes correct vmcall/vmmcall instruction when running on Intel/AMD hosts. This avoids exit to KVM which anyway tries to patch the instruction according to the cpu type.

As part of this change, sync the global variable is_cpu_amd into the guest so the guest can determine which hypercall instruction to use without needing to re-execute CPUID for every hypercall.

---
 .../testing/selftests/kvm/lib/x86_64/processor.c  | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/lib/x86_64/processor.c +12 -3
index 25ae972f5c71..c0ae938772f6 100644
@@ -19,6 +19,7 @@
 #define MAX_NR_CPUID_ENTRIES 100
 
 vm_vaddr_t exception_handlers;
+static bool is_cpu_amd;
 
 static void regs_dump(FILE *stream, struct kvm_regs *regs, uint8_t indent)
 {
@@ -1019,7 +1020,7 @@ static bool cpu_vendor_string_is(const char *vendor)
 
 bool is_intel_cpu(void)
 {
-	return cpu_vendor_string_is("GenuineIntel");
+	return (is_cpu_amd == false);
 }
 
 /*
@@ -1027,7 +1028,7 @@ bool is_intel_cpu(void)
  */
 bool is_amd_cpu(void)
 {
-	return cpu_vendor_string_is("AuthenticAMD");
+	return (is_cpu_amd == true);
 }
 
 void kvm_get_cpu_address_width(unsigned int *pa_bits, unsigned int *va_bits)
@@ -1182,9 +1183,15 @@ uint64_t kvm_hypercall(uint64_t nr, uint64_t a0, uint64_t a1, uint64_t a2,
 {
 	uint64_t r;
 
-	asm volatile("vmcall"
+	if (is_amd_cpu())
+		asm volatile("vmmcall"
 		     : "=a"(r)
 		     : "a"(nr), "b"(a0), "c"(a1), "d"(a2), "S"(a3));
+	else
+		asm volatile("vmcall"
+		     : "=a"(r)
+		     : "a"(nr), "b"(a0), "c"(a1), "d"(a2), "S"(a3));
+
 	return r;
 }
 
@@ -1314,8 +1321,10 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 
 void kvm_selftest_arch_init(void)
 {
+	is_cpu_amd = cpu_vendor_string_is("AuthenticAMD");
 }
 
 void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
 {
+	sync_global_to_guest(vm, is_cpu_amd);
 }
Signature · 1 line
2.37.2.789.g6183377224-goog
#16 lore raw

Re: [V2 PATCH 4/8] KVM: selftests: x86: Precompute the result for is_{intel,amd}_cpu()

Quoted 7 lines · Vishal Annapurve

Cache the vendor CPU type in a global variable so that multiple calls to is_intel_cpu() do not need to re-execute CPUID.

Add cpu vendor check in kvm_hypercall() so that it executes correct vmcall/vmmcall instruction when running on Intel/AMD hosts. This avoids exit to KVM which anyway tries to patch the instruction according to the cpu type.

The commit shortlog makes no mention (nor even implies) that this commit adds AMD support to kvm_hypercall(). Please break this commit up into 2. One to precompute the result of is_{intel,amd}_cpu() and one to add AMD support to kvm_hypercall().

If you really want to keep this as one commit (I don't know what the benefit would be), please change the shortlog and commit message to focus on the kvm_hypercall() change, as that is the real goal of this commit. The precomputation is arguably and implementation detail. e.g.

KVM: selftest: Add support for AMD to kvm_hypercall()

Make it possible to use kvm_hypercall() on AMD by checking if running on an AMD CPU and, if so, using vmmcall instead of vmcall. In order to avoid executing CPUID in the guest on every call t kvm_hypercall() (which would be slow), pre-compute the result of is_{intel,amd}_cpu() as part of kvm_selftest_arch_init() and sync it into the guest after loading the ELF image.

But again, it'd be cleaner just to split it up. Caching the result of is_{intel,amd}_cpu() is useful in its own right, independent of the kvm_hypercall() change.

Quoted 72 lines

As part of this change, sync the global variable is_cpu_amd into the guest so the guest can determine which hypercall instruction to use without needing to re-execute CPUID for every hypercall.

---
 .../testing/selftests/kvm/lib/x86_64/processor.c  | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/lib/x86_64/processor.c +12 -3
index 25ae972f5c71..c0ae938772f6 100644
@@ -19,6 +19,7 @@
 #define MAX_NR_CPUID_ENTRIES 100
 
 vm_vaddr_t exception_handlers;
+static bool is_cpu_amd;
 
 static void regs_dump(FILE *stream, struct kvm_regs *regs, uint8_t indent)
 {
@@ -1019,7 +1020,7 @@ static bool cpu_vendor_string_is(const char *vendor)
 
 bool is_intel_cpu(void)
 {
-	return cpu_vendor_string_is("GenuineIntel");
+	return (is_cpu_amd == false);
 }
 
 /*
@@ -1027,7 +1028,7 @@ bool is_intel_cpu(void)
  */
 bool is_amd_cpu(void)
 {
-	return cpu_vendor_string_is("AuthenticAMD");
+	return (is_cpu_amd == true);
 }
 
 void kvm_get_cpu_address_width(unsigned int *pa_bits, unsigned int *va_bits)
@@ -1182,9 +1183,15 @@ uint64_t kvm_hypercall(uint64_t nr, uint64_t a0, uint64_t a1, uint64_t a2,
 {
 	uint64_t r;
 
-	asm volatile("vmcall"
+	if (is_amd_cpu())
+		asm volatile("vmmcall"
 		     : "=a"(r)
 		     : "a"(nr), "b"(a0), "c"(a1), "d"(a2), "S"(a3));
+	else
+		asm volatile("vmcall"
+		     : "=a"(r)
+		     : "a"(nr), "b"(a0), "c"(a1), "d"(a2), "S"(a3));
+
 	return r;
 }
 
@@ -1314,8 +1321,10 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 
 void kvm_selftest_arch_init(void)
 {
+	is_cpu_amd = cpu_vendor_string_is("AuthenticAMD");
 }
 
 void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
 {
+	sync_global_to_guest(vm, is_cpu_amd);
 }
Signature · 1 line
2.37.2.789.g6183377224-goog
#17 lore raw

Re: [V2 PATCH 4/8] KVM: selftests: x86: Precompute the result for is_{intel,amd}_cpu()

Quoted 15 lines · David Matlack
Quoted 7 lines · Vishal Annapurve

Cache the vendor CPU type in a global variable so that multiple calls to is_intel_cpu() do not need to re-execute CPUID.

Add cpu vendor check in kvm_hypercall() so that it executes correct vmcall/vmmcall instruction when running on Intel/AMD hosts. This avoids exit to KVM which anyway tries to patch the instruction according to the cpu type.

The commit shortlog makes no mention (nor even implies) that this commit adds AMD support to kvm_hypercall(). Please break this commit up into 2. One to precompute the result of is_{intel,amd}_cpu() and one to add AMD support to kvm_hypercall().

Quoted 5 lines

If you really want to keep this as one commit (I don't know what the benefit would be), please change the shortlog and commit message to focus on the kvm_hypercall() change, as that is the real goal of this commit. The precomputation is arguably and implementation detail. e.g.

is_amd_cpu is used by guest code within fix_hypercall_test.c, just caching the result will break the guest code execution. I have clubbed these two changes together in order to ensure that is_amd_cpu works fine for both host userspace and guest vm logic.

Quoted 30 lines

KVM: selftest: Add support for AMD to kvm_hypercall()

Make it possible to use kvm_hypercall() on AMD by checking if running on an AMD CPU and, if so, using vmmcall instead of vmcall. In order to avoid executing CPUID in the guest on every call t kvm_hypercall() (which would be slow), pre-compute the result of is_{intel,amd}_cpu() as part of kvm_selftest_arch_init() and sync it into the guest after loading the ELF image.

But again, it'd be cleaner just to split it up. Caching the result of is_{intel,amd}_cpu() is useful in its own right, independent of the kvm_hypercall() change.

Quoted 17 lines

...

tools/testing/selftests/kvm/lib/x86_64/processor.c +2 -1
@@ -1314,8 +1321,10 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 
 void kvm_selftest_arch_init(void)
 {
+     is_cpu_amd = cpu_vendor_string_is("AuthenticAMD");
 }
 
 void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
 {
+     sync_global_to_guest(vm, is_cpu_amd);
 }
--

2.37.2.789.g6183377224-goog

#18 lore raw

Re: [V2 PATCH 4/8] KVM: selftests: x86: Precompute the result for is_{intel,amd}_cpu()

Quoted 23 lines · Vishal Annapurve
Quoted 15 lines · David Matlack
Quoted 7 lines · Vishal Annapurve

Cache the vendor CPU type in a global variable so that multiple calls to is_intel_cpu() do not need to re-execute CPUID.

Add cpu vendor check in kvm_hypercall() so that it executes correct vmcall/vmmcall instruction when running on Intel/AMD hosts. This avoids exit to KVM which anyway tries to patch the instruction according to the cpu type.

The commit shortlog makes no mention (nor even implies) that this commit adds AMD support to kvm_hypercall(). Please break this commit up into 2. One to precompute the result of is_{intel,amd}_cpu() and one to add AMD support to kvm_hypercall().

Quoted 5 lines

If you really want to keep this as one commit (I don't know what the benefit would be), please change the shortlog and commit message to focus on the kvm_hypercall() change, as that is the real goal of this commit. The precomputation is arguably and implementation detail. e.g.

Quoted 4 lines

is_amd_cpu is used by guest code within fix_hypercall_test.c, just caching the result will break the guest code execution. I have clubbed these two changes together in order to ensure that is_amd_cpu works fine for both host userspace and guest vm logic.

Ah, so the sync_global_to_guest() part needs to go in the patch that adds caching to is_amd_cpu().

But the point still stands that adding AMD support to kvm_hypercall() is a logically independent change.

Quoted 31 lines
Quoted 31 lines

KVM: selftest: Add support for AMD to kvm_hypercall()

Make it possible to use kvm_hypercall() on AMD by checking if running on an AMD CPU and, if so, using vmmcall instead of vmcall. In order to avoid executing CPUID in the guest on every call t kvm_hypercall() (which would be slow), pre-compute the result of is_{intel,amd}_cpu() as part of kvm_selftest_arch_init() and sync it into the guest after loading the ELF image.

But again, it'd be cleaner just to split it up. Caching the result of is_{intel,amd}_cpu() is useful in its own right, independent of the kvm_hypercall() change.

Quoted 17 lines

...

tools/testing/selftests/kvm/lib/x86_64/processor.c +2 -1
@@ -1314,8 +1321,10 @@ bool vm_is_unrestricted_guest(struct kvm_vm *vm)
 
 void kvm_selftest_arch_init(void)
 {
+     is_cpu_amd = cpu_vendor_string_is("AuthenticAMD");
 }
 
 void kvm_arch_post_vm_elf_load(struct kvm_vm *vm)
 {
+     sync_global_to_guest(vm, is_cpu_amd);
 }
--

2.37.2.789.g6183377224-goog

#19 lore raw

Re: [V2 PATCH 4/8] KVM: selftests: x86: Precompute the result for is_{intel,amd}_cpu()

Quoted 11 lines · David Matlack
Quoted 8 lines · Vishal Annapurve
Quoted 1 line · David Matlack

...

is_amd_cpu is used by guest code within fix_hypercall_test.c, just caching the result will break the guest code execution. I have clubbed these two changes together in order to ensure that is_amd_cpu works fine for both host userspace and guest vm logic.

Quoted 6 lines

Ah, so the sync_global_to_guest() part needs to go in the patch that adds caching to is_amd_cpu().

But the point still stands that adding AMD support to kvm_hypercall() is a logically independent change.

I see what you mean. Will split this change into two in the next series.

Quoted 4 lines
Quoted 4 lines

...

Quoted 3 lines
Quoted 3 lines

--
2.37.2.789.g6183377224-goog

#20 lore raw

Re: [V2 PATCH 4/8] KVM: selftests: x86: Precompute the result for is_{intel,amd}_cpu()

In reply to Vishal Annapurve
Quoted 7 lines · Vishal Annapurve

Cache the vendor CPU type in a global variable so that multiple calls to is_intel_cpu() do not need to re-execute CPUID.

Add cpu vendor check in kvm_hypercall() so that it executes correct vmcall/vmmcall instruction when running on Intel/AMD hosts. This avoids exit to KVM which anyway tries to patch the instruction according to the cpu type.

Out of curiousity, why do we want to avoid this exit?

#21 lore raw

Re: [V2 PATCH 4/8] KVM: selftests: x86: Precompute the result for is_{intel,amd}_cpu()

Quoted 11 lines · David Matlack
Quoted 7 lines · Vishal Annapurve

Cache the vendor CPU type in a global variable so that multiple calls to is_intel_cpu() do not need to re-execute CPUID.

Add cpu vendor check in kvm_hypercall() so that it executes correct vmcall/vmmcall instruction when running on Intel/AMD hosts. This avoids exit to KVM which anyway tries to patch the instruction according to the cpu type.

Out of curiousity, why do we want to avoid this exit?

Referring to the patch set posted for UPM selftests with non-confidential VMs [1], vmcall patching will not work for selftests executed with UPM feature enabled since guest memory can not be modified by KVM. So I tried to add a kvm_hypercall implementation that will execute the hypercall according to the cpu type.

Hypercall updates in this series are done to ensure that such a change is done for all callers to allow consistency and avoid relying on KVM behavior to patch the vmmcall/vmcall instruction.

[1] https://lore.kernel.org/lkml/[email protected]/

#22 lore raw

Re: [V2 PATCH 4/8] KVM: selftests: x86: Precompute the result for is_{intel,amd}_cpu()

Quoted 20 lines · Vishal Annapurve
Quoted 11 lines · David Matlack
Quoted 7 lines · Vishal Annapurve

Cache the vendor CPU type in a global variable so that multiple calls to is_intel_cpu() do not need to re-execute CPUID.

Add cpu vendor check in kvm_hypercall() so that it executes correct vmcall/vmmcall instruction when running on Intel/AMD hosts. This avoids exit to KVM which anyway tries to patch the instruction according to the cpu type.

Out of curiousity, why do we want to avoid this exit?

Referring to the patch set posted for UPM selftests with non-confidential VMs [1], vmcall patching will not work for selftests executed with UPM feature enabled since guest memory can not be modified by KVM. So I tried to add a kvm_hypercall implementation that will execute the hypercall according to the cpu type.

Quoted 5 lines

Hypercall updates in this series are done to ensure that such a change is done for all callers to allow consistency and avoid relying on KVM behavior to patch the vmmcall/vmcall instruction.

[1] https://lore.kernel.org/lkml/[email protected]/

Thanks! That makes a ton of sense. Please include that in the cover letter and any of the commit messages that avoid hypercall patching. That will help reviewers understand the context of why the changes are being made, and help anyone reviewing the git history in the future understand the same.

#23 lore raw

PATCH v2 5/8 KVM: selftests: x86: delete svm_vmcall_test

In reply to Vishal Annapurve

svm_vmcall_test is superseded by fix_hypercall_test.

---
 tools/testing/selftests/kvm/.gitignore        |  1 -
 .../selftests/kvm/x86_64/svm_vmcall_test.c    | 74 -------------------
 2 files changed, 75 deletions(-)
 delete mode 100644 tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
tools/testing/selftests/kvm/.gitignore +0 -1
index d625a3f83780..22e9a5b5488c 100644
@@ -36,7 +36,6 @@
 /x86_64/sev_migrate_tests
 /x86_64/smm_test
 /x86_64/state_test
-/x86_64/svm_vmcall_test
 /x86_64/svm_int_ctl_test
 /x86_64/svm_nested_soft_inject_test
 /x86_64/sync_regs_test
tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c deleted +0 -74

74 lines not on this page · raw message

Signature · 1 line
2.37.2.789.g6183377224-goog
#24 lore raw

Re: [V2 PATCH 5/8] KVM: selftests: x86: delete svm_vmcall_test

Quoted 1 line · Vishal Annapurve

svm_vmcall_test is superseded by fix_hypercall_test.

Please provide a more detailed description of why svm_vmcall_test is being dropped. e.g. What do you mean by "superseded" specifically?

I ask because this will be helpful context for future readers of this commit. I also ask because it's not clear to me that fix_hypercall_test is a 1:1 replacement of svm_vmcall_test. e.g. svm_vmcall_test sets up an L2 while fix_hypercall_test just executes in L1.

Quoted 104 lines
---
 tools/testing/selftests/kvm/.gitignore        |  1 -
 .../selftests/kvm/x86_64/svm_vmcall_test.c    | 74 -------------------
 2 files changed, 75 deletions(-)
 delete mode 100644 tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
tools/testing/selftests/kvm/.gitignore +0 -1
index d625a3f83780..22e9a5b5488c 100644
@@ -36,7 +36,6 @@
 /x86_64/sev_migrate_tests
 /x86_64/smm_test
 /x86_64/state_test
-/x86_64/svm_vmcall_test
 /x86_64/svm_int_ctl_test
 /x86_64/svm_nested_soft_inject_test
 /x86_64/sync_regs_test
tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c deleted +0 -74
deleted file mode 100644
index c3ac45df7483..000000000000
@@ -1,74 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * svm_vmcall_test
- *
- * Copyright (C) 2020, Red Hat, Inc.
- *
- * Nested SVM testing: VMCALL
- */
-
-#include "test_util.h"
-#include "kvm_util.h"
-#include "processor.h"
-#include "svm_util.h"
-
-static void l2_guest_code(struct svm_test_data *svm)
-{
-	__asm__ __volatile__("vmcall");
-}
-
-static void l1_guest_code(struct svm_test_data *svm)
-{
-	#define L2_GUEST_STACK_SIZE 64
-	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
-	struct vmcb *vmcb = svm->vmcb;
-
-	/* Prepare for L2 execution. */
-	generic_svm_setup(svm, l2_guest_code,
-			  &l2_guest_stack[L2_GUEST_STACK_SIZE]);
-
-	run_guest(vmcb, svm->vmcb_gpa);
-
-	GUEST_ASSERT(vmcb->control.exit_code == SVM_EXIT_VMMCALL);
-	GUEST_DONE();
-}
-
-int main(int argc, char *argv[])
-{
-	struct kvm_vcpu *vcpu;
-	vm_vaddr_t svm_gva;
-	struct kvm_vm *vm;
-
-	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SVM));
-
-	vm = vm_create_with_one_vcpu(&vcpu, l1_guest_code);
-
-	vcpu_alloc_svm(vm, &svm_gva);
-	vcpu_args_set(vcpu, 1, svm_gva);
-
-	for (;;) {
-		volatile struct kvm_run *run = vcpu->run;
-		struct ucall uc;
-
-		vcpu_run(vcpu);
-		TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
-			    "Got exit_reason other than KVM_EXIT_IO: %u (%s)\n",
-			    run->exit_reason,
-			    exit_reason_str(run->exit_reason));
-
-		switch (get_ucall(vcpu, &uc)) {
-		case UCALL_ABORT:
-			REPORT_GUEST_ASSERT(uc);
-			/* NOT REACHED */
-		case UCALL_SYNC:
-			break;
-		case UCALL_DONE:
-			goto done;
-		default:
-			TEST_FAIL("Unknown ucall 0x%lx.", uc.cmd);
-		}
-	}
-done:
-	kvm_vm_free(vm);
-	return 0;
-}
Signature · 1 line
2.37.2.789.g6183377224-goog
#25 lore raw

Re: [V2 PATCH 5/8] KVM: selftests: x86: delete svm_vmcall_test

Quoted 4 lines · David Matlack
Quoted 1 line · Vishal Annapurve

svm_vmcall_test is superseded by fix_hypercall_test.

Quoted 8 lines

Please provide a more detailed description of why svm_vmcall_test is being dropped. e.g. What do you mean by "superseded" specifically?

I ask because this will be helpful context for future readers of this commit. I also ask because it's not clear to me that fix_hypercall_test is a 1:1 replacement of svm_vmcall_test. e.g. svm_vmcall_test sets up an L2 while fix_hypercall_test just executes in L1.

You are right, I earlier wrongly interpreted it to be just testing vmcall, but it also does vmcall from L2.
I will exclude this change in the next series.

Quoted 7 lines
Quoted 7 lines

---
--
2.37.2.789.g6183377224-goog

#26 lore raw

PATCH v2 6/8 KVM: selftests: x86: Execute cpu specific hypercall from nested guests

In reply to Vishal Annapurve

Execute vmcall/vmmcall from nested guests according to the cpu type. This avoid exit to KVM which would anyway patch the hypercall instruction according to the cpu type.

---
 .../selftests/kvm/include/x86_64/processor.h      |  2 ++
 tools/testing/selftests/kvm/include/x86_64/vmx.h  |  9 ---------
 .../selftests/kvm/lib/x86_64/perf_test_util.c     |  2 +-
 .../testing/selftests/kvm/lib/x86_64/processor.c  | 15 +++++++++++++++
 tools/testing/selftests/kvm/x86_64/smm_test.c     |  2 +-
 tools/testing/selftests/kvm/x86_64/state_test.c   |  8 ++++----
 .../selftests/kvm/x86_64/vmx_dirty_log_test.c     |  2 +-
 .../kvm/x86_64/vmx_preemption_timer_test.c        |  2 +-
 8 files changed, 25 insertions(+), 17 deletions(-)
tools/testing/selftests/kvm/include/x86_64/processor.h +2 -0
index 0cbc71b7af50..18a8a6a2b786 100644
@@ -833,6 +833,8 @@ void vm_set_page_table_entry(struct kvm_vm *vm, struct kvm_vcpu *vcpu,
 uint64_t kvm_hypercall(uint64_t nr, uint64_t a0, uint64_t a1, uint64_t a2,
 		       uint64_t a3);
 
+void nested_guest_vmcall(void);
+
 void __vm_xsave_require_permission(int bit, const char *name);
 
 #define vm_xsave_require_permission(perm)	\
tools/testing/selftests/kvm/include/x86_64/vmx.h +0 -9
index 99fa1410964c..d8d4fd3353e5 100644
@@ -480,15 +480,6 @@ static inline int vmresume(void)
 	return ret;
 }
 
-static inline void vmcall(void)
-{
-	/* Currently, L1 destroys our GPRs during vmexits.  */
-	__asm__ __volatile__("push %%rbp; vmcall; pop %%rbp" : : :
-			     "rax", "rbx", "rcx", "rdx",
-			     "rsi", "rdi", "r8", "r9", "r10", "r11", "r12",
-			     "r13", "r14", "r15");
-}
-
 static inline int vmread(uint64_t encoding, uint64_t *value)
 {
 	uint64_t tmp;
tools/testing/selftests/kvm/lib/x86_64/perf_test_util.c +1 -1
index 0f344a7c89c4..b420b35b7f45 100644
@@ -18,7 +18,7 @@
 void perf_test_l2_guest_code(uint64_t vcpu_id)
 {
 	perf_test_guest_code(vcpu_id);
-	vmcall();
+	nested_guest_vmcall();
 }
 
 extern char perf_test_l2_guest_entry[];
tools/testing/selftests/kvm/lib/x86_64/processor.c +15 -0
index c0ae938772f6..e12c8b543b8f 100644
@@ -1195,6 +1195,21 @@ uint64_t kvm_hypercall(uint64_t nr, uint64_t a0, uint64_t a1, uint64_t a2,
 	return r;
 }
 
+void nested_guest_vmcall(void)
+{
+	/* Currently, L1 destroys our GPRs during vmexits.  */
+	if (is_amd_cpu())
+		__asm__ __volatile__("push %%rbp; vmmcall; pop %%rbp" : : :
+			     "rax", "rbx", "rcx", "rdx",
+			     "rsi", "rdi", "r8", "r9", "r10", "r11", "r12",
+			     "r13", "r14", "r15");
+	else
+		__asm__ __volatile__("push %%rbp; vmcall; pop %%rbp" : : :
+			     "rax", "rbx", "rcx", "rdx",
+			     "rsi", "rdi", "r8", "r9", "r10", "r11", "r12",
+			     "r13", "r14", "r15");
+}
+
 const struct kvm_cpuid2 *kvm_get_supported_hv_cpuid(void)
 {
 	static struct kvm_cpuid2 *cpuid;
tools/testing/selftests/kvm/x86_64/smm_test.c +1 -1
index 1f136a81858e..bf04c78c9c8e 100644
@@ -61,7 +61,7 @@ static void l2_guest_code(void)
 
 	sync_with_host(10);
 
-	vmcall();
+	nested_guest_vmcall();
 }
 
 static void guest_code(void *arg)
tools/testing/selftests/kvm/x86_64/state_test.c +4 -4
index ea578971fb9f..a9634c06dc60 100644
@@ -26,10 +26,10 @@ void svm_l2_guest_code(void)
 {
 	GUEST_SYNC(4);
 	/* Exit to L1 */
-	vmcall();
+	nested_guest_vmcall();
 	GUEST_SYNC(6);
 	/* Done, exit to L1 and never come back.  */
-	vmcall();
+	nested_guest_vmcall();
 }
 
 static void svm_l1_guest_code(struct svm_test_data *svm)
@@ -57,7 +57,7 @@ void vmx_l2_guest_code(void)
 	GUEST_SYNC(6);
 
 	/* Exit to L1 */
-	vmcall();
+	nested_guest_vmcall();
 
 	/* L1 has now set up a shadow VMCS for us.  */
 	GUEST_ASSERT(vmreadz(GUEST_RIP) == 0xc0ffee);
@@ -70,7 +70,7 @@ void vmx_l2_guest_code(void)
 	GUEST_SYNC(12);
 
 	/* Done, exit to L1 and never come back.  */
-	vmcall();
+	nested_guest_vmcall();
 }
 
 static void vmx_l1_guest_code(struct vmx_pages *vmx_pages)
tools/testing/selftests/kvm/x86_64/vmx_dirty_log_test.c +1 -1
index 2d8c23d639f7..fa24e69a806c 100644
@@ -42,7 +42,7 @@ static void l2_guest_code(void)
 	GUEST_SYNC(false);
 
 	/* Exit to L1 and never come back.  */
-	vmcall();
+	nested_guest_vmcall();
 }
 
 void l1_guest_code(struct vmx_pages *vmx)
tools/testing/selftests/kvm/x86_64/vmx_preemption_timer_test.c +1 -1
index 0efdc05969a5..04bae6995344 100644
@@ -38,7 +38,7 @@ void l2_guest_code(void)
 {
 	u64 vmx_pt_delta;
 
-	vmcall();
+	nested_guest_vmcall();
 	l2_vmx_pt_start = (rdtsc() >> vmx_pt_rate) << vmx_pt_rate;
 
 	/*
Signature · 1 line
2.37.2.789.g6183377224-goog
#27 lore raw

PATCH v2 7/8 Kvm: selftests: x86: Execute cpu specific vmcall instruction

In reply to Vishal Annapurve

Update the vmcall instruction invocation to happen according to the cpu type.

---
 tools/testing/selftests/kvm/include/x86_64/processor.h    | 8 ++++++++
 tools/testing/selftests/kvm/x86_64/vmx_apic_access_test.c | 2 +-
 .../selftests/kvm/x86_64/vmx_nested_tsc_scaling_test.c    | 2 +-
 tools/testing/selftests/kvm/x86_64/vmx_tsc_adjust_test.c  | 2 +-
 4 files changed, 11 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/include/x86_64/processor.h +8 -0
index 18a8a6a2b786..74893a7a80f8 100644
@@ -833,6 +833,14 @@ void vm_set_page_table_entry(struct kvm_vm *vm, struct kvm_vcpu *vcpu,
 uint64_t kvm_hypercall(uint64_t nr, uint64_t a0, uint64_t a1, uint64_t a2,
 		       uint64_t a3);
 
+/*
+ * Execute vmcall instruction.
+ */
+static inline void vmcall(void)
+{
+	kvm_hypercall(0, 0, 0, 0, 0);
+}
+
 void nested_guest_vmcall(void);
 
 void __vm_xsave_require_permission(int bit, const char *name);
tools/testing/selftests/kvm/x86_64/vmx_apic_access_test.c +1 -1
index 5abecf06329e..8180711c8684 100644
@@ -31,7 +31,7 @@
 static void l2_guest_code(void)
 {
 	/* Exit to L1 */
-	__asm__ __volatile__("vmcall");
+	vmcall();
 }
 
 static void l1_guest_code(struct vmx_pages *vmx_pages, unsigned long high_gpa)
tools/testing/selftests/kvm/x86_64/vmx_nested_tsc_scaling_test.c +1 -1
index 465a9434d61c..37da9d01d5d6 100644
@@ -76,7 +76,7 @@ static void l2_guest_code(void)
 	check_tsc_freq(UCHECK_L2);
 
 	/* exit to L1 */
-	__asm__ __volatile__("vmcall");
+	vmcall();
 }
 
 static void l1_guest_code(struct vmx_pages *vmx_pages)
tools/testing/selftests/kvm/x86_64/vmx_tsc_adjust_test.c +1 -1
index 5943187e8594..00192f564d9b 100644
@@ -74,7 +74,7 @@ static void l2_guest_code(void)
 	check_ia32_tsc_adjust(-2 * TSC_ADJUST_VALUE);
 
 	/* Exit to L1 */
-	__asm__ __volatile__("vmcall");
+	vmcall();
 }
 
 static void l1_guest_code(struct vmx_pages *vmx_pages)
Signature · 1 line
2.37.2.789.g6183377224-goog
#28 lore raw

Re: [V2 PATCH 7/8] Kvm: selftests: x86: Execute cpu specific vmcall instruction

Quoted 10 lines · Vishal Annapurve

Update the vmcall instruction invocation to happen according to the cpu type.

---
 tools/testing/selftests/kvm/include/x86_64/processor.h    | 8 ++++++++
 tools/testing/selftests/kvm/x86_64/vmx_apic_access_test.c | 2 +-
 .../selftests/kvm/x86_64/vmx_nested_tsc_scaling_test.c    | 2 +-
 tools/testing/selftests/kvm/x86_64/vmx_tsc_adjust_test.c  | 2 +-

What's the reason to use kvm_hypercall() for these tests? All of these are Intel-specific. i.e. is_amd_cpu() will always return false.

Quoted 63 lines
 4 files changed, 11 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/include/x86_64/processor.h +8 -0
index 18a8a6a2b786..74893a7a80f8 100644
@@ -833,6 +833,14 @@ void vm_set_page_table_entry(struct kvm_vm *vm, struct kvm_vcpu *vcpu,
 uint64_t kvm_hypercall(uint64_t nr, uint64_t a0, uint64_t a1, uint64_t a2,
 		       uint64_t a3);
 
+/*
+ * Execute vmcall instruction.
+ */
+static inline void vmcall(void)
+{
+	kvm_hypercall(0, 0, 0, 0, 0);
+}
+
 void nested_guest_vmcall(void);
 
 void __vm_xsave_require_permission(int bit, const char *name);
tools/testing/selftests/kvm/x86_64/vmx_apic_access_test.c +1 -1
index 5abecf06329e..8180711c8684 100644
@@ -31,7 +31,7 @@
 static void l2_guest_code(void)
 {
 	/* Exit to L1 */
-	__asm__ __volatile__("vmcall");
+	vmcall();
 }
 
 static void l1_guest_code(struct vmx_pages *vmx_pages, unsigned long high_gpa)
tools/testing/selftests/kvm/x86_64/vmx_nested_tsc_scaling_test.c +1 -1
index 465a9434d61c..37da9d01d5d6 100644
@@ -76,7 +76,7 @@ static void l2_guest_code(void)
 	check_tsc_freq(UCHECK_L2);
 
 	/* exit to L1 */
-	__asm__ __volatile__("vmcall");
+	vmcall();
 }
 
 static void l1_guest_code(struct vmx_pages *vmx_pages)
tools/testing/selftests/kvm/x86_64/vmx_tsc_adjust_test.c +1 -1
index 5943187e8594..00192f564d9b 100644
@@ -74,7 +74,7 @@ static void l2_guest_code(void)
 	check_ia32_tsc_adjust(-2 * TSC_ADJUST_VALUE);
 
 	/* Exit to L1 */
-	__asm__ __volatile__("vmcall");
+	vmcall();
 }
 
 static void l1_guest_code(struct vmx_pages *vmx_pages)
Signature · 1 line
2.37.2.789.g6183377224-goog
#29 lore raw

Re: [V2 PATCH 7/8] Kvm: selftests: x86: Execute cpu specific vmcall instruction

Quoted 13 lines · David Matlack
Quoted 10 lines · Vishal Annapurve

Update the vmcall instruction invocation to happen according to the cpu type.

---
 tools/testing/selftests/kvm/include/x86_64/processor.h    | 8 ++++++++
 tools/testing/selftests/kvm/x86_64/vmx_apic_access_test.c | 2 +-
 .../selftests/kvm/x86_64/vmx_nested_tsc_scaling_test.c    | 2 +-
 tools/testing/selftests/kvm/x86_64/vmx_tsc_adjust_test.c  | 2 +-
Quoted 3 lines

What's the reason to use kvm_hypercall() for these tests? All of these are Intel-specific. i.e. is_amd_cpu() will always return false.

That's right. This change is attempting to have hypercalls from guest code done via a common API as far as possible.

Quoted 7 lines
Quoted 7 lines
 4 files changed, 11 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/include/x86_64/processor.h +0 -1
...
--

2.37.2.789.g6183377224-goog

#30 lore raw

PATCH v2 8/8 KVM: selftests: x86: xen: Execute cpu specific vmcall instruction

In reply to Vishal Annapurve

Update xen specific hypercall invocation to execute cpu specific vmcall instructions.

---
 .../selftests/kvm/x86_64/xen_shinfo_test.c    | 64 +++++++------------
 .../selftests/kvm/x86_64/xen_vmcall_test.c    | 14 ++--
 2 files changed, 34 insertions(+), 44 deletions(-)
tools/testing/selftests/kvm/x86_64/xen_shinfo_test.c +24 -40
index 8a5cb800f50e..92ed07f1c772 100644
@@ -145,6 +145,23 @@ static void guest_wait_for_irq(void)
 	guest_saw_irq = false;
 }
 
+static unsigned long vmcall_helper(unsigned long reg_a, unsigned long reg_di,
+	unsigned long reg_si)
+{
+	unsigned long ret;
+
+	if (is_amd_cpu())
+		__asm__ __volatile__ ("vmmcall" :
+			"=a" (ret) :
+			"a" (reg_a), "D" (reg_di), "S" (reg_si));
+	else
+		__asm__ __volatile__ ("vmcall" :
+			"=a" (ret) :
+			"a" (reg_a), "D" (reg_di), "S" (reg_si));
+
+	return ret;
+}
+
 static void guest_code(void)
 {
 	struct vcpu_runstate_info *rs = (void *)RUNSTATE_VADDR;
@@ -217,12 +234,7 @@ static void guest_code(void)
 	 * EVTCHNOP_send hypercall. */
 	unsigned long rax;
 	struct evtchn_send s = { .port = 127 };
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_event_channel_op),
-			      "D" (EVTCHNOP_send),
-			      "S" (&s));
-
+	rax = vmcall_helper(__HYPERVISOR_event_channel_op, EVTCHNOP_send, (unsigned long)&s);
 	GUEST_ASSERT(rax == 0);
 
 	guest_wait_for_irq();
@@ -232,12 +244,7 @@ static void guest_code(void)
 	/* Deliver "outbound" event channel to an eventfd which
 	 * happens to be one of our own irqfds. */
 	s.port = 197;
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_event_channel_op),
-			      "D" (EVTCHNOP_send),
-			      "S" (&s));
-
+	rax = vmcall_helper(__HYPERVISOR_event_channel_op, EVTCHNOP_send, (unsigned long)&s);
 	GUEST_ASSERT(rax == 0);
 
 	guest_wait_for_irq();
@@ -245,10 +252,7 @@ static void guest_code(void)
 	GUEST_SYNC(13);
 
 	/* Set a timer 100ms in the future. */
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_set_timer_op),
-			      "D" (rs->state_entry_time + 100000000));
+	rax = vmcall_helper(__HYPERVISOR_set_timer_op, (rs->state_entry_time + 100000000), 0);
 	GUEST_ASSERT(rax == 0);
 
 	GUEST_SYNC(14);
@@ -271,36 +275,21 @@ static void guest_code(void)
 		.timeout = 0,
 	};
 
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_sched_op),
-			      "D" (SCHEDOP_poll),
-			      "S" (&p));
-
+	rax = vmcall_helper(__HYPERVISOR_sched_op, SCHEDOP_poll, (unsigned long)&p);
 	GUEST_ASSERT(rax == 0);
 
 	GUEST_SYNC(17);
 
 	/* Poll for an unset port and wait for the timeout. */
 	p.timeout = 100000000;
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_sched_op),
-			      "D" (SCHEDOP_poll),
-			      "S" (&p));
-
+	rax = vmcall_helper(__HYPERVISOR_sched_op, SCHEDOP_poll, (unsigned long)&p);
 	GUEST_ASSERT(rax == 0);
 
 	GUEST_SYNC(18);
 
 	/* A timer will wake the masked port we're waiting on, while we poll */
 	p.timeout = 0;
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_sched_op),
-			      "D" (SCHEDOP_poll),
-			      "S" (&p));
-
+	rax = vmcall_helper(__HYPERVISOR_sched_op, SCHEDOP_poll, (unsigned long)&p);
 	GUEST_ASSERT(rax == 0);
 
 	GUEST_SYNC(19);
@@ -309,12 +298,7 @@ static void guest_code(void)
 	 * actual interrupt, while we're polling on a different port. */
 	ports[0]++;
 	p.timeout = 0;
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_sched_op),
-			      "D" (SCHEDOP_poll),
-			      "S" (&p));
-
+	rax = vmcall_helper(__HYPERVISOR_sched_op, SCHEDOP_poll, (unsigned long)&p);
 	GUEST_ASSERT(rax == 0);
 
 	guest_wait_for_irq();
tools/testing/selftests/kvm/x86_64/xen_vmcall_test.c +10 -4
index 88914d48c65e..e78f1b5d3af8 100644
@@ -37,10 +37,16 @@ static void guest_code(void)
 	register unsigned long r9 __asm__("r9") = ARGVALUE(6);
 
 	/* First a direct invocation of 'vmcall' */
-	__asm__ __volatile__("vmcall" :
-			     "=a"(rax) :
-			     "a"(rax), "D"(rdi), "S"(rsi), "d"(rdx),
-			     "r"(r10), "r"(r8), "r"(r9));
+	if (is_amd_cpu())
+		__asm__ __volatile__("vmmcall" :
+			"=a"(rax) :
+			"a"(rax), "D"(rdi), "S"(rsi), "d"(rdx),
+			"r"(r10), "r"(r8), "r"(r9));
+	else
+		__asm__ __volatile__("vmcall" :
+			"=a"(rax) :
+			"a"(rax), "D"(rdi), "S"(rsi), "d"(rdx),
+			"r"(r10), "r"(r8), "r"(r9));
 	GUEST_ASSERT(rax == RETVALUE);
 
 	/* Fill in the Xen hypercall page */
Signature · 1 line
2.37.2.789.g6183377224-goog
#31 lore raw

Re: [V2 PATCH 8/8] KVM: selftests: x86: xen: Execute cpu specific vmcall instruction

Quoted 144 lines · Vishal Annapurve

Update xen specific hypercall invocation to execute cpu specific vmcall instructions.

---
 .../selftests/kvm/x86_64/xen_shinfo_test.c    | 64 +++++++------------
 .../selftests/kvm/x86_64/xen_vmcall_test.c    | 14 ++--
 2 files changed, 34 insertions(+), 44 deletions(-)
tools/testing/selftests/kvm/x86_64/xen_shinfo_test.c +24 -40
index 8a5cb800f50e..92ed07f1c772 100644
@@ -145,6 +145,23 @@ static void guest_wait_for_irq(void)
 	guest_saw_irq = false;
 }
 
+static unsigned long vmcall_helper(unsigned long reg_a, unsigned long reg_di,
+	unsigned long reg_si)
+{
+	unsigned long ret;
+
+	if (is_amd_cpu())
+		__asm__ __volatile__ ("vmmcall" :
+			"=a" (ret) :
+			"a" (reg_a), "D" (reg_di), "S" (reg_si));
+	else
+		__asm__ __volatile__ ("vmcall" :
+			"=a" (ret) :
+			"a" (reg_a), "D" (reg_di), "S" (reg_si));
+
+	return ret;
+}
+
 static void guest_code(void)
 {
 	struct vcpu_runstate_info *rs = (void *)RUNSTATE_VADDR;
@@ -217,12 +234,7 @@ static void guest_code(void)
 	 * EVTCHNOP_send hypercall. */
 	unsigned long rax;
 	struct evtchn_send s = { .port = 127 };
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_event_channel_op),
-			      "D" (EVTCHNOP_send),
-			      "S" (&s));
-
+	rax = vmcall_helper(__HYPERVISOR_event_channel_op, EVTCHNOP_send, (unsigned long)&s);
 	GUEST_ASSERT(rax == 0);
 
 	guest_wait_for_irq();
@@ -232,12 +244,7 @@ static void guest_code(void)
 	/* Deliver "outbound" event channel to an eventfd which
 	 * happens to be one of our own irqfds. */
 	s.port = 197;
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_event_channel_op),
-			      "D" (EVTCHNOP_send),
-			      "S" (&s));
-
+	rax = vmcall_helper(__HYPERVISOR_event_channel_op, EVTCHNOP_send, (unsigned long)&s);
 	GUEST_ASSERT(rax == 0);
 
 	guest_wait_for_irq();
@@ -245,10 +252,7 @@ static void guest_code(void)
 	GUEST_SYNC(13);
 
 	/* Set a timer 100ms in the future. */
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_set_timer_op),
-			      "D" (rs->state_entry_time + 100000000));
+	rax = vmcall_helper(__HYPERVISOR_set_timer_op, (rs->state_entry_time + 100000000), 0);
 	GUEST_ASSERT(rax == 0);
 
 	GUEST_SYNC(14);
@@ -271,36 +275,21 @@ static void guest_code(void)
 		.timeout = 0,
 	};
 
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_sched_op),
-			      "D" (SCHEDOP_poll),
-			      "S" (&p));
-
+	rax = vmcall_helper(__HYPERVISOR_sched_op, SCHEDOP_poll, (unsigned long)&p);
 	GUEST_ASSERT(rax == 0);
 
 	GUEST_SYNC(17);
 
 	/* Poll for an unset port and wait for the timeout. */
 	p.timeout = 100000000;
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_sched_op),
-			      "D" (SCHEDOP_poll),
-			      "S" (&p));
-
+	rax = vmcall_helper(__HYPERVISOR_sched_op, SCHEDOP_poll, (unsigned long)&p);
 	GUEST_ASSERT(rax == 0);
 
 	GUEST_SYNC(18);
 
 	/* A timer will wake the masked port we're waiting on, while we poll */
 	p.timeout = 0;
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_sched_op),
-			      "D" (SCHEDOP_poll),
-			      "S" (&p));
-
+	rax = vmcall_helper(__HYPERVISOR_sched_op, SCHEDOP_poll, (unsigned long)&p);
 	GUEST_ASSERT(rax == 0);
 
 	GUEST_SYNC(19);
@@ -309,12 +298,7 @@ static void guest_code(void)
 	 * actual interrupt, while we're polling on a different port. */
 	ports[0]++;
 	p.timeout = 0;
-	__asm__ __volatile__ ("vmcall" :
-			      "=a" (rax) :
-			      "a" (__HYPERVISOR_sched_op),
-			      "D" (SCHEDOP_poll),
-			      "S" (&p));
-
+	rax = vmcall_helper(__HYPERVISOR_sched_op, SCHEDOP_poll, (unsigned long)&p);
 	GUEST_ASSERT(rax == 0);
 
 	guest_wait_for_irq();
tools/testing/selftests/kvm/x86_64/xen_vmcall_test.c +0 -4
index 88914d48c65e..e78f1b5d3af8 100644
@@ -37,10 +37,16 @@ static void guest_code(void)
 	register unsigned long r9 __asm__("r9") = ARGVALUE(6);
 
 	/* First a direct invocation of 'vmcall' */
-	__asm__ __volatile__("vmcall" :
-			     "=a"(rax) :
-			     "a"(rax), "D"(rdi), "S"(rsi), "d"(rdx),
-			     "r"(r10), "r"(r8), "r"(r9));

tools/testing/selftests/kvm/x86_64/xen_vmcall_test.c static void guest_code(void) · line 49

Quoted 10 lines
tools/testing/selftests/kvm/x86_64/xen_vmcall_test.c +10 -0
+	if (is_amd_cpu())
+		__asm__ __volatile__("vmmcall" :
+			"=a"(rax) :
+			"a"(rax), "D"(rdi), "S"(rsi), "d"(rdx),
+			"r"(r10), "r"(r8), "r"(r9));
+	else
+		__asm__ __volatile__("vmcall" :
+			"=a"(rax) :
+			"a"(rax), "D"(rdi), "S"(rsi), "d"(rdx),
+			"r"(r10), "r"(r8), "r"(r9));

Can we create common helper functions or macros for doing hypercalls to reduce the amount of duplicated inline assembly?

Quoted 6 lines
tools/testing/selftests/kvm/x86_64/xen_vmcall_test.c +0 -0
 	GUEST_ASSERT(rax == RETVALUE);
 
 	/* Fill in the Xen hypercall page */
Signature · 1 line
2.37.2.789.g6183377224-goog
#32 lore raw

Re: [V2 PATCH 8/8] KVM: selftests: x86: xen: Execute cpu specific vmcall instruction

Quoted 29 lines · David Matlack
Quoted 27 lines · Vishal Annapurve

Update xen specific hypercall invocation to execute cpu specific vmcall instructions.

---
 .../selftests/kvm/x86_64/xen_shinfo_test.c    | 64 +++++++------------
 .../selftests/kvm/x86_64/xen_vmcall_test.c    | 14 ++--
 2 files changed, 34 insertions(+), 44 deletions(-)

...

tools/testing/selftests/kvm/x86_64/xen_vmcall_test.c +5 -4
index 88914d48c65e..e78f1b5d3af8 100644
@@ -37,10 +37,16 @@ static void guest_code(void)
      register unsigned long r9 __asm__("r9") = ARGVALUE(6);
 
      /* First a direct invocation of 'vmcall' */
-     __asm__ __volatile__("vmcall" :
-                          "=a"(rax) :
-                          "a"(rax), "D"(rdi), "S"(rsi), "d"(rdx),
-                          "r"(r10), "r"(r8), "r"(r9));
+     if (is_amd_cpu())
+             __asm__ __volatile__("vmmcall" :
+                     "=a"(rax) :
+                     "a"(rax), "D"(rdi), "S"(rsi), "d"(rdx),
+                     "r"(r10), "r"(r8), "r"(r9));
Quoted 9 lines

tools/testing/selftests/kvm/x86_64/xen_vmcall_test.c static void guest_code(void) · line 49

Quoted 5 lines
tools/testing/selftests/kvm/x86_64/xen_vmcall_test.c +5 -0
+     else
+             __asm__ __volatile__("vmcall" :
+                     "=a"(rax) :
+                     "a"(rax), "D"(rdi), "S"(rsi), "d"(rdx),
+                     "r"(r10), "r"(r8), "r"(r9));

Can we create common helper functions or macros for doing hypercalls to reduce the amount of duplicated inline assembly?

Ack, will fix this in the next series.

Quoted 6 lines
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/xen_vmcall_test.c +0 -1
      GUEST_ASSERT(rax == RETVALUE);
 
      /* Fill in the Xen hypercall page */
--

2.37.2.789.g6183377224-goog