Skip to contentSkip to conversation

KVM: selftests: Add a test for gPAT handling in L2

PATCH In review Yosry Ahmed 2 participants 4 messages May 21 – May 28

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 KVM: selftests: Add a test for gPAT handling in L2

When KVM_X86_QUIRK_NESTED_SVM_SHARED_PAT is disabled, verify that KVM correctly virtualizes the host PAT MSR and the guest PAT register for nested SVM guests.

With nested NPT disabled:
* L1 and L2 share the same PAT
* The vmcb12.g_pat is ignored

With nested NPT enabled:
* An invalid g_pat in vmcb12 causes VMEXIT_INVALID
* RDMSR(IA32_PAT) from L2 returns the value of the guest PAT register
* WRMSR(IA32_PAT) from L2 is reflected in vmcb12's g_pat on VMEXIT
* RDMSR(IA32_PAT) from L1 returns the value of the host PAT MSR
* Save/restore with the vCPU in guest mode preserves both hPAT and gPAT

---

v2 -> v3:
- Fixed multiple VM-Entries test case.
- Fixed typos.

v1 -> v2:
- Rewrote most of the test to dedup L1 and L2 code, move assertions to L2 where possible to simplify the test, and drop the shared test struct.

v1 is essentially the patch in v7 of Jim's gPAT series [*] [*]https://lore.kernel.org/kvm/[email protected]/

---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/svm_nested_pat_test.c   | 202 ++++++++++++++++++
 2 files changed, 203 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/svm_nested_pat_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 9118a5a51b89f..d658dc7cd2b14 100644
@@ -117,6 +117,7 @@ TEST_GEN_PROGS_x86 += x86/svm_nested_clear_efer_svme
 TEST_GEN_PROGS_x86 += x86/svm_nested_shutdown_test
 TEST_GEN_PROGS_x86 += x86/svm_nested_soft_inject_test
 TEST_GEN_PROGS_x86 += x86/svm_nested_vmcb12_gpa
+TEST_GEN_PROGS_x86 += x86/svm_nested_pat_test
 TEST_GEN_PROGS_x86 += x86/svm_lbr_nested_state
 TEST_GEN_PROGS_x86 += x86/tsc_scaling_sync
 TEST_GEN_PROGS_x86 += x86/sync_regs_test
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +202 -0

202 lines not on this page · raw message

base-commit: 4f256d5770febb9d61f9b57a4c79c491bf4987f1

Signature · 1 line
2.54.0.794.g4f17f83d09-goog
#1 lore raw
Quoted 41 lines · Yosry Ahmed
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +41 -0
+static void l1_guest_code_invalid_gpat(struct svm_test_data *svm)
+{
+	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
+	struct vmcb *vmcb = svm->vmcb;
+
+	/* VMRUN should fail without running L2 */
+	generic_svm_setup(svm, NULL, &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	vmcb->save.g_pat = INVALID_PAT_VALUE;
+	run_guest(vmcb, svm->vmcb_gpa);
+
+	GUEST_ASSERT_EQ(vmcb->control.exit_code, SVM_EXIT_ERR);
+	GUEST_DONE();
+}
+
+static void run_test(const char *test_name, void *guest_code, bool do_save_restore)
+{
+	struct kvm_x86_state *state;
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	struct ucall uc;
+	gva_t svm_gva;
+
+	pr_info("Testing: %s\n", test_name);
+
+	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+	vm_enable_cap(vm, KVM_CAP_DISABLE_QUIRKS2,
+		      KVM_X86_QUIRK_NESTED_SVM_SHARED_PAT);
+
+	if (npt_enabled) {
+		vm_enable_npt(vm);
+		tdp_identity_map_default_memslots(vm);
+	}
+
+	vcpu_alloc_svm(vm, &svm_gva);
+	vcpu_args_set(vcpu, 1, svm_gva);
+	sync_global_to_guest(vm, npt_enabled);
+	sync_global_to_guest(vm, nr_iterations);
+
+	for (;;) {
+		vcpu_run(vcpu);

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 145

Quoted 10 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +10 -0
+		TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO);
+
+		switch (get_ucall(vcpu, &uc)) {
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			/* NOT REACHED */
+		case UCALL_SYNC:
+			if (do_save_restore) {
+				pr_info("  Save/restore at sync point %ld\n",
+					uc.args[1]);

Meh, this is noise to me. Maybe it's marginally useful if there's a failure, but I don't see how it can possibly provide enough information to have to avoid hacking on the test.

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 156

Quoted 11 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +11 -0
+				state = vcpu_save_state(vcpu);
+				kvm_vm_release(vm);
+				vcpu = vm_recreate_with_one_vcpu(vm);
+				vm_enable_cap(vm, KVM_CAP_DISABLE_QUIRKS2,
+					      KVM_X86_QUIRK_NESTED_SVM_SHARED_PAT);
+				vcpu_load_state(vcpu, state);
+				kvm_x86_state_cleanup(state);
+			}
+			break;
+		case UCALL_DONE:
+			pr_info("  PASSED\n");

This is useless and noisy. The test runs in ~.25 seconds, and most of that is the printks. If there's a failure, I'll see. Otherwise it's a pretty safe assumption the test passed.

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 169

Quoted 13 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +13 -0
+			kvm_vm_free(vm);
+			return;
+		default:
+			TEST_FAIL("Unknown ucall %lu", uc.cmd);
+		}
+	}
+}
+
+#define NPT_DISABLED 0
+#define NPT_ENABLED 1
+
+#define NO_SAVE_RESTORE 0
+#define DO_SAVE_RESTORE 1

Eww.

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 172

Quoted 3 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +3 -0
+
+#define NESTED_PAT_TEST(test_name, guest_code, npt, sr, iter)	\
+({								\

Unless you need to "return" a value, do-while (0) is preferred over ({}). There's a technical reason, but I can't remember it off the top of my head.

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 175

Quoted 3 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +3 -0
+	npt_enabled = npt;					\
+	nr_iterations = iter;					\
+	run_test(test_name, guest_code, sr);			\

LOL, all of this, and the one thing that _doesn't_ use macro shenanigans is string concatenation.

Quoted 14 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +14 -0
+})
+
+int main(int argc, char *argv[])
+{
+	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SVM));
+	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_NPT));
+	TEST_REQUIRE(kvm_has_cap(KVM_CAP_NESTED_STATE));
+	TEST_REQUIRE(kvm_check_cap(KVM_CAP_DISABLE_QUIRKS2) &
+		     KVM_X86_QUIRK_NESTED_SVM_SHARED_PAT);
+
+	NESTED_PAT_TEST("Invalid gPAT", l1_guest_code_invalid_gpat,
+			NPT_ENABLED, NO_SAVE_RESTORE, 1);
+
+	NESTED_PAT_TEST("Nested NPT disabled", l1_guest_code,

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 199

Quoted 10 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +10 -0
+			NPT_DISABLED, NO_SAVE_RESTORE, 1);
+
+	NESTED_PAT_TEST("Nested NPT enabled", l1_guest_code,
+			NPT_ENABLED, NO_SAVE_RESTORE, 1);
+
+	NESTED_PAT_TEST("Save/Restore", l1_guest_code,
+			NPT_ENABLED, DO_SAVE_RESTORE, 1);
+
+	NESTED_PAT_TEST("Multiple VM-Entries", l1_guest_code,
+			NPT_ENABLED, NO_SAVE_RESTORE, 10);

I appreciated trying to avoid true/false literals, but this is harder to read. At least "true" and "false" are visually different, {N,D}O_SAVE_RESTORE are one char, and NPT_{DIS,EN}ABLED are two.

And explicitly defining the number of iterations and whether or not to do save/restore is silly. It requires _more_ code to provide _less_ coverage. Just run the combos for npt_enabled=false.

If we're going to use macro shenanigans, let's actually use them! I've got this working, I'll post a v4.

#define gpat_test(test_name, guest_code, npt_input) \

do {									\
	npt_input;							\
									\
	pr_info("Testing: " test_name "\n");				\
	run_test(guest_code, false, 1);					\
									\
	if (guest_code == l1_guest_code) {				\
		pr_info("Testing: " test_name " Save/Restore\n");	\
		run_test(guest_code, true, 1);				\
									\
		pr_info("Testing: " test_name " Multiple VMRUNs\n");	\
		run_test(guest_code, false, 10);			\
	}								\
} while (0)

int main(int argc, char *argv[])

{
	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SVM));
	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_NPT));
	TEST_REQUIRE(kvm_has_cap(KVM_CAP_NESTED_STATE));
	TEST_REQUIRE(kvm_check_cap(KVM_CAP_DISABLE_QUIRKS2) &
		     KVM_X86_QUIRK_NESTED_SVM_SHARED_PAT);

	gpat_test("Invalid gPAT", l1_guest_code_invalid_gpat, npt_enabled = true);
	gpat_test("Nested NPT disabled", l1_guest_code, npt_enabled = false);
	gpat_test("Nested NPT enabled", l1_guest_code, npt_enabled = true);

	return 0;
}
Quoted 8 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +3 -0
+
+	return 0;
+}

base-commit: 4f256d5770febb9d61f9b57a4c79c491bf4987f1

Signature · 1 line
2.54.0.794.g4f17f83d09-goog
#2 lore raw
Quoted 8 lines

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 145

Quoted 4 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +4 -0
+             case UCALL_SYNC:
+                     if (do_save_restore) {
+                             pr_info("  Save/restore at sync point %ld\n",
+                                     uc.args[1]);

Meh, this is noise to me. Maybe it's marginally useful if there's a failure, but I don't see how it can possibly provide enough information to have to avoid hacking on the test.

I kept the logging in the test mostly as-is from the original patch, I don't have a preference.

[..]

Quoted 6 lines

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 156

Quoted 2 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +2 -0
+             case UCALL_DONE:
+                     pr_info("  PASSED\n");

This is useless and noisy. The test runs in ~.25 seconds, and most of that is the printks. If there's a failure, I'll see. Otherwise it's a pretty safe assumption the test passed.

Same here.

[..]

Quoted 7 lines

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 169

Quoted 5 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +5 -0
+#define NPT_DISABLED 0
+#define NPT_ENABLED 1
+
+#define NO_SAVE_RESTORE 0
+#define DO_SAVE_RESTORE 1

Eww.

I can't say that didn't hurt.

Quoted 7 lines

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 172

Quoted 4 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +3 -0
+
+#define NESTED_PAT_TEST(test_name, guest_code, npt, sr, iter)        \
+({                                                           \

Unless you need to "return" a value, do-while (0) is preferred over ({}). There's a technical reason, but I can't remember it off the top of my head.

Yeah I also couldn't remember it so I convinced myself that older compilers just couldn't handle ({}).

Looking at https://kernelnewbies.org/FAQ/DoWhile0, the last section seems to corroborate my story?

Quoted 28 lines

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 175

Quoted 4 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +3 -0
+     npt_enabled = npt;                                      \
+     nr_iterations = iter;                                   \
+     run_test(test_name, guest_code, sr);                    \

LOL, all of this, and the one thing that _doesn't_ use macro shenanigans is string concatenation.

Quoted 20 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +20 -0
+})
+
+int main(int argc, char *argv[])
+{
+     TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SVM));
+     TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_NPT));
+     TEST_REQUIRE(kvm_has_cap(KVM_CAP_NESTED_STATE));
+     TEST_REQUIRE(kvm_check_cap(KVM_CAP_DISABLE_QUIRKS2) &
+                  KVM_X86_QUIRK_NESTED_SVM_SHARED_PAT);
+
+     NESTED_PAT_TEST("Invalid gPAT", l1_guest_code_invalid_gpat,
+                     NPT_ENABLED, NO_SAVE_RESTORE, 1);
+
+     NESTED_PAT_TEST("Nested NPT disabled", l1_guest_code,
+                     NPT_DISABLED, NO_SAVE_RESTORE, 1);
+
+     NESTED_PAT_TEST("Nested NPT enabled", l1_guest_code,
+                     NPT_ENABLED, NO_SAVE_RESTORE, 1);
+
+     NESTED_PAT_TEST("Save/Restore", l1_guest_code,
Quoted 8 lines

tools/testing/selftests/kvm/x86/svm_nested_pat_test.c line 199

Quoted 4 lines
tools/testing/selftests/kvm/x86/svm_nested_pat_test.c new file +4 -0
+                     NPT_ENABLED, DO_SAVE_RESTORE, 1);
+
+     NESTED_PAT_TEST("Multiple VM-Entries", l1_guest_code,
+                     NPT_ENABLED, NO_SAVE_RESTORE, 10);

I appreciated trying to avoid true/false literals, but this is harder to read. At least "true" and "false" are visually different, {N,D}O_SAVE_RESTORE are one char, and NPT_{DIS,EN}ABLED are two.

That's fair.

Quoted 4 lines

And explicitly defining the number of iterations and whether or not to do save/restore is silly. It requires _more_ code to provide _less_ coverage. Just run the combos for npt_enabled=false.

Also fair.

Quoted 7 lines

If we're going to use macro shenanigans, let's actually use them! I've got this working, I'll post a v4.

#define gpat_test(test_name, guest_code, npt_input)                     \
do {                                                                    \
        npt_input;                                                      \

Oh I like this very much. Passing in "npt = true" or "npt = false" is great. I wouldn't call it npt_input (maybe npt_setting or sth), but that doesn't matter much.

Quoted 5 lines
                                                                        \
        pr_info("Testing: " test_name "\n");                            \
        run_test(guest_code, false, 1);                                 \
                                                                        \
        if (guest_code == l1_guest_code) {                              \

I thought about doing this, but I was worried about it breaking if we add more L1 guest code functions. In retrospect that was too paranoid (as usual).

Quoted 9 lines
                pr_info("Testing: " test_name " Save/Restore\n");       \
                run_test(guest_code, true, 1);                          \
                                                                        \
                pr_info("Testing: " test_name " Multiple VMRUNs\n");    \
                run_test(guest_code, false, 10);                        \
        }                                                               \
} while (0)

int main(int argc, char *argv[])

Quoted 10 lines
{
        TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SVM));
        TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_NPT));
        TEST_REQUIRE(kvm_has_cap(KVM_CAP_NESTED_STATE));
        TEST_REQUIRE(kvm_check_cap(KVM_CAP_DISABLE_QUIRKS2) &
                     KVM_X86_QUIRK_NESTED_SVM_SHARED_PAT);

        gpat_test("Invalid gPAT", l1_guest_code_invalid_gpat, npt_enabled = true);
        gpat_test("Nested NPT disabled", l1_guest_code, npt_enabled = false);
        gpat_test("Nested NPT enabled", l1_guest_code, npt_enabled = true);

Overall looks great, thanks Sean.

#3 lore raw
Quoted 10 lines · Yosry Ahmed
Quoted 6 lines

If we're going to use macro shenanigans, let's actually use them! I've got this working, I'll post a v4.

#define gpat_test(test_name, guest_code, npt_input)                     \
do {                                                                    \
        npt_input;                                                      \

Oh I like this very much. Passing in "npt = true" or "npt = false" is great. I wouldn't call it npt_input (maybe npt_setting or sth), but that doesn't matter much.

I like npt_setting.