Skip to contentSkip to conversation

x86: nSVM: Fix bugs in LBRV tests

PATCH In review Sean Christopherson 1 participant 8 messages Jun 7, 2023

Subsystems (2)
lore
Share & tools
lore

Sean Christopherson

author reviewer
#0 lore raw

cover x86: nSVM: Fix bugs in LBRV tests

Fix bugs in the LBRV tests, and try to make them a bit less painful to debug (goes from "excruitating" to just "awful"). The tests have been failing on our Milan systems for quite some time, but debugging them was painful and I wasn't sufficiently motivated until I wanted to clean up KVM's code :-/

The Milan failures are due to AMD shoving a "mispredict" bit in the LBR records, i.e. the tests fail if the CPU mispredicts the branch being tested. Debug was especially difficult because the tests also neglected to setup a guest stack, i.e. adding debug code created a completely different failure that looked similar at first glance.

Sean Christopherson (6):
x86: nSVM: Set up a guest stack in LBRV tests
lib: Expose a subset of VMX's assertion macros
x86: Add defines for the various LBR record bit definitions
x86: nSVM: Ignore mispredict bit in LBR records
x86: nSVM: Replace check_dbgctl() with TEST_EXPECT_EQ() in LBRV test
x86: nSVM: Print out RIP and LBRs from VMCB if LBRV guest test fails

 lib/util.h      |  31 ++++++++++++++
 lib/x86/msr.h   |  11 +++++
 x86/svm.c       |   7 +++-
 x86/svm.h       |   2 +-
 x86/svm_tests.c | 108 +++++++++++++++++++++---------------------------
 x86/vmx.h       |  32 ++------------
 6 files changed, 99 insertions(+), 92 deletions(-)

base-commit: 02d8befe99f8205d4caea402d8b0800354255681

Signature · 1 line
2.41.0.162.gfafddb0af9-goog
author reviewer
#1 lore raw

PATCH 1/6 x86: nSVM: Set up a guest stack in LBRV tests

Add a helper to configure save.rip and save.rsp, and use it in the LBRV tests, which use a "bare" VMRUN to avoid branches around VMRUN. This fixes a bug where the LBRV tests explode in confusing ways if the compiler generates guest code that touches the stack in *any* way.

---
 x86/svm.c       | 7 ++++++-
 x86/svm.h       | 2 +-
 x86/svm_tests.c | 8 ++++----
 3 files changed, 11 insertions(+), 6 deletions(-)
x86/svm.c +6 -1
index ba435b4a..c24cb97c 100644
@@ -212,10 +212,15 @@ struct svm_test *v2_test;
 
 u64 guest_stack[10000];
 
-int __svm_vmrun(u64 rip)
+void svm_setup_vmrun(u64 rip)
 {
 	vmcb->save.rip = (ulong)rip;
 	vmcb->save.rsp = (ulong)(guest_stack + ARRAY_SIZE(guest_stack));
+}
+
+int __svm_vmrun(u64 rip)
+{
+	svm_setup_vmrun(rip);
 	regs.rdi = (ulong)v2_test;
 
 	asm volatile (
x86/svm.h +1 -1
index 766ff7e3..4857212b 100644
@@ -425,8 +425,8 @@ void inc_test_stage(struct svm_test *test);
 void vmcb_ident(struct vmcb *vmcb);
 struct regs get_regs(void);
 void vmmcall(void);
+void svm_setup_vmrun(u64 rip);
 int __svm_vmrun(u64 rip);
-void __svm_bare_vmrun(void);
 int svm_vmrun(void);
 void test_set_guest(test_guest_func func);
 u64* get_npt_pte(u64 *pml4, u64 guest_addr, int level);
x86/svm_tests.c +4 -4
index 27ce47b4..e20f6697 100644
@@ -2895,7 +2895,7 @@ static void svm_lbrv_test1(void)
 {
 	report(true, "Test that without LBRV enabled, guest LBR state does 'leak' to the host(1)");
 
-	vmcb->save.rip = (ulong)svm_lbrv_test_guest1;
+	svm_setup_vmrun((u64)svm_lbrv_test_guest1);
 	vmcb->control.virt_ext = 0;
 
 	wrmsr(MSR_IA32_DEBUGCTLMSR, DEBUGCTLMSR_LBR);
@@ -2917,7 +2917,7 @@ static void svm_lbrv_test2(void)
 {
 	report(true, "Test that without LBRV enabled, guest LBR state does 'leak' to the host(2)");
 
-	vmcb->save.rip = (ulong)svm_lbrv_test_guest2;
+	svm_setup_vmrun((u64)svm_lbrv_test_guest2);
 	vmcb->control.virt_ext = 0;
 
 	wrmsr(MSR_IA32_DEBUGCTLMSR, DEBUGCTLMSR_LBR);
@@ -2945,7 +2945,7 @@ static void svm_lbrv_nested_test1(void)
 	}
 
 	report(true, "Test that with LBRV enabled, guest LBR state doesn't leak (1)");
-	vmcb->save.rip = (ulong)svm_lbrv_test_guest1;
+	svm_setup_vmrun((u64)svm_lbrv_test_guest1);
 	vmcb->control.virt_ext = LBR_CTL_ENABLE_MASK;
 	vmcb->save.dbgctl = DEBUGCTLMSR_LBR;
 
@@ -2978,7 +2978,7 @@ static void svm_lbrv_nested_test2(void)
 	}
 
 	report(true, "Test that with LBRV enabled, guest LBR state doesn't leak (2)");
-	vmcb->save.rip = (ulong)svm_lbrv_test_guest2;
+	svm_setup_vmrun((u64)svm_lbrv_test_guest2);
 	vmcb->control.virt_ext = LBR_CTL_ENABLE_MASK;
 
 	vmcb->save.dbgctl = 0;
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
author reviewer
#2 lore raw

PATCH 2/6 lib: Expose a subset of VMX's assertion macros

In reply to Sean Christopherson

Expose VMX's TEST_EXPECT_EQ() macros so that they can be used outside of x86's nested VMX tests, e.g. in x86's nested SVM tests. Leave the "full" assert macros in vmx.h, as gracfeully bailing from an arbitrary point in a test requires additional infrastructure, e.g. to do setjmp() and longjmp().

---
 lib/util.h | 31 +++++++++++++++++++++++++++++++
 x86/vmx.h  | 32 +++-----------------------------
 2 files changed, 34 insertions(+), 29 deletions(-)
lib/util.h +31 -0
index 4c4b4413..f86af6d3 100644
@@ -20,4 +20,35 @@
  */
 extern int parse_keyval(char *s, long *val);
 
+#define __TEST_EQ(a, b, a_str, b_str, assertion, do_abort, fmt, args...)		\
+do {											\
+	typeof(a) _a = a;								\
+	typeof(b) _b = b;								\
+	if (_a != _b) {									\
+		char _bin_a[BINSTR_SZ];							\
+		char _bin_b[BINSTR_SZ];							\
+		binstr(_a, _bin_a);							\
+		binstr(_b, _bin_b);							\
+		report_fail("%s:%d: %s failed: (%s) == (%s)\n"				\
+			    "\tLHS: %#018lx - %s - %lu\n"				\
+			    "\tRHS: %#018lx - %s - %lu%s" fmt,				\
+			    __FILE__, __LINE__,						\
+			    assertion ? "Assertion" : "Expectation", a_str, b_str,	\
+			    (unsigned long) _a, _bin_a, (unsigned long) _a,		\
+			    (unsigned long) _b, _bin_b, (unsigned long) _b,		\
+			    fmt[0] == '\0' ? "" : "\n", ## args);			\
+		dump_stack();								\
+		if (assertion)								\
+			do_abort();							\
+	}										\
+	report_passed();								\
+} while (0)
+
+/* FIXME: Extend VMX's assert/abort framework to SVM and other environs. */
+static inline void dummy_abort(void) {}
+
+#define TEST_EXPECT_EQ(a, b) __TEST_EQ(a, b, #a, #b, 0, dummy_abort, "")
+#define TEST_EXPECT_EQ_MSG(a, b, fmt, args...) \
+	__TEST_EQ(a, b, #a, #b, 0, dummy_abort fmt, ## args)
+
 #endif
x86/vmx.h +3 -29
index 604c78f6..bc61a586 100644
@@ -4,6 +4,7 @@
 #include "libcflat.h"
 #include "processor.h"
 #include "bitops.h"
+#include "util.h"
 #include "asm/page.h"
 #include "asm/io.h"
 
@@ -36,36 +37,9 @@ do {								\
 	report_passed();					\
 } while (0)
 
-#define __TEST_EQ(a, b, a_str, b_str, assertion, fmt, args...)	\
-do {								\
-	typeof(a) _a = a;					\
-	typeof(b) _b = b;					\
-	if (_a != _b) {						\
-		char _bin_a[BINSTR_SZ];				\
-		char _bin_b[BINSTR_SZ];				\
-		binstr(_a, _bin_a);				\
-		binstr(_b, _bin_b);				\
-		report_fail("%s:%d: %s failed: (%s) == (%s)\n"	\
-			    "\tLHS: %#018lx - %s - %lu\n"	\
-			    "\tRHS: %#018lx - %s - %lu%s" fmt,	\
-			    __FILE__, __LINE__,			\
-			    assertion ? "Assertion" : "Expectation", a_str, b_str,	\
-			    (unsigned long) _a, _bin_a, (unsigned long) _a,		\
-			    (unsigned long) _b, _bin_b, (unsigned long) _b,		\
-			    fmt[0] == '\0' ? "" : "\n", ## args);			\
-		dump_stack();					\
-		if (assertion)					\
-			__abort_test();				\
-	}							\
-	report_passed();					\
-} while (0)
-
-#define TEST_ASSERT_EQ(a, b) __TEST_EQ(a, b, #a, #b, 1, "")
+#define TEST_ASSERT_EQ(a, b) __TEST_EQ(a, b, #a, #b, 1, __abort_test, "")
 #define TEST_ASSERT_EQ_MSG(a, b, fmt, args...) \
-	__TEST_EQ(a, b, #a, #b, 1, fmt, ## args)
-#define TEST_EXPECT_EQ(a, b) __TEST_EQ(a, b, #a, #b, 0, "")
-#define TEST_EXPECT_EQ_MSG(a, b, fmt, args...) \
-	__TEST_EQ(a, b, #a, #b, 0, fmt, ## args)
+	__TEST_EQ(a, b, #a, #b, 1, __abort_test, fmt, ## args)
 
 struct vmcs_hdr {
 	u32 revision_id:31;
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
author reviewer
#3 lore raw

PATCH 3/6 x86: Add defines for the various LBR record bit definitions

In reply to Sean Christopherson

Add defines for Intel's LBR info MSRs, and a define (along with a snarky message) for AMD's lovely mispredict bit that is shoved into the LBR MSRs themselves. The AMD mispredict bit will be used to address false positives in the LBRV tests.

---
 lib/x86/msr.h | 11 +++++++++++
 1 file changed, 11 insertions(+)
lib/x86/msr.h +11 -0
index 0e3fd037..8abccf86 100644
@@ -81,6 +81,17 @@
 #define MSR_IA32_LASTINTFROMIP		0x000001dd
 #define MSR_IA32_LASTINTTOIP		0x000001de
 
+/* Yes, AMD does indeed record mispredict info in the LBR records themselves. */
+#define AMD_LBR_RECORD_MISPREDICT	BIT_ULL(63)
+
+#define LBR_INFO_MISPRED		BIT_ULL(63)
+#define LBR_INFO_IN_TX			BIT_ULL(62)
+#define LBR_INFO_ABORT			BIT_ULL(61)
+#define LBR_INFO_CYC_CNT_VALID		BIT_ULL(60)
+#define LBR_INFO_CYCLES			0xffff
+#define LBR_INFO_BR_TYPE_OFFSET		56
+#define LBR_INFO_BR_TYPE		(0xfull << LBR_INFO_BR_TYPE_OFFSET)
+
 /* DEBUGCTLMSR bits (others vary by model): */
 #define DEBUGCTLMSR_LBR			(1UL <<  0) /* last branch recording */
 #define DEBUGCTLMSR_BTF			(1UL <<  1) /* single-step on branches */
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
author reviewer
#4 lore raw

PATCH 4/6 x86: nSVM: Ignore mispredict bit in LBR records

In reply to Sean Christopherson

Ignore the mispredict bit when comparing the expected versus actual LBR records. Unsurprisingly, relying on the whims of the CPU's branch predictor results in false failures.

Fixes 537d39df ("svm: add tests for LBR virtualization")
---
 x86/svm_tests.c | 65 ++++++++++++++++++++++---------------------------
 1 file changed, 29 insertions(+), 36 deletions(-)
x86/svm_tests.c +29 -36
index e20f6697..9a89155a 100644
@@ -10,6 +10,7 @@
 #include "isr.h"
 #include "apic.h"
 #include "delay.h"
+#include "util.h"
 #include "x86/usermode.h"
 
 #define SVM_EXIT_MAX_DR_INTERCEPT 0x3f
@@ -2767,26 +2768,29 @@ static void svm_no_nm_test(void)
 	       "fnop with CR0.TS and CR0.EM unset no #NM excpetion");
 }
 
-static bool check_lbr(u64 *from_excepted, u64 *to_expected)
+static u64 amd_get_lbr_rip(u32 msr)
 {
-	u64 from = rdmsr(MSR_IA32_LASTBRANCHFROMIP);
-	u64 to = rdmsr(MSR_IA32_LASTBRANCHTOIP);
-
-	if ((u64)from_excepted != from) {
-		report(false, "MSR_IA32_LASTBRANCHFROMIP, expected=0x%lx, actual=0x%lx",
-		       (u64)from_excepted, from);
-		return false;
-	}
-
-	if ((u64)to_expected != to) {
-		report(false, "MSR_IA32_LASTBRANCHFROMIP, expected=0x%lx, actual=0x%lx",
-		       (u64)from_excepted, from);
-		return false;
-	}
-
-	return true;
+	return rdmsr(msr) & ~AMD_LBR_RECORD_MISPREDICT;
 }
 
+#define HOST_CHECK_LBR(from_expected, to_expected)					\
+do {											\
+	TEST_EXPECT_EQ((u64)from_expected, amd_get_lbr_rip(MSR_IA32_LASTBRANCHFROMIP));	\
+	TEST_EXPECT_EQ((u64)to_expected, amd_get_lbr_rip(MSR_IA32_LASTBRANCHTOIP));	\
+} while (0)
+
+/*
+ * FIXME: Do something other than generate an exception to communicate failure.
+ * Debugging without expected vs. actual is an absolute nightmare.
+ */
+#define GUEST_CHECK_LBR(from_expected, to_expected)				\
+do {										\
+	if ((u64)(from_expected) != amd_get_lbr_rip(MSR_IA32_LASTBRANCHFROMIP))	\
+		asm volatile("ud2");						\
+	if ((u64)(to_expected) != amd_get_lbr_rip(MSR_IA32_LASTBRANCHTOIP))	\
+		asm volatile("ud2");						\
+} while (0)
+
 static bool check_dbgctl(u64 dbgctl, u64 dbgctl_expected)
 {
 	if (dbgctl != dbgctl_expected) {
@@ -2796,7 +2800,6 @@ static bool check_dbgctl(u64 dbgctl, u64 dbgctl_expected)
 	return true;
 }
 
-
 #define DO_BRANCH(branch_name)				\
 	asm volatile (					\
 		      # branch_name "_from:"		\
@@ -2834,11 +2837,8 @@ static void svm_lbrv_test_guest1(void)
 		asm volatile("ud2\n");
 	if (rdmsr(MSR_IA32_DEBUGCTLMSR) != 0)
 		asm volatile("ud2\n");
-	if (rdmsr(MSR_IA32_LASTBRANCHFROMIP) != (u64)&guest_branch0_from)
-		asm volatile("ud2\n");
-	if (rdmsr(MSR_IA32_LASTBRANCHTOIP) != (u64)&guest_branch0_to)
-		asm volatile("ud2\n");
 
+	GUEST_CHECK_LBR(&guest_branch0_from, &guest_branch0_to);
 	asm volatile ("vmmcall\n");
 }
 
@@ -2855,11 +2855,7 @@ static void svm_lbrv_test_guest2(void)
 	if (dbgctl != 0)
 		asm volatile("ud2\n");
 
-	if (rdmsr(MSR_IA32_LASTBRANCHFROMIP) != (u64)&host_branch2_from)
-		asm volatile("ud2\n");
-	if (rdmsr(MSR_IA32_LASTBRANCHTOIP) != (u64)&host_branch2_to)
-		asm volatile("ud2\n");
-
+	GUEST_CHECK_LBR(&host_branch2_from, &host_branch2_to);
 
 	wrmsr(MSR_IA32_DEBUGCTLMSR, DEBUGCTLMSR_LBR);
 	dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
@@ -2868,10 +2864,7 @@ static void svm_lbrv_test_guest2(void)
 
 	if (dbgctl != DEBUGCTLMSR_LBR)
 		asm volatile("ud2\n");
-	if (rdmsr(MSR_IA32_LASTBRANCHFROMIP) != (u64)&guest_branch2_from)
-		asm volatile("ud2\n");
-	if (rdmsr(MSR_IA32_LASTBRANCHTOIP) != (u64)&guest_branch2_to)
-		asm volatile("ud2\n");
+	GUEST_CHECK_LBR(&guest_branch2_from, &guest_branch2_to);
 
 	asm volatile ("vmmcall\n");
 }
@@ -2888,7 +2881,7 @@ static void svm_lbrv_test0(void)
 	dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
 	check_dbgctl(dbgctl, 0);
 
-	check_lbr(&host_branch0_from, &host_branch0_to);
+	HOST_CHECK_LBR(&host_branch0_from, &host_branch0_to);
 }
 
 static void svm_lbrv_test1(void)
@@ -2910,7 +2903,7 @@ static void svm_lbrv_test1(void)
 	}
 
 	check_dbgctl(dbgctl, 0);
-	check_lbr(&guest_branch0_from, &guest_branch0_to);
+	HOST_CHECK_LBR(&guest_branch0_from, &guest_branch0_to);
 }
 
 static void svm_lbrv_test2(void)
@@ -2934,7 +2927,7 @@ static void svm_lbrv_test2(void)
 	}
 
 	check_dbgctl(dbgctl, 0);
-	check_lbr(&guest_branch2_from, &guest_branch2_to);
+	HOST_CHECK_LBR(&guest_branch2_from, &guest_branch2_to);
 }
 
 static void svm_lbrv_nested_test1(void)
@@ -2967,7 +2960,7 @@ static void svm_lbrv_nested_test1(void)
 	}
 
 	check_dbgctl(dbgctl, DEBUGCTLMSR_LBR);
-	check_lbr(&host_branch3_from, &host_branch3_to);
+	HOST_CHECK_LBR(&host_branch3_from, &host_branch3_to);
 }
 
 static void svm_lbrv_nested_test2(void)
@@ -2998,7 +2991,7 @@ static void svm_lbrv_nested_test2(void)
 	}
 
 	check_dbgctl(dbgctl, DEBUGCTLMSR_LBR);
-	check_lbr(&host_branch4_from, &host_branch4_to);
+	HOST_CHECK_LBR(&host_branch4_from, &host_branch4_to);
 }
 
 
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
author reviewer
#5 lore raw

PATCH 5/6 x86: nSVM: Replace check_dbgctl() with TEST_EXPECT_EQ() in LBRV test

In reply to Sean Christopherson

Use TEST_EXPECT_EQ() to check the expected vs. actual DEBUGCTL instead of using a one-off implementation that doesn't even print the expected value.

---
 x86/svm_tests.c | 21 ++++++---------------
 1 file changed, 6 insertions(+), 15 deletions(-)
x86/svm_tests.c +6 -15
index 9a89155a..de2cedc8 100644
@@ -2791,15 +2791,6 @@ do {										\
 		asm volatile("ud2");						\
 } while (0)
 
-static bool check_dbgctl(u64 dbgctl, u64 dbgctl_expected)
-{
-	if (dbgctl != dbgctl_expected) {
-		report(false, "Unexpected MSR_IA32_DEBUGCTLMSR value 0x%lx", dbgctl);
-		return false;
-	}
-	return true;
-}
-
 #define DO_BRANCH(branch_name)				\
 	asm volatile (					\
 		      # branch_name "_from:"		\
@@ -2877,9 +2868,9 @@ static void svm_lbrv_test0(void)
 	dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
 	wrmsr(MSR_IA32_DEBUGCTLMSR, 0);
 
-	check_dbgctl(dbgctl, DEBUGCTLMSR_LBR);
+	TEST_EXPECT_EQ(dbgctl, DEBUGCTLMSR_LBR);
 	dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
-	check_dbgctl(dbgctl, 0);
+	TEST_EXPECT_EQ(dbgctl, 0);
 
 	HOST_CHECK_LBR(&host_branch0_from, &host_branch0_to);
 }
@@ -2902,7 +2893,7 @@ static void svm_lbrv_test1(void)
 		return;
 	}
 
-	check_dbgctl(dbgctl, 0);
+	TEST_EXPECT_EQ(dbgctl, 0);
 	HOST_CHECK_LBR(&guest_branch0_from, &guest_branch0_to);
 }
 
@@ -2926,7 +2917,7 @@ static void svm_lbrv_test2(void)
 		return;
 	}
 
-	check_dbgctl(dbgctl, 0);
+	TEST_EXPECT_EQ(dbgctl, 0);
 	HOST_CHECK_LBR(&guest_branch2_from, &guest_branch2_to);
 }
 
@@ -2959,7 +2950,7 @@ static void svm_lbrv_nested_test1(void)
 		return;
 	}
 
-	check_dbgctl(dbgctl, DEBUGCTLMSR_LBR);
+	TEST_EXPECT_EQ(dbgctl, DEBUGCTLMSR_LBR);
 	HOST_CHECK_LBR(&host_branch3_from, &host_branch3_to);
 }
 
@@ -2990,7 +2981,7 @@ static void svm_lbrv_nested_test2(void)
 		return;
 	}
 
-	check_dbgctl(dbgctl, DEBUGCTLMSR_LBR);
+	TEST_EXPECT_EQ(dbgctl, DEBUGCTLMSR_LBR);
 	HOST_CHECK_LBR(&host_branch4_from, &host_branch4_to);
 }
 
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
author reviewer
#6 lore raw

PATCH 6/6 x86: nSVM: Print out RIP and LBRs from VMCB if LBRV guest test fails

In reply to Sean Christopherson

Print out the RIP and LBRs from the VMCB's save are if a LBRV guest test fails, i.e. exits for a reason other than VMMCALL. Debugging the guests is still beyond painful, but with the guest RIP and LBRs, it's at least possible to piece together things like shutdowns due to unexpected #PF because the tests neglected to setup the guest stack.

---
 x86/svm_tests.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
x86/svm_tests.c +10 -8
index de2cedc8..e56501c1 100644
@@ -2791,6 +2791,12 @@ do {										\
 		asm volatile("ud2");						\
 } while (0)
 
+#define REPORT_GUEST_LBR_ERROR(vmcb)						\
+	report(false, "LBR guest test failed.  Exit reason 0x%x, RIP = %lx, from = %lx, to = %lx, ex from = %lx, ex to = %lx", \
+		       vmcb->control.exit_code, vmcb->save.rip,			\
+		       vmcb->save.br_from, vmcb->save.br_to,			\
+		       vmcb->save.last_excp_from, vmcb->save.last_excp_to)
+
 #define DO_BRANCH(branch_name)				\
 	asm volatile (					\
 		      # branch_name "_from:"		\
@@ -2888,8 +2894,7 @@ static void svm_lbrv_test1(void)
 	dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
 
 	if (vmcb->control.exit_code != SVM_EXIT_VMMCALL) {
-		report(false, "VMEXIT not due to vmmcall. Exit reason 0x%x",
-		       vmcb->control.exit_code);
+		REPORT_GUEST_LBR_ERROR(vmcb);
 		return;
 	}
 
@@ -2912,8 +2917,7 @@ static void svm_lbrv_test2(void)
 	wrmsr(MSR_IA32_DEBUGCTLMSR, 0);
 
 	if (vmcb->control.exit_code != SVM_EXIT_VMMCALL) {
-		report(false, "VMEXIT not due to vmmcall. Exit reason 0x%x",
-		       vmcb->control.exit_code);
+		REPORT_GUEST_LBR_ERROR(vmcb);
 		return;
 	}
 
@@ -2940,8 +2944,7 @@ static void svm_lbrv_nested_test1(void)
 	wrmsr(MSR_IA32_DEBUGCTLMSR, 0);
 
 	if (vmcb->control.exit_code != SVM_EXIT_VMMCALL) {
-		report(false, "VMEXIT not due to vmmcall. Exit reason 0x%x",
-		       vmcb->control.exit_code);
+		REPORT_GUEST_LBR_ERROR(vmcb);
 		return;
 	}
 
@@ -2976,8 +2979,7 @@ static void svm_lbrv_nested_test2(void)
 	wrmsr(MSR_IA32_DEBUGCTLMSR, 0);
 
 	if (vmcb->control.exit_code != SVM_EXIT_VMMCALL) {
-		report(false, "VMEXIT not due to vmmcall. Exit reason 0x%x",
-		       vmcb->control.exit_code);
+		REPORT_GUEST_LBR_ERROR(vmcb);
 		return;
 	}
 
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
author reviewer
#7 lore raw
In reply to Sean Christopherson
Quoted 6 lines · Sean Christopherson

Fix bugs in the LBRV tests, and try to make them a bit less painful to debug (goes from "excruitating" to just "awful"). The tests have been failing on our Milan systems for quite some time, but debugging them was painful and I wasn't sufficiently motivated until I wanted to clean up KVM's code :-/

Quoted 7 lines

The Milan failures are due to AMD shoving a "mispredict" bit in the LBR records, i.e. the tests fail if the CPU mispredicts the branch being tested. Debug was especially difficult because the tests also neglected to setup a guest stack, i.e. adding debug code created a completely different failure that looked similar at first glance.

[...]

Applied quickly to kvm-x86 next, as I'm OOO the rest of this week. I won't send a pull request until sometime next week, e.g. if anyone objects to putting the assertion macros in common util.h.

[1/6] x86: nSVM: Set up a guest stack in LBRV tests

      https://github.com/kvm-x86/kvm-unit-tests/commit/bef5a3677b60
[2/6] lib: Expose a subset of VMX's assertion macros
      https://github.com/kvm-x86/kvm-unit-tests/commit/11e5dc0f9ed4
[3/6] x86: Add defines for the various LBR record bit definitions
      https://github.com/kvm-x86/kvm-unit-tests/commit/691df8f964df
[4/6] x86: nSVM: Ignore mispredict bit in LBR records
      https://github.com/kvm-x86/kvm-unit-tests/commit/3ffa0914672e
[5/6] x86: nSVM: Replace check_dbgctl() with TEST_EXPECT_EQ() in LBRV test
      https://github.com/kvm-x86/kvm-unit-tests/commit/d8849d8f3702
[6/6] x86: nSVM: Print out RIP and LBRs from VMCB if LBRV guest test fails
      https://github.com/kvm-x86/kvm-unit-tests/commit/11c7d0604a50

--
https://github.com/kvm-x86/kvm-unit-tests/tree/next