Skip to contentSkip to conversation

My set of KVM unit tests + fixes

PATCH In review Maxim Levitsky 2 participants 13 messages Feb 8, 2022 – Feb 24, 2022 KVM

lore
Share & tools
lore
#0 lore raw

cover My set of KVM unit tests + fixes

Those are few kvm unit tests tha I developed.

Best regards,
Maxim Levitsky

Maxim Levitsky (7):
pmu_lbr: few fixes
svm: Fix reg_corruption test, to avoid timer interrupt firing in later
tests.
svm: NMI is an "exception" and not interrupt in x86 land
svm: intercept shutdown in all svm tests by default
svm: add SVM_BARE_VMRUN
svm: add tests for LBR virtualization
svm: add tests for case when L1 intercepts various hardware interrupts
(an interrupt, SMI, NMI), but lets L2 control either EFLAG.IF or GIF

 lib/x86/processor.h |   1 +
 x86/pmu_lbr.c       |   6 +
 x86/svm.c           |  41 +---
 x86/svm.h           |  63 ++++++-
 x86/svm_tests.c     | 447 +++++++++++++++++++++++++++++++++++++++++++-
 x86/unittests.cfg   |   3 +-
 6 files changed, 521 insertions(+), 40 deletions(-)
Signature · 1 line
2.26.3
#1 lore raw

PATCH 1/7 pmu_lbr: few fixes

* don't run this test on AMD since AMD's LBR is not the same as Intel's LBR and needs a different test.

* don't run this test on 32 bit as it is not built for 32 bit anyway

---
 x86/pmu_lbr.c     | 6 ++++++
 x86/unittests.cfg | 1 +
 2 files changed, 7 insertions(+)
x86/pmu_lbr.c +6 -0
index 5ff805a..688634d 100644
@@ -68,6 +68,12 @@ int main(int ac, char **av)
 	int max, i;
 
 	setup_vm();
+
+	if (!is_intel()) {
+		report_skip("PMU_LBR test is for intel CPU's only");
+		return 0;
+	}
+
 	perf_cap = rdmsr(MSR_IA32_PERF_CAPABILITIES);
 	eax.full = id.a;
 
x86/unittests.cfg +1 -0
index 9a70ba3..89ff949 100644
@@ -179,6 +179,7 @@ check = /proc/sys/kernel/nmi_watchdog=0
 accel = kvm
 
 [pmu_lbr]
+arch = x86_64
 file = pmu_lbr.flat
 extra_params = -cpu host,migratable=no
 check = /sys/module/kvm/parameters/ignore_msrs=N
Signature · 1 line
2.26.3
#2 lore raw

PATCH 2/7 svm: Fix reg_corruption test, to avoid timer interrupt firing in later tests.

In reply to Maxim Levitsky

The test were setting APIC periodic timer but not disabling it later.

Fixes da338a3 ("SVM: add test for nested guest RIP corruption")
---
 x86/svm_tests.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
x86/svm_tests.c +7 -2
index 0707786..7a97847 100644
@@ -1847,7 +1847,7 @@ static bool reg_corruption_finished(struct svm_test *test)
         report_pass("No RIP corruption detected after %d timer interrupts",
                     isr_cnt);
         set_test_stage(test, 1);
-        return true;
+        goto cleanup;
     }
 
     if (vmcb->control.exit_code == SVM_EXIT_INTR) {
@@ -1861,11 +1861,16 @@ static bool reg_corruption_finished(struct svm_test *test)
         if (guest_rip == insb_instruction_label && io_port_var != 0xAA) {
             report_fail("RIP corruption detected after %d timer interrupts",
                         isr_cnt);
-            return true;
+            goto cleanup;
         }
 
     }
     return false;
+cleanup:
+    apic_write(APIC_LVTT, APIC_LVT_TIMER_MASK);
+    apic_write(APIC_TMICT, 0);
+    return true;
+
 }
 
 static bool reg_corruption_check(struct svm_test *test)
Signature · 1 line
2.26.3
#3 lore raw

PATCH 3/7 svm: NMI is an "exception" and not interrupt in x86 land

In reply to Maxim Levitsky

This can interfere with later tests which do treat it as an exception.

Fixes d4db486 ("svm: Add test cases around NMI injection")
---
 x86/svm_tests.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
x86/svm_tests.c +2 -3
index 7a97847..7586ef7 100644
@@ -1384,17 +1384,16 @@ static bool interrupt_check(struct svm_test *test)
 
 static volatile bool nmi_fired;
 
-static void nmi_handler(isr_regs_t *regs)
+static void nmi_handler(struct ex_regs *regs)
 {
     nmi_fired = true;
-    apic_write(APIC_EOI, 0);
 }
 
 static void nmi_prepare(struct svm_test *test)
 {
     default_prepare(test);
     nmi_fired = false;
-    handle_irq(NMI_VECTOR, nmi_handler);
+    handle_exception(NMI_VECTOR, nmi_handler);
     set_test_stage(test, 0);
 }
 
Signature · 1 line
2.26.3
#4 lore raw

PATCH 4/7 svm: intercept shutdown in all svm tests by default

In reply to Maxim Levitsky

If L1 doesn't intercept shutdown, then L1 itself gets it, which doesn't allow it to report the error that happened.

---
 x86/svm.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
x86/svm.c +3 -1
index 3f94b2a..62da2af 100644
@@ -174,7 +174,9 @@ void vmcb_ident(struct vmcb *vmcb)
 	save->cr2 = read_cr2();
 	save->g_pat = rdmsr(MSR_IA32_CR_PAT);
 	save->dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
-	ctrl->intercept = (1ULL << INTERCEPT_VMRUN) | (1ULL << INTERCEPT_VMMCALL);
+	ctrl->intercept = (1ULL << INTERCEPT_VMRUN) |
+			  (1ULL << INTERCEPT_VMMCALL) |
+			  (1ULL << INTERCEPT_SHUTDOWN);
 	ctrl->iopm_base_pa = virt_to_phys(io_bitmap);
 	ctrl->msrpm_base_pa = virt_to_phys(msr_bitmap);
 
Signature · 1 line
2.26.3
#5 lore raw

PATCH 5/7 svm: add SVM_BARE_VMRUN

In reply to Maxim Levitsky

This will be useful in nested LBR tests to ensure that no extra branches are made in the guest entry.

---
 x86/svm.c | 32 --------------------------------
 x86/svm.h | 47 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+), 32 deletions(-)
x86/svm.c +0 -32
index 62da2af..6f4e023 100644
@@ -196,41 +196,9 @@ struct regs get_regs(void)
 
 // rax handled specially below
 
-#define SAVE_GPR_C                              \
-        "xchg %%rbx, regs+0x8\n\t"              \
-        "xchg %%rcx, regs+0x10\n\t"             \
-        "xchg %%rdx, regs+0x18\n\t"             \
-        "xchg %%rbp, regs+0x28\n\t"             \
-        "xchg %%rsi, regs+0x30\n\t"             \
-        "xchg %%rdi, regs+0x38\n\t"             \
-        "xchg %%r8, regs+0x40\n\t"              \
-        "xchg %%r9, regs+0x48\n\t"              \
-        "xchg %%r10, regs+0x50\n\t"             \
-        "xchg %%r11, regs+0x58\n\t"             \
-        "xchg %%r12, regs+0x60\n\t"             \
-        "xchg %%r13, regs+0x68\n\t"             \
-        "xchg %%r14, regs+0x70\n\t"             \
-        "xchg %%r15, regs+0x78\n\t"
-
-#define LOAD_GPR_C      SAVE_GPR_C
 
 struct svm_test *v2_test;
 
-#define ASM_PRE_VMRUN_CMD                       \
-                "vmload %%rax\n\t"              \
-                "mov regs+0x80, %%r15\n\t"      \
-                "mov %%r15, 0x170(%%rax)\n\t"   \
-                "mov regs, %%r15\n\t"           \
-                "mov %%r15, 0x1f8(%%rax)\n\t"   \
-                LOAD_GPR_C                      \
-
-#define ASM_POST_VMRUN_CMD                      \
-                SAVE_GPR_C                      \
-                "mov 0x170(%%rax), %%r15\n\t"   \
-                "mov %%r15, regs+0x80\n\t"      \
-                "mov 0x1f8(%%rax), %%r15\n\t"   \
-                "mov %%r15, regs\n\t"           \
-                "vmsave %%rax\n\t"              \
 
 u64 guest_stack[10000];
 
x86/svm.h +47 -0

57 lines not on this page · raw message

Signature · 1 line
2.26.3
#6 lore raw

PATCH 6/7 svm: add tests for LBR virtualization

In reply to Maxim Levitsky
---
 lib/x86/processor.h |   1 +
 x86/svm.c           |   5 +
 x86/svm.h           |   5 +-
 x86/svm_tests.c     | 239 ++++++++++++++++++++++++++++++++++++++++++++
 x86/unittests.cfg   |   2 +-
 5 files changed, 250 insertions(+), 2 deletions(-)
lib/x86/processor.h +1 -0
index fe5add5..9147a47 100644
@@ -187,6 +187,7 @@ static inline bool is_intel(void)
 #define	X86_FEATURE_RDPRU		(CPUID(0x80000008, 0, EBX, 4))
 #define	X86_FEATURE_AMD_IBPB		(CPUID(0x80000008, 0, EBX, 12))
 #define	X86_FEATURE_NPT			(CPUID(0x8000000A, 0, EDX, 0))
+#define	X86_FEATURE_LBRV		(CPUID(0x8000000A, 0, EDX, 1))
 #define	X86_FEATURE_NRIPS		(CPUID(0x8000000A, 0, EDX, 3))
 #define	X86_FEATURE_VGIF		(CPUID(0x8000000A, 0, EDX, 16))
 
x86/svm.c +5 -0
index 6f4e023..bb58d7c 100644
@@ -70,6 +70,11 @@ bool vgif_supported(void)
 	return this_cpu_has(X86_FEATURE_VGIF);
 }
 
+bool lbrv_supported(void)
+{
+    return this_cpu_has(X86_FEATURE_LBRV);
+}
+
 void default_prepare(struct svm_test *test)
 {
 	vmcb_ident(vmcb);
x86/svm.h +4 -1
index 6d072f4..58b9410 100644
@@ -98,7 +98,7 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
 	u32 event_inj;
 	u32 event_inj_err;
 	u64 nested_cr3;
-	u64 lbr_ctl;
+	u64 virt_ext;
 	u32 clean;
 	u32 reserved_5;
 	u64 next_rip;
@@ -360,6 +360,8 @@ struct __attribute__ ((__packed__)) vmcb {
 
 #define MSR_BITMAP_SIZE 8192
 
+#define LBR_CTL_ENABLE_MASK BIT_ULL(0)
+
 struct svm_test {
 	const char *name;
 	bool (*supported)(void);
@@ -405,6 +407,7 @@ u64 *npt_get_pml4e(void);
 bool smp_supported(void);
 bool default_supported(void);
 bool vgif_supported(void);
+bool lbrv_supported(void);
 void default_prepare(struct svm_test *test);
 void default_prepare_gif_clear(struct svm_test *test);
 bool default_finished(struct svm_test *test);
x86/svm_tests.c +239 -0

250 lines not on this page · raw message

x86/unittests.cfg +1 -1
index 89ff949..fa4ff69 100644
@@ -238,7 +238,7 @@ arch = x86_64
 [svm]
 file = svm.flat
 smp = 2
-extra_params = -cpu max,+svm -m 4g
+extra_params = -cpu max,+svm -m 4g -append "-svm_lbrv_test*"
 arch = x86_64
 
 [taskswitch]
Signature · 1 line
2.26.3
#7 lore raw

PATCH 7/7 svm: add tests for case when L1 intercepts various hardware interrupts (an interrupt, SMI, NMI), but lets L2 control either EFLAG.IF or GIF

In reply to Maxim Levitsky
---
 x86/svm.h       |  11 +++
 x86/svm_tests.c | 194 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 205 insertions(+)
x86/svm.h +11 -0
index 58b9410..df1b1ac 100644
@@ -426,6 +426,17 @@ void test_set_guest(test_guest_func func);
 extern struct vmcb *vmcb;
 extern struct svm_test svm_tests[];
 
+static inline void stgi(void)
+{
+    asm volatile ("stgi");
+}
+
+static inline void clgi(void)
+{
+    asm volatile ("clgi");
+}
+
+
 
 #define SAVE_GPR_C                              \
         "xchg %%rbx, regs+0x8\n\t"              \
x86/svm_tests.c +194 -0

205 lines not on this page · raw message

Signature · 1 line
2.26.3
#8 lore raw
In reply to Maxim Levitsky
Quoted 16 lines · Maxim Levitsky

Those are few kvm unit tests tha I developed.

Best regards,
Maxim Levitsky

Maxim Levitsky (7):
pmu_lbr: few fixes
svm: Fix reg_corruption test, to avoid timer interrupt firing in later
tests.
svm: NMI is an "exception" and not interrupt in x86 land
svm: intercept shutdown in all svm tests by default
svm: add SVM_BARE_VMRUN
svm: add tests for LBR virtualization
svm: add tests for case when L1 intercepts various hardware interrupts
(an interrupt, SMI, NMI), but lets L2 control either EFLAG.IF or GIF

Quoted 12 lines
 lib/x86/processor.h |   1 +
 x86/pmu_lbr.c       |   6 +
 x86/svm.c           |  41 +---
 x86/svm.h           |  63 ++++++-
 x86/svm_tests.c     | 447 +++++++++++++++++++++++++++++++++++++++++++-
 x86/unittests.cfg   |   3 +-
 6 files changed, 521 insertions(+), 40 deletions(-)
Signature · 1 line
2.26.3

Any update on these patches?

Best regards,
Maxim Levitsky

#9 lore raw

Hi,

Quoted 25 lines · Maxim Levitsky
Quoted 24 lines · Maxim Levitsky

Those are few kvm unit tests tha I developed.

Best regards,
Maxim Levitsky

Maxim Levitsky (7):
pmu_lbr: few fixes
svm: Fix reg_corruption test, to avoid timer interrupt firing in later
tests.
svm: NMI is an "exception" and not interrupt in x86 land
svm: intercept shutdown in all svm tests by default
svm: add SVM_BARE_VMRUN
svm: add tests for LBR virtualization
svm: add tests for case when L1 intercepts various hardware interrupts
(an interrupt, SMI, NMI), but lets L2 control either EFLAG.IF or GIF

 lib/x86/processor.h |   1 +
 x86/pmu_lbr.c       |   6 +
 x86/svm.c           |  41 +---
 x86/svm.h           |  63 ++++++-
 x86/svm_tests.c     | 447 +++++++++++++++++++++++++++++++++++++++++++-
 x86/unittests.cfg   |   3 +-
 6 files changed, 521 insertions(+), 40 deletions(-)
Quoted 5 lines
Quoted 4 lines
Signature · 1 line
2.26.3

Any update on these patches?

It is possible that because you haven't sent the patches to the x86 maintainer (as per the MAINTAINERS file), this series has gone unnoticed. Also, each patch should start with "kvm-unit-tests PATCH" (have a look at the README file), so people can easily tell them apart from KVM patches, which go to the same mailing list.

You could try resending the entire series to the x86 mailing list and to the relevant maintainers. To resend them, the convention is to modify the subject prefix to "kvm-unit-tests PATCH RESEND" and send them without any changes (though you can mention in the cover letter why you resent the series).

Hope this helps!

Thanks,
Alex

Quoted 4 lines

Best regards,
Maxim Levitsky

#10 lore raw
Quoted 32 lines · Alexandru Elisei

Hi,

Quoted 29 lines · Maxim Levitsky
Quoted 28 lines · Maxim Levitsky

Those are few kvm unit tests tha I developed.

Best regards,
Maxim Levitsky

Maxim Levitsky (7):
pmu_lbr: few fixes
svm: Fix reg_corruption test, to avoid timer interrupt firing in later
tests.
svm: NMI is an "exception" and not interrupt in x86 land
svm: intercept shutdown in all svm tests by default
svm: add SVM_BARE_VMRUN
svm: add tests for LBR virtualization
svm: add tests for case when L1 intercepts various hardware interrupts
(an interrupt, SMI, NMI), but lets L2 control either EFLAG.IF or GIF

 lib/x86/processor.h |   1 +
 x86/pmu_lbr.c       |   6 +
 x86/svm.c           |  41 +---
 x86/svm.h           |  63 ++++++-
 x86/svm_tests.c     | 447 +++++++++++++++++++++++++++++++++++++++++++-
 x86/unittests.cfg   |   3 +-
 6 files changed, 521 insertions(+), 40 deletions(-)
Signature · 1 line
2.26.3
Quoted 7 lines
Quoted 1 line

Any update on these patches?

It is possible that because you haven't sent the patches to the x86 maintainer (as per the MAINTAINERS file), this series has gone unnoticed. Also, each patch should start with "kvm-unit-tests PATCH" (have a look at the README file), so people can easily tell them apart from KVM patches, which go to the same mailing list.

Do kvm-unit tests have MAINTAINERS file? Those patches are not for the kernel but for the kvm-unit test project.

Quoted 6 lines

You could try resending the entire series to the x86 mailing list and to the relevant maintainers. To resend them, the convention is to modify the subject prefix to "kvm-unit-tests PATCH RESEND" and send them without any changes (though you can mention in the cover letter why you resent the series).

Thank you, I missed the prefix to be used!

Quoted 5 lines

Hope this helps!

Thanks,
Alex

Thanks!
Best regards,
Maxim Levitsky

Quoted 4 lines
Quoted 4 lines

Best regards,
Maxim Levitsky

#11 lore raw

Hi,

Quoted 35 lines · Maxim Levitsky
Quoted 34 lines · Alexandru Elisei

Hi,

Quoted 30 lines · Maxim Levitsky
Quoted 16 lines · Maxim Levitsky

Those are few kvm unit tests tha I developed.

Best regards,
Maxim Levitsky

Maxim Levitsky (7):
pmu_lbr: few fixes
svm: Fix reg_corruption test, to avoid timer interrupt firing in later
tests.
svm: NMI is an "exception" and not interrupt in x86 land
svm: intercept shutdown in all svm tests by default
svm: add SVM_BARE_VMRUN
svm: add tests for LBR virtualization
svm: add tests for case when L1 intercepts various hardware interrupts
(an interrupt, SMI, NMI), but lets L2 control either EFLAG.IF or GIF

Quoted 12 lines
 lib/x86/processor.h |   1 +
 x86/pmu_lbr.c       |   6 +
 x86/svm.c           |  41 +---
 x86/svm.h           |  63 ++++++-
 x86/svm_tests.c     | 447 +++++++++++++++++++++++++++++++++++++++++++-
 x86/unittests.cfg   |   3 +-
 6 files changed, 521 insertions(+), 40 deletions(-)
Signature · 1 line
2.26.3

Any update on these patches?

Quoted 7 lines
Quoted 5 lines

It is possible that because you haven't sent the patches to the x86 maintainer (as per the MAINTAINERS file), this series has gone unnoticed. Also, each patch should start with "kvm-unit-tests PATCH" (have a look at the README file), so people can easily tell them apart from KVM patches, which go to the same mailing list.

Do kvm-unit tests have MAINTAINERS file? Those patches are not for the kernel but for the kvm-unit test project.

A simple ls should be sufficient [1] to answer that question.

[1] https://gitlab.com/kvm-unit-tests/kvm-unit-tests/-/blob/master/MAINTAINERS

Thanks,
Alex

Quoted 25 lines
Quoted 7 lines

You could try resending the entire series to the x86 mailing list and to the relevant maintainers. To resend them, the convention is to modify the subject prefix to "kvm-unit-tests PATCH RESEND" and send them without any changes (though you can mention in the cover letter why you resent the series).

Thank you, I missed the prefix to be used!

Quoted 5 lines

Hope this helps!

Thanks,
Alex

Thanks!
Best regards,
Maxim Levitsky

Quoted 4 lines
Quoted 4 lines

Best regards,
Maxim Levitsky

#12 lore raw
Quoted 43 lines · Alexandru Elisei

Hi,

Quoted 40 lines · Maxim Levitsky
Quoted 39 lines · Alexandru Elisei

Hi,

Quoted 25 lines · Maxim Levitsky
Quoted 24 lines · Maxim Levitsky

Those are few kvm unit tests tha I developed.

Best regards,
Maxim Levitsky

Maxim Levitsky (7):
pmu_lbr: few fixes
svm: Fix reg_corruption test, to avoid timer interrupt firing in later
tests.
svm: NMI is an "exception" and not interrupt in x86 land
svm: intercept shutdown in all svm tests by default
svm: add SVM_BARE_VMRUN
svm: add tests for LBR virtualization
svm: add tests for case when L1 intercepts various hardware interrupts
(an interrupt, SMI, NMI), but lets L2 control either EFLAG.IF or GIF

 lib/x86/processor.h |   1 +
 x86/pmu_lbr.c       |   6 +
 x86/svm.c           |  41 +---
 x86/svm.h           |  63 ++++++-
 x86/svm_tests.c     | 447 +++++++++++++++++++++++++++++++++++++++++++-
 x86/unittests.cfg   |   3 +-
 6 files changed, 521 insertions(+), 40 deletions(-)
Quoted 5 lines
Quoted 4 lines
Signature · 1 line
2.26.3

Any update on these patches?

It is possible that because you haven't sent the patches to the x86 maintainer (as per the MAINTAINERS file), this series has gone unnoticed. Also, each patch should start with "kvm-unit-tests PATCH" (have a look at the README file), so people can easily tell them apart from KVM patches, which go to the same mailing list.

Quoted 6 lines
Quoted 2 lines

Do kvm-unit tests have MAINTAINERS file? Those patches are not for the kernel but for the kvm-unit test project.

A simple ls should be sufficient [1] to answer that question.

[1] https://gitlab.com/kvm-unit-tests/kvm-unit-tests/-/blob/master/MAINTAINERS

I am blind. just that. I checked it and haven't seen the file. Now I see it of course.

Best regards,
Maixm Levitsky

Quoted 23 lines

Thanks,
Alex

Quoted 19 lines
Quoted 5 lines

You could try resending the entire series to the x86 mailing list and to the relevant maintainers. To resend them, the convention is to modify the subject prefix to "kvm-unit-tests PATCH RESEND" and send them without any changes (though you can mention in the cover letter why you resent the series).

Thank you, I missed the prefix to be used!

Quoted 4 lines

Hope this helps!

Thanks,
Alex

Thanks!
Best regards,
Maxim Levitsky

Quoted 3 lines
Quoted 3 lines

Best regards,
Maxim Levitsky