Skip to contentSkip to conversation

Add additional testing for routing L2 exceptions

PATCH Superseded Aaron Lewis 3 participants 9 messages Jan 21, 2022 – Jan 26, 2022 KVM

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover Add additional testing for routing L2 exceptions

In a previous series testing was added to verify that when a #PF occured in L2 the exception was routed to the correct place. In this series other exceptions are tested (ie: #GP, #UD, #DE, #DB, #BP, #AC).

v3 -> v4:
- Add vmx_exception_test to vmx.

v2 -> v3:
- Commits 1 and 2 from v2 were accepted upstream (bug fixes).
- Moved exception_mnemonic() into a separate commit.
- Moved support for running a nested guest multiple times in one test into a separate commit.
- Moved the test framework into the same commit as the test itself.
- Simplified the test framework and test code based on Sean's recommendations.

v1 -> v2:
- Add guest_stack_top and guest_syscall_stack_top for aligning L2's stacks.
- Refactor test to make it more extensible (ie: Added vmx_exception_tests array and framework around it).
- Split test into 2 commits:
1. Test infrustructure.
2. Test cases.

Aaron Lewis (3):
x86: Make exception_mnemonic() visible to the tests
x86: Add support for running a nested guest multiple times in one test
x86: Add test coverage for nested_vmx_reflect_vmexit() testing

 lib/x86/desc.c    |   2 +-
 lib/x86/desc.h    |   1 +
 x86/unittests.cfg |   9 +++-
 x86/vmx.c         |  24 ++++++++-
 x86/vmx.h         |   2 +
 x86/vmx_tests.c   | 129 ++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 163 insertions(+), 4 deletions(-)
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
#1 lore raw

PATCH v4 1/3 x86: Make exception_mnemonic() visible to the tests

exception_mnemonic() is a useful function for more than just desc.c. Make it global, so it can be used in other KUT tests.

---
 lib/x86/desc.c | 2 +-
 lib/x86/desc.h | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)
lib/x86/desc.c +1 -1
index 16b7256..c2eb16e 100644
@@ -91,7 +91,7 @@ struct ex_record {
 
 extern struct ex_record exception_table_start, exception_table_end;
 
-static const char* exception_mnemonic(int vector)
+const char* exception_mnemonic(int vector)
 {
 	switch(vector) {
 	case 0: return "#DE";
lib/x86/desc.h +1 -0
index 9b81da0..ad6277b 100644
@@ -224,6 +224,7 @@ void set_intr_alt_stack(int e, void *fn);
 void print_current_tss_info(void);
 handler handle_exception(u8 v, handler fn);
 void unhandled_exception(struct ex_regs *regs, bool cpu);
+const char* exception_mnemonic(int vector);
 
 bool test_for_exception(unsigned int ex, void (*trigger_func)(void *data),
 			void *data);
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
#3 lore raw

PATCH v4 2/3 x86: Add support for running a nested guest multiple times in one test

In reply to Aaron Lewis

KUT has a limit of only being able to run one nested guest per vmx test. This is limiting and not necessary. Add support for allowing a test to run guest code multiple times.

---
 x86/vmx.c | 24 ++++++++++++++++++++++--
 x86/vmx.h |  2 ++
 2 files changed, 24 insertions(+), 2 deletions(-)
x86/vmx.c +22 -2
index f4fbb94..51eed8c 100644
@@ -1884,15 +1884,35 @@ void test_add_teardown(test_teardown_func func, void *data)
 	step->data = data;
 }
 
+static void __test_set_guest(test_guest_func func)
+{
+	assert(current->v2);
+	v2_guest_main = func;
+}
+
 /*
  * Set the target of the first enter_guest call. Can only be called once per
  * test. Must be called before first enter_guest call.
  */
 void test_set_guest(test_guest_func func)
 {
-	assert(current->v2);
 	TEST_ASSERT_MSG(!v2_guest_main, "Already set guest func.");
-	v2_guest_main = func;
+	__test_set_guest(func);
+}
+
+/*
+ * Set the target of the enter_guest call and reset the RIP so 'func' will
+ * start from the beginning.  This can be called multiple times per test.
+ */
+void test_override_guest(test_guest_func func)
+{
+	__test_set_guest(func);
+	init_vmcs_guest();
+}
+
+void test_set_guest_finished(void)
+{
+	guest_finished = 1;
 }
 
 static void check_for_guest_termination(union exit_reason exit_reason)
x86/vmx.h +2 -0
index 4423986..11cb665 100644
@@ -1055,7 +1055,9 @@ void hypercall(u32 hypercall_no);
 typedef void (*test_guest_func)(void);
 typedef void (*test_teardown_func)(void *data);
 void test_set_guest(test_guest_func func);
+void test_override_guest(test_guest_func func);
 void test_add_teardown(test_teardown_func func, void *data);
 void test_skip(const char *msg);
+void test_set_guest_finished(void);
 
 #endif
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
#4 lore raw

Re: [kvm-unit-tests PATCH v4 2/3] x86: Add support for running a nested guest multiple times in one test

Quoted 39 lines · Aaron Lewis

KUT has a limit of only being able to run one nested guest per vmx test. This is limiting and not necessary. Add support for allowing a test to run guest code multiple times.

---
 x86/vmx.c | 24 ++++++++++++++++++++++--
 x86/vmx.h |  2 ++
 2 files changed, 24 insertions(+), 2 deletions(-)
x86/vmx.c +12 -2
index f4fbb94..51eed8c 100644
@@ -1884,15 +1884,35 @@ void test_add_teardown(test_teardown_func func, void *data)
 	step->data = data;
 }
 
+static void __test_set_guest(test_guest_func func)
+{
+	assert(current->v2);
+	v2_guest_main = func;
+}
+
 /*
  * Set the target of the first enter_guest call. Can only be called once per
  * test. Must be called before first enter_guest call.
  */
 void test_set_guest(test_guest_func func)
 {
-	assert(current->v2);
 	TEST_ASSERT_MSG(!v2_guest_main, "Already set guest func.");
-	v2_guest_main = func;
+	__test_set_guest(func);
+}
+
+/*
+ * Set the target of the enter_guest call and reset the RIP so 'func' will
+ * start from the beginning.  This can be called multiple times per test.

x86/vmx.c void test_add_teardown(test_teardown_func func, void *data) · line 1915

Quoted 10 lines
x86/vmx.c +10 -0
+ */
+void test_override_guest(test_guest_func func)
+{
+	__test_set_guest(func);
+	init_vmcs_guest();
+}
+
+void test_set_guest_finished(void)
+{
+	guest_finished = 1;

Adding test_set_guest_finished() should be a separate commit.

Quoted 20 lines
x86/vmx.c +0 -0
 }
 
 static void check_for_guest_termination(union exit_reason exit_reason)
x86/vmx.h +2 -0
index 4423986..11cb665 100644
@@ -1055,7 +1055,9 @@ void hypercall(u32 hypercall_no);
 typedef void (*test_guest_func)(void);
 typedef void (*test_teardown_func)(void *data);
 void test_set_guest(test_guest_func func);
+void test_override_guest(test_guest_func func);
 void test_add_teardown(test_teardown_func func, void *data);
 void test_skip(const char *msg);
+void test_set_guest_finished(void);
 
 #endif
Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
#5 lore raw

PATCH v4 3/3 x86: Add test coverage for nested_vmx_reflect_vmexit() testing

In reply to Aaron Lewis

Add a framework and test cases to ensure exceptions that occur in L2 are forwarded to the correct place by nested_vmx_reflect_vmexit().

Add testing for exceptions: #GP, #UD, #DE, #DB, #BP, and #AC.

Change-Id I0196071571671f06165983b5055ed7382fa3e1fb
---
 x86/unittests.cfg |   9 +++-
 x86/vmx_tests.c   | 129 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 137 insertions(+), 1 deletion(-)
x86/unittests.cfg +8 -1
index 9a70ba3..6ec7a98 100644
@@ -288,7 +288,7 @@ arch = i386
 
 [vmx]
 file = vmx.flat
-extra_params = -cpu max,+vmx -append "-exit_monitor_from_l2_test -ept_access* -vmx_smp* -vmx_vmcs_shadow_test -atomic_switch_overflow_msrs_test -vmx_init_signal_test -vmx_apic_passthrough_tpr_threshold_test -apic_reg_virt_test -virt_x2apic_mode_test -vmx_pf_exception_test -vmx_pf_no_vpid_test -vmx_pf_invvpid_test -vmx_pf_vpid_test"
+extra_params = -cpu max,+vmx -append "-exit_monitor_from_l2_test -ept_access* -vmx_smp* -vmx_vmcs_shadow_test -atomic_switch_overflow_msrs_test -vmx_init_signal_test -vmx_apic_passthrough_tpr_threshold_test -apic_reg_virt_test -virt_x2apic_mode_test -vmx_pf_exception_test -vmx_pf_no_vpid_test -vmx_pf_invvpid_test -vmx_pf_vpid_test -vmx_exception_test"
 arch = x86_64
 groups = vmx
 
@@ -390,6 +390,13 @@ arch = x86_64
 groups = vmx nested_exception
 check = /sys/module/kvm_intel/parameters/allow_smaller_maxphyaddr=Y
 
+[vmx_exception_test]
+file = vmx.flat
+extra_params = -cpu max,+vmx -append vmx_exception_test
+arch = x86_64
+groups = vmx nested_exception
+timeout = 10
+
 [debug]
 file = debug.flat
 arch = x86_64
x86/vmx_tests.c +129 -0

146 lines not on this page · raw message

Signature · 1 line
2.35.0.rc0.227.g00780c9af4-goog
#6 lore raw

Re: [kvm-unit-tests PATCH v4 3/3] x86: Add test coverage for nested_vmx_reflect_vmexit() testing

Quoted 7 lines · Aaron Lewis

Add a framework and test cases to ensure exceptions that occur in L2 are forwarded to the correct place by nested_vmx_reflect_vmexit().

Add testing for exceptions: #GP, #UD, #DE, #DB, #BP, and #AC.

Change-Id I0196071571671f06165983b5055ed7382fa3e1fb

Don't forget to strip the Change-Id before posting.

Quoted 19 lines
---
 x86/unittests.cfg |   9 +++-
 x86/vmx_tests.c   | 129 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 137 insertions(+), 1 deletion(-)
x86/unittests.cfg +1 -1
index 9a70ba3..6ec7a98 100644
@@ -288,7 +288,7 @@ arch = i386
 
 [vmx]
 file = vmx.flat
-extra_params = -cpu max,+vmx -append "-exit_monitor_from_l2_test -ept_access* -vmx_smp* -vmx_vmcs_shadow_test -atomic_switch_overflow_msrs_test -vmx_init_signal_test -vmx_apic_passthrough_tpr_threshold_test -apic_reg_virt_test -virt_x2apic_mode_test -vmx_pf_exception_test -vmx_pf_no_vpid_test -vmx_pf_invvpid_test -vmx_pf_vpid_test"
+extra_params = -cpu max,+vmx -append "-exit_monitor_from_l2_test -ept_access* -vmx_smp* -vmx_vmcs_shadow_test -atomic_switch_overflow_msrs_test -vmx_init_signal_test -vmx_apic_passthrough_tpr_threshold_test -apic_reg_virt_test -virt_x2apic_mode_test -vmx_pf_exception_test -vmx_pf_no_vpid_test -vmx_pf_invvpid_test -vmx_pf_vpid_test -vmx_exception_test"
 arch = x86_64
 groups = vmx
 

x86/unittests.cfg arch = x86_64 · line 398

Quoted 9 lines
x86/unittests.cfg +6 -0
@@ -390,6 +390,13 @@ arch = x86_64
 groups = vmx nested_exception
 check = /sys/module/kvm_intel/parameters/allow_smaller_maxphyaddr=Y
 
+[vmx_exception_test]
+file = vmx.flat
+extra_params = -cpu max,+vmx -append vmx_exception_test
+arch = x86_64
+groups = vmx nested_exception
+timeout = 10

Leave this out (for now), including it in the main "vmx" test is sufficient. I'm definitely in favor of splitting up the "vmx" behemoth, but it's probably best to do that in a separate commit/series so that we can waste time bikeshedding over how to organize things :-)

Quoted 47 lines
x86/unittests.cfg +1 -0
+
 [debug]
 file = debug.flat
 arch = x86_64
x86/vmx_tests.c +28 -0
index 3d57ed6..af6f33b 100644
@@ -21,6 +21,7 @@
 #include "smp.h"
 #include "delay.h"
 #include "access.h"
+#include "x86/usermode.h"
 
 #define VPID_CAP_INVVPID_TYPES_SHIFT 40
 
@@ -10701,6 +10702,133 @@ static void vmx_pf_vpid_test(void)
 	__vmx_pf_vpid_test(invalidate_tlb_new_vpid, 1);
 }
 
+static void vmx_l2_gp_test(void)
+{
+	*(volatile u64 *)NONCANONICAL = 0;
+}
+
+static void vmx_l2_ud_test(void)
+{
+	asm volatile ("ud2");
+}
+
+static void vmx_l2_de_test(void)
+{
+	asm volatile (
+		"xor %%eax, %%eax\n\t"
+		"xor %%ebx, %%ebx\n\t"
+		"xor %%edx, %%edx\n\t"
+		"idiv %%ebx\n\t"
+		::: "eax", "ebx", "edx");
+}
+
+static void vmx_l2_bp_test(void)
+{
+	asm volatile ("int3");
+}
+
+static void vmx_l2_db_test(void)
+{

x86/vmx_tests.c static void vmx_pf_vpid_test(void) · line 10741

Quoted 10 lines
x86/vmx_tests.c +10 -0
+	write_rflags(read_rflags() | X86_EFLAGS_TF);
+}
+
+static uint64_t usermode_callback(void)
+{
+	/* Trigger an #AC by writing 8 bytes to a 4-byte aligned address. */
+	asm volatile(
+		"sub $0x10, %rsp\n\t"
+		"movq $0, 0x4(%rsp)\n\t"
+		"add $0x10, %rsp\n\t");

Sorry, didn't look closely at this before. This can simply be:

	asm volatile("movq $0, 0x4(%rsp)\n\t");

as the access is expected to fault. Or if you want to be paranoid about not overwriting the stack:

	asm volatile("movq $0, -0x4(%rsp)\n\t");

It's probably also a good idea to call out that the stack is aligned on a 16-byte boundary. If you were trying to guarnatee alignment, then you would need to use AND instead of SUB. E.g.

        asm volatile("push  %rbp\n\t"
                     "movq  %rsp, %rbp\n\t"
                     "andq  $-0x10, %rsp\n\t"
                     "movq  $0, -0x4(%rsp)\n\t"
                     "movq  %rbp, %rsp\n\t"
                     "popq  %rbp\n\t");

But my vote would be to just add a comment, I would consider it a test bug if the stack isn't properly aligned.

x86/vmx_tests.c static void vmx_pf_vpid_test(void) · line 10748

Quoted 7 lines
x86/vmx_tests.c +7 -0
+
+	return 0;
+}
+
+static void vmx_l2_ac_test(void)
+{
+	bool raised_vector = false;

Nit, hit_ac or so is more intuive.

x86/vmx_tests.c static void vmx_pf_vpid_test(void) · line 10754

Quoted 6 lines
x86/vmx_tests.c +6 -0
+
+	write_cr0(read_cr0() | X86_CR0_AM);
+	write_rflags(read_rflags() | X86_EFLAGS_AC);
+
+	run_in_user(usermode_callback, AC_VECTOR, 0, 0, 0, 0, &raised_vector);
+	report(raised_vector, "#AC vector raised from usermode in L2");

And continuing the nits, "Usermode #AC handled in L2".

Quoted 3 lines
x86/vmx_tests.c +3 -0
+	vmcall();
+}
+
#7 lore raw

Re: [kvm-unit-tests PATCH v4 3/3] x86: Add test coverage for nested_vmx_reflect_vmexit() testing

Quoted 11 lines · Sean Christopherson
Quoted 7 lines · Aaron Lewis

Add a framework and test cases to ensure exceptions that occur in L2 are forwarded to the correct place by nested_vmx_reflect_vmexit().

Add testing for exceptions: #GP, #UD, #DE, #DB, #BP, and #AC.

Change-Id I0196071571671f06165983b5055ed7382fa3e1fb

Don't forget to strip the Change-Id before posting.

D'oh... Good catch.

Quoted 12 lines
Quoted 11 lines
---
 x86/unittests.cfg |   9 +++-
 x86/vmx_tests.c   | 129 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 137 insertions(+), 1 deletion(-)
x86/unittests.cfg +0 -0
index 9a70ba3..6ec7a98 100644
@@ -288,7 +288,7 @@ arch = i386
Quoted 12 lines

x86/unittests.cfg arch = x86_64 · line 398

Quoted 6 lines
x86/unittests.cfg +6 -0
+[vmx_exception_test]
+file = vmx.flat
+extra_params = -cpu max,+vmx -append vmx_exception_test
+arch = x86_64
+groups = vmx nested_exception
+timeout = 10

Leave this out (for now), including it in the main "vmx" test is sufficient. I'm definitely in favor of splitting up the "vmx" behemoth, but it's probably best to do that in a separate commit/series so that we can waste time bikeshedding over how to organize things :-)

Why leave this out when vmx_pf_exception_test, vmx_pf_no_vpid_test, vmx_pf_invvpid_test, and vmx_pf_vpid_test have their own? They seem similar to me.

Quoted 29 lines

x86/unittests.cfg arch = i386

Quoted 8 lines
x86/unittests.cfg +8 -0
+
+static uint64_t usermode_callback(void)
+{
+     /* Trigger an #AC by writing 8 bytes to a 4-byte aligned address. */
+     asm volatile(
+             "sub $0x10, %rsp\n\t"
+             "movq $0, 0x4(%rsp)\n\t"
+             "add $0x10, %rsp\n\t");

Sorry, didn't look closely at this before. This can simply be:

        asm volatile("movq $0, 0x4(%rsp)\n\t");

as the access is expected to fault. Or if you want to be paranoid about not overwriting the stack:

        asm volatile("movq $0, -0x4(%rsp)\n\t");

It's probably also a good idea to call out that the stack is aligned on a 16-byte boundary. If you were trying to guarnatee alignment, then you would need to use AND instead of SUB. E.g.

        asm volatile("push  %rbp\n\t"
                     "movq  %rsp, %rbp\n\t"
                     "andq  $-0x10, %rsp\n\t"
                     "movq  $0, -0x4(%rsp)\n\t"
                     "movq  %rbp, %rsp\n\t"
                     "popq  %rbp\n\t");
Quoted 3 lines

But my vote would be to just add a comment, I would consider it a test bug if the stack isn't properly aligned.

I can improve the comment, and I agree that it would be a test bug if the stack isn't properly aligned.

I'll switch this to using the more paranoid approach if I'm not going to modify RSP.

Quoted 4 lines
Quoted 4 lines
x86/unittests.cfg +4 -0
+
+     return 0;
+}
+
maintainer
#8 lore raw

Re: [kvm-unit-tests PATCH v4 3/3] x86: Add test coverage for nested_vmx_reflect_vmexit() testing

Quoted 11 lines · Aaron Lewis
Quoted 11 lines

x86/unittests.cfg arch = x86_64 · line 398

Quoted 6 lines

+[vmx_exception_test]

x86/unittests.cfg +5 -0
+file = vmx.flat
+extra_params = -cpu max,+vmx -append vmx_exception_test
+arch = x86_64
+groups = vmx nested_exception
+timeout = 10

Leave this out (for now), including it in the main "vmx" test is sufficient. I'm definitely in favor of splitting up the "vmx" behemoth, but it's probably best to do that in a separate commit/series so that we can waste time bikeshedding over how to organize things:-)

Quoted 4 lines

Why leave this out when vmx_pf_exception_test, vmx_pf_no_vpid_test, vmx_pf_invvpid_test, and vmx_pf_vpid_test have their own? They seem similar to me.

Because they take a very long time (minutes).

Paolo