Skip to contentSkip to conversation

KVM: selftests: arm64 exception handling and debug test

PATCH Superseded Ricardo Koller 6 participants 33 messages Apr 30, 2021 – May 24, 2021

Subsystems (3)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover KVM: selftests: arm64 exception handling and debug test

Hi,

These patches add a debug exception test in aarch64 KVM selftests while also adding basic exception handling support.

The structure of the exception handling is based on its x86 counterpart. Tests use the same calls to initialize exception handling and both architectures allow tests to override the handler for a particular vector, or (vector, ec) for synchronous exceptions in the arm64 case.

The debug test is similar to x86_64/debug_regs, except that the x86 one controls the debugging from outside the VM. This proposed arm64 test controls and handles debug exceptions from the inside.

Thanks,
Ricardo

v1 -> v2:

Addressed comments from Andrew and Marc (thank you very much):
- rename vm_handle_exception in all tests.
- introduce UCALL_UNHANDLED in x86 first.
- move GUEST_ASSERT_EQ to common utils header.
- handle sync and other exceptions separately: use two tables (like kvm-unit-tests).
- add two separate functions for installing sync versus other exceptions
- changes in handlers.S: use the same layout as user_pt_regs, treat the EL1t vectors as invalid, refactor the vector table creation to not use manual numbering, add comments, remove LR from the stored registers.
- changes in debug-exceptions.c: remove unused headers, use the common GUEST_ASSERT_EQ, use vcpu_run instead of _vcpu_run.
- changes in processor.h: write_sysreg with support for xzr, replace EL1 with current in macro names, define ESR_EC_MASK as ESR_EC_NUM-1.

Ricardo Koller (5):
KVM: selftests: Rename vm_handle_exception
KVM: selftests: Introduce UCALL_UNHANDLED for unhandled vector
reporting
KVM: selftests: Move GUEST_ASSERT_EQ to utils header
KVM: selftests: Add exception handling support for aarch64
KVM: selftests: Add aarch64/debug-exceptions test

 tools/testing/selftests/kvm/.gitignore        |   1 +
 tools/testing/selftests/kvm/Makefile          |   3 +-
 .../selftests/kvm/aarch64/debug-exceptions.c  | 244 ++++++++++++++++++
 .../selftests/kvm/include/aarch64/processor.h |  90 ++++++-
 .../testing/selftests/kvm/include/kvm_util.h  |  10 +
 .../selftests/kvm/include/x86_64/processor.h  |   4 +-
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++
 .../selftests/kvm/lib/x86_64/processor.c      |  19 +-
 .../selftests/kvm/x86_64/kvm_pv_test.c        |   2 +-
 .../selftests/kvm/x86_64/tsc_msrs_test.c      |   9 -
 .../kvm/x86_64/userspace_msr_exit_test.c      |   8 +-
 .../selftests/kvm/x86_64/xapic_ipi_test.c     |   2 +-
 13 files changed, 611 insertions(+), 35 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/aarch64/debug-exceptions.c
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
Signature · 1 line
2.31.1.527.g47e6f16901-goog
#1 lore raw

PATCH v2 1/5 KVM: selftests: Rename vm_handle_exception

Rename the vm_handle_exception function to a name that indicates more clearly that it installs something: vm_install_vector_handler.

---
 tools/testing/selftests/kvm/include/x86_64/processor.h    | 2 +-
 tools/testing/selftests/kvm/lib/x86_64/processor.c        | 4 ++--
 tools/testing/selftests/kvm/x86_64/kvm_pv_test.c          | 2 +-
 .../selftests/kvm/x86_64/userspace_msr_exit_test.c        | 8 ++++----
 tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c       | 2 +-
 5 files changed, 9 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/include/x86_64/processor.h +1 -1
index 0b30b4e15c38..12889d3e8948 100644
@@ -391,7 +391,7 @@ struct ex_regs {
 
 void vm_init_descriptor_tables(struct kvm_vm *vm);
 void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
-void vm_handle_exception(struct kvm_vm *vm, int vector,
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
 			void (*handler)(struct ex_regs *));
 
 /*
tools/testing/selftests/kvm/lib/x86_64/processor.c +2 -2
index a8906e60a108..e156061263a6 100644
@@ -1250,8 +1250,8 @@ void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
 	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
 }
 
-void vm_handle_exception(struct kvm_vm *vm, int vector,
-			 void (*handler)(struct ex_regs *))
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			       void (*handler)(struct ex_regs *))
 {
 	vm_vaddr_t *handlers = (vm_vaddr_t *)addr_gva2hva(vm, vm->handlers);
 
tools/testing/selftests/kvm/x86_64/kvm_pv_test.c +1 -1
index 732b244d6956..5ae5f748723a 100644
@@ -227,7 +227,7 @@ int main(void)
 
 	vm_init_descriptor_tables(vm);
 	vcpu_init_descriptor_tables(vm, VCPU_ID);
-	vm_handle_exception(vm, GP_VECTOR, guest_gp_handler);
+	vm_install_vector_handler(vm, GP_VECTOR, guest_gp_handler);
 
 	enter_guest(vm);
 	kvm_vm_free(vm);
tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c +4 -4
index 72c0d0797522..20c373e2d329 100644
@@ -574,7 +574,7 @@ static void test_msr_filter_allow(void) {
 	vm_init_descriptor_tables(vm);
 	vcpu_init_descriptor_tables(vm, VCPU_ID);
 
-	vm_handle_exception(vm, GP_VECTOR, guest_gp_handler);
+	vm_install_vector_handler(vm, GP_VECTOR, guest_gp_handler);
 
 	/* Process guest code userspace exits. */
 	run_guest_then_process_rdmsr(vm, MSR_IA32_XSS);
@@ -588,12 +588,12 @@ static void test_msr_filter_allow(void) {
 	run_guest_then_process_wrmsr(vm, MSR_NON_EXISTENT);
 	run_guest_then_process_rdmsr(vm, MSR_NON_EXISTENT);
 
-	vm_handle_exception(vm, UD_VECTOR, guest_ud_handler);
+	vm_install_vector_handler(vm, UD_VECTOR, guest_ud_handler);
 	run_guest(vm);
-	vm_handle_exception(vm, UD_VECTOR, NULL);
+	vm_install_vector_handler(vm, UD_VECTOR, NULL);
 
 	if (process_ucall(vm) != UCALL_DONE) {
-		vm_handle_exception(vm, GP_VECTOR, guest_fep_gp_handler);
+		vm_install_vector_handler(vm, GP_VECTOR, guest_fep_gp_handler);
 
 		/* Process emulated rdmsr and wrmsr instructions. */
 		run_guest_then_process_rdmsr(vm, MSR_IA32_XSS);
tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c +1 -1
index 2f964cdc273c..ded70ff465d5 100644
@@ -462,7 +462,7 @@ int main(int argc, char *argv[])
 
 	vm_init_descriptor_tables(vm);
 	vcpu_init_descriptor_tables(vm, HALTER_VCPU_ID);
-	vm_handle_exception(vm, IPI_VECTOR, guest_ipi_handler);
+	vm_install_vector_handler(vm, IPI_VECTOR, guest_ipi_handler);
 
 	virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA, 0);
 
Signature · 1 line
2.31.1.527.g47e6f16901-goog
Reviewed-by on v2 1/5
#2 lore raw

Re: [PATCH v2 1/5] KVM: selftests: Rename vm_handle_exception

Quoted 6 lines · Ricardo Koller

Rename the vm_handle_exception function to a name that indicates more clearly that it installs something: vm_install_vector_handler.

Quoted 8 lines
---
 tools/testing/selftests/kvm/include/x86_64/processor.h    | 2 +-
 tools/testing/selftests/kvm/lib/x86_64/processor.c        | 4 ++--
 tools/testing/selftests/kvm/x86_64/kvm_pv_test.c          | 2 +-
 .../selftests/kvm/x86_64/userspace_msr_exit_test.c        | 8 ++++----
 tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c       | 2 +-
 5 files changed, 9 insertions(+), 9 deletions(-)
Reviewed-by on v2 1/5
#3 lore raw

Re: [PATCH v2 1/5] KVM: selftests: Rename vm_handle_exception

In reply to Ricardo Koller

Hi Ricardo,

Quoted 6 lines · Ricardo Koller

Rename the vm_handle_exception function to a name that indicates more clearly that it installs something: vm_install_vector_handler.

Thanks

Eric

Quoted 92 lines
---
 tools/testing/selftests/kvm/include/x86_64/processor.h    | 2 +-
 tools/testing/selftests/kvm/lib/x86_64/processor.c        | 4 ++--
 tools/testing/selftests/kvm/x86_64/kvm_pv_test.c          | 2 +-
 .../selftests/kvm/x86_64/userspace_msr_exit_test.c        | 8 ++++----
 tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c       | 2 +-
 5 files changed, 9 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/include/x86_64/processor.h +1 -1
index 0b30b4e15c38..12889d3e8948 100644
@@ -391,7 +391,7 @@ struct ex_regs {
 
 void vm_init_descriptor_tables(struct kvm_vm *vm);
 void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
-void vm_handle_exception(struct kvm_vm *vm, int vector,
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
 			void (*handler)(struct ex_regs *));
 
 /*
tools/testing/selftests/kvm/lib/x86_64/processor.c +2 -2
index a8906e60a108..e156061263a6 100644
@@ -1250,8 +1250,8 @@ void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
 	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
 }
 
-void vm_handle_exception(struct kvm_vm *vm, int vector,
-			 void (*handler)(struct ex_regs *))
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			       void (*handler)(struct ex_regs *))
 {
 	vm_vaddr_t *handlers = (vm_vaddr_t *)addr_gva2hva(vm, vm->handlers);
 
tools/testing/selftests/kvm/x86_64/kvm_pv_test.c +1 -1
index 732b244d6956..5ae5f748723a 100644
@@ -227,7 +227,7 @@ int main(void)
 
 	vm_init_descriptor_tables(vm);
 	vcpu_init_descriptor_tables(vm, VCPU_ID);
-	vm_handle_exception(vm, GP_VECTOR, guest_gp_handler);
+	vm_install_vector_handler(vm, GP_VECTOR, guest_gp_handler);
 
 	enter_guest(vm);
 	kvm_vm_free(vm);
tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c +4 -4
index 72c0d0797522..20c373e2d329 100644
@@ -574,7 +574,7 @@ static void test_msr_filter_allow(void) {
 	vm_init_descriptor_tables(vm);
 	vcpu_init_descriptor_tables(vm, VCPU_ID);
 
-	vm_handle_exception(vm, GP_VECTOR, guest_gp_handler);
+	vm_install_vector_handler(vm, GP_VECTOR, guest_gp_handler);
 
 	/* Process guest code userspace exits. */
 	run_guest_then_process_rdmsr(vm, MSR_IA32_XSS);
@@ -588,12 +588,12 @@ static void test_msr_filter_allow(void) {
 	run_guest_then_process_wrmsr(vm, MSR_NON_EXISTENT);
 	run_guest_then_process_rdmsr(vm, MSR_NON_EXISTENT);
 
-	vm_handle_exception(vm, UD_VECTOR, guest_ud_handler);
+	vm_install_vector_handler(vm, UD_VECTOR, guest_ud_handler);
 	run_guest(vm);
-	vm_handle_exception(vm, UD_VECTOR, NULL);
+	vm_install_vector_handler(vm, UD_VECTOR, NULL);
 
 	if (process_ucall(vm) != UCALL_DONE) {
-		vm_handle_exception(vm, GP_VECTOR, guest_fep_gp_handler);
+		vm_install_vector_handler(vm, GP_VECTOR, guest_fep_gp_handler);
 
 		/* Process emulated rdmsr and wrmsr instructions. */
 		run_guest_then_process_rdmsr(vm, MSR_IA32_XSS);
tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c +1 -1
index 2f964cdc273c..ded70ff465d5 100644
@@ -462,7 +462,7 @@ int main(int argc, char *argv[])
 
 	vm_init_descriptor_tables(vm);
 	vcpu_init_descriptor_tables(vm, HALTER_VCPU_ID);
-	vm_handle_exception(vm, IPI_VECTOR, guest_ipi_handler);
+	vm_install_vector_handler(vm, IPI_VECTOR, guest_ipi_handler);
 
 	virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA, 0);
 
#4 lore raw

PATCH v2 2/5 KVM: selftests: Introduce UCALL_UNHANDLED for unhandled vector reporting

In reply to Ricardo Koller

x86, the only arch implementing exception handling, reports unhandled vectors using port IO at a specific port number. This replicates what ucall already does.

Introduce a new ucall type, UCALL_UNHANDLED, for guests to report unhandled exceptions. Then replace the x86 unhandled vector exception reporting to use it instead of port IO. This new ucall type will be used in the next commits by arm64 to report unhandled vectors as well.

Tested: Forcing a page fault in the ./x86_64/xapic_ipi_test

	halter_guest_code() shows this:

	$ ./x86_64/xapic_ipi_test
	...
	  Unexpected vectored event in guest (vector:0xe)
---
 tools/testing/selftests/kvm/include/kvm_util.h    |  1 +
 .../selftests/kvm/include/x86_64/processor.h      |  2 --
 .../testing/selftests/kvm/lib/x86_64/processor.c  | 15 ++++++---------
 3 files changed, 7 insertions(+), 11 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +1 -0
index bea4644d645d..7880929ea548 100644
@@ -347,6 +347,7 @@ enum {
 	UCALL_SYNC,
 	UCALL_ABORT,
 	UCALL_DONE,
+	UCALL_UNHANDLED,
 };
 
 #define UCALL_MAX_ARGS 6
tools/testing/selftests/kvm/include/x86_64/processor.h +0 -2
index 12889d3e8948..ff4da2f95b13 100644
@@ -53,8 +53,6 @@
 #define CPUID_PKU		(1ul << 3)
 #define CPUID_LA57		(1ul << 16)
 
-#define UNEXPECTED_VECTOR_PORT 0xfff0u
-
 /* General Registers in 64-Bit Mode */
 struct gpr64_regs {
 	u64 rax;
tools/testing/selftests/kvm/lib/x86_64/processor.c +6 -9
index e156061263a6..96e2bd9d66eb 100644
@@ -1207,7 +1207,7 @@ static void set_idt_entry(struct kvm_vm *vm, int vector, unsigned long addr,
 
 void kvm_exit_unexpected_vector(uint32_t value)
 {
-	outl(UNEXPECTED_VECTOR_PORT, value);
+	ucall(UCALL_UNHANDLED, 1, value);
 }
 
 void route_exception(struct ex_regs *regs)
@@ -1260,16 +1260,13 @@ void vm_install_vector_handler(struct kvm_vm *vm, int vector,
 
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
-	if (vcpu_state(vm, vcpuid)->exit_reason == KVM_EXIT_IO
-		&& vcpu_state(vm, vcpuid)->io.port == UNEXPECTED_VECTOR_PORT
-		&& vcpu_state(vm, vcpuid)->io.size == 4) {
-		/* Grab pointer to io data */
-		uint32_t *data = (void *)vcpu_state(vm, vcpuid)
-			+ vcpu_state(vm, vcpuid)->io.data_offset;
+	struct ucall uc;
 
+	if (get_ucall(vm, vcpuid, &uc) == UCALL_UNHANDLED) {
+		uint64_t vector = uc.args[0];
 		TEST_ASSERT(false,
-			    "Unexpected vectored event in guest (vector:0x%x)",
-			    *data);
+			    "Unexpected vectored event in guest (vector:0x%lx)",
+			    vector);
 	}
 }
 
Signature · 1 line
2.31.1.527.g47e6f16901-goog
Reviewed-by on v2 2/5
#5 lore raw

Re: [PATCH v2 2/5] KVM: selftests: Introduce UCALL_UNHANDLED for unhandled vector reporting

Quoted 70 lines · Ricardo Koller

x86, the only arch implementing exception handling, reports unhandled vectors using port IO at a specific port number. This replicates what ucall already does.

Introduce a new ucall type, UCALL_UNHANDLED, for guests to report unhandled exceptions. Then replace the x86 unhandled vector exception reporting to use it instead of port IO. This new ucall type will be used in the next commits by arm64 to report unhandled vectors as well.

Tested: Forcing a page fault in the ./x86_64/xapic_ipi_test

	halter_guest_code() shows this:

	$ ./x86_64/xapic_ipi_test
	...
	  Unexpected vectored event in guest (vector:0xe)
---
 tools/testing/selftests/kvm/include/kvm_util.h    |  1 +
 .../selftests/kvm/include/x86_64/processor.h      |  2 --
 .../testing/selftests/kvm/lib/x86_64/processor.c  | 15 ++++++---------
 3 files changed, 7 insertions(+), 11 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +1 -0
index bea4644d645d..7880929ea548 100644
@@ -347,6 +347,7 @@ enum {
 	UCALL_SYNC,
 	UCALL_ABORT,
 	UCALL_DONE,
+	UCALL_UNHANDLED,
 };
 
 #define UCALL_MAX_ARGS 6
tools/testing/selftests/kvm/include/x86_64/processor.h +0 -2
index 12889d3e8948..ff4da2f95b13 100644
@@ -53,8 +53,6 @@
 #define CPUID_PKU		(1ul << 3)
 #define CPUID_LA57		(1ul << 16)
 
-#define UNEXPECTED_VECTOR_PORT 0xfff0u
-
 /* General Registers in 64-Bit Mode */
 struct gpr64_regs {
 	u64 rax;
tools/testing/selftests/kvm/lib/x86_64/processor.c +1 -6
index e156061263a6..96e2bd9d66eb 100644
@@ -1207,7 +1207,7 @@ static void set_idt_entry(struct kvm_vm *vm, int vector, unsigned long addr,
 
 void kvm_exit_unexpected_vector(uint32_t value)
 {
-	outl(UNEXPECTED_VECTOR_PORT, value);
+	ucall(UCALL_UNHANDLED, 1, value);
 }
 
 void route_exception(struct ex_regs *regs)
@@ -1260,16 +1260,13 @@ void vm_install_vector_handler(struct kvm_vm *vm, int vector,
 
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
-	if (vcpu_state(vm, vcpuid)->exit_reason == KVM_EXIT_IO
-		&& vcpu_state(vm, vcpuid)->io.port == UNEXPECTED_VECTOR_PORT
-		&& vcpu_state(vm, vcpuid)->io.size == 4) {
-		/* Grab pointer to io data */
-		uint32_t *data = (void *)vcpu_state(vm, vcpuid)

tools/testing/selftests/kvm/lib/x86_64/processor.c void vm_install_vector_handler(struct kvm_vm *vm, int vector, · line 1269

Quoted 10 lines
tools/testing/selftests/kvm/lib/x86_64/processor.c +5 -3
-			+ vcpu_state(vm, vcpuid)->io.data_offset;
+	struct ucall uc;
 
+	if (get_ucall(vm, vcpuid, &uc) == UCALL_UNHANDLED) {
+		uint64_t vector = uc.args[0];
 		TEST_ASSERT(false,
-			    "Unexpected vectored event in guest (vector:0x%x)",
-			    *data);
+			    "Unexpected vectored event in guest (vector:0x%lx)",
+			    vector);

nit: Could have changed this TEST_ASSERT(false, ...) to TEST_FAIL while touching it.

Quoted 6 lines
tools/testing/selftests/kvm/lib/x86_64/processor.c +0 -0
 	}
 }
 
Signature · 1 line
2.31.1.527.g47e6f16901-goog

Thanks,
drew

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Reviewed-by
#6 lore raw

Re: [PATCH v2 2/5] KVM: selftests: Introduce UCALL_UNHANDLED for unhandled vector reporting

Hi Ricardo,

Quoted 10 lines · Andrew Jones
Quoted 9 lines · Ricardo Koller

x86, the only arch implementing exception handling, reports unhandled vectors using port IO at a specific port number. This replicates what ucall already does.

Introduce a new ucall type, UCALL_UNHANDLED, for guests to report unhandled exceptions. Then replace the x86 unhandled vector exception reporting to use it instead of port IO. This new ucall type will be used in the next commits by arm64 to report unhandled vectors as well.

Quoted 8 lines
Quoted 8 lines

Tested: Forcing a page fault in the ./x86_64/xapic_ipi_test

	halter_guest_code() shows this:

	$ ./x86_64/xapic_ipi_test
	...
	  Unexpected vectored event in guest (vector:0xe)

Eric

Quoted 79 lines
Quoted 53 lines
---
 tools/testing/selftests/kvm/include/kvm_util.h    |  1 +
 .../selftests/kvm/include/x86_64/processor.h      |  2 --
 .../testing/selftests/kvm/lib/x86_64/processor.c  | 15 ++++++---------
 3 files changed, 7 insertions(+), 11 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +1 -0
index bea4644d645d..7880929ea548 100644
@@ -347,6 +347,7 @@ enum {
 	UCALL_SYNC,
 	UCALL_ABORT,
 	UCALL_DONE,
+	UCALL_UNHANDLED,
 };
 
 #define UCALL_MAX_ARGS 6
tools/testing/selftests/kvm/include/x86_64/processor.h +0 -2
index 12889d3e8948..ff4da2f95b13 100644
@@ -53,8 +53,6 @@
 #define CPUID_PKU		(1ul << 3)
 #define CPUID_LA57		(1ul << 16)
 
-#define UNEXPECTED_VECTOR_PORT 0xfff0u
-
 /* General Registers in 64-Bit Mode */
 struct gpr64_regs {
 	u64 rax;
tools/testing/selftests/kvm/lib/x86_64/processor.c +1 -6
index e156061263a6..96e2bd9d66eb 100644
@@ -1207,7 +1207,7 @@ static void set_idt_entry(struct kvm_vm *vm, int vector, unsigned long addr,
 
 void kvm_exit_unexpected_vector(uint32_t value)
 {
-	outl(UNEXPECTED_VECTOR_PORT, value);
+	ucall(UCALL_UNHANDLED, 1, value);
 }
 
 void route_exception(struct ex_regs *regs)
@@ -1260,16 +1260,13 @@ void vm_install_vector_handler(struct kvm_vm *vm, int vector,
 
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
-	if (vcpu_state(vm, vcpuid)->exit_reason == KVM_EXIT_IO
-		&& vcpu_state(vm, vcpuid)->io.port == UNEXPECTED_VECTOR_PORT
-		&& vcpu_state(vm, vcpuid)->io.size == 4) {
-		/* Grab pointer to io data */
-		uint32_t *data = (void *)vcpu_state(vm, vcpuid)

tools/testing/selftests/kvm/lib/x86_64/processor.c void vm_install_vector_handler(struct kvm_vm *vm, int vector, · line 1269

Quoted 10 lines
tools/testing/selftests/kvm/lib/x86_64/processor.c +5 -3
-			+ vcpu_state(vm, vcpuid)->io.data_offset;
+	struct ucall uc;
 
+	if (get_ucall(vm, vcpuid, &uc) == UCALL_UNHANDLED) {
+		uint64_t vector = uc.args[0];
 		TEST_ASSERT(false,
-			    "Unexpected vectored event in guest (vector:0x%x)",
-			    *data);
+			    "Unexpected vectored event in guest (vector:0x%lx)",
+			    vector);

nit: Could have changed this TEST_ASSERT(false, ...) to TEST_FAIL while touching it.

Quoted 6 lines
tools/testing/selftests/kvm/lib/x86_64/processor.c +0 -0
 	}
 }
 
Signature · 1 line
2.31.1.527.g47e6f16901-goog

Thanks,
drew

#7 lore raw

PATCH v2 3/5 KVM: selftests: Move GUEST_ASSERT_EQ to utils header

In reply to Ricardo Koller

Move GUEST_ASSERT_EQ to a common header, kvm_util.h, for other architectures and tests to use.

---
 tools/testing/selftests/kvm/include/kvm_util.h     | 9 +++++++++
 tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c | 9 ---------
 2 files changed, 9 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +9 -0
index 7880929ea548..bd26dd93ab56 100644
@@ -388,4 +388,13 @@ uint64_t get_ucall(struct kvm_vm *vm, uint32_t vcpu_id, struct ucall *uc);
 #define GUEST_ASSERT_4(_condition, arg1, arg2, arg3, arg4) \
 	__GUEST_ASSERT((_condition), 4, (arg1), (arg2), (arg3), (arg4))
 
+#define GUEST_ASSERT_EQ(a, b) do {				\
+	__typeof(a) _a = (a);					\
+	__typeof(b) _b = (b);					\
+	if (_a != _b)						\
+		ucall(UCALL_ABORT, 4,				\
+			"Failed guest assert: "			\
+			#a " == " #b, __LINE__, _a, _b);	\
+} while(0)
+
 #endif /* SELFTEST_KVM_UTIL_H */
tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c +0 -9
index e357d8e222d4..5a6a662f2e59 100644
@@ -18,15 +18,6 @@
 #define rounded_rdmsr(x)       ROUND(rdmsr(x))
 #define rounded_host_rdmsr(x)  ROUND(vcpu_get_msr(vm, 0, x))
 
-#define GUEST_ASSERT_EQ(a, b) do {				\
-	__typeof(a) _a = (a);					\
-	__typeof(b) _b = (b);					\
-	if (_a != _b)						\
-                ucall(UCALL_ABORT, 4,				\
-                        "Failed guest assert: "			\
-                        #a " == " #b, __LINE__, _a, _b);	\
-  } while(0)
-
 static void guest_code(void)
 {
 	u64 val = 0;
Signature · 1 line
2.31.1.527.g47e6f16901-goog
#8 lore raw

Re: [PATCH v2 3/5] KVM: selftests: Move GUEST_ASSERT_EQ to utils header

Quoted 50 lines · Ricardo Koller

Move GUEST_ASSERT_EQ to a common header, kvm_util.h, for other architectures and tests to use.

---
 tools/testing/selftests/kvm/include/kvm_util.h     | 9 +++++++++
 tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c | 9 ---------
 2 files changed, 9 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/include/kvm_util.h +9 -0
index 7880929ea548..bd26dd93ab56 100644
@@ -388,4 +388,13 @@ uint64_t get_ucall(struct kvm_vm *vm, uint32_t vcpu_id, struct ucall *uc);
 #define GUEST_ASSERT_4(_condition, arg1, arg2, arg3, arg4) \
 	__GUEST_ASSERT((_condition), 4, (arg1), (arg2), (arg3), (arg4))
 
+#define GUEST_ASSERT_EQ(a, b) do {				\
+	__typeof(a) _a = (a);					\
+	__typeof(b) _b = (b);					\
+	if (_a != _b)						\
+		ucall(UCALL_ABORT, 4,				\
+			"Failed guest assert: "			\
+			#a " == " #b, __LINE__, _a, _b);	\
+} while(0)
+
 #endif /* SELFTEST_KVM_UTIL_H */
tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c +0 -9
index e357d8e222d4..5a6a662f2e59 100644
@@ -18,15 +18,6 @@
 #define rounded_rdmsr(x)       ROUND(rdmsr(x))
 #define rounded_host_rdmsr(x)  ROUND(vcpu_get_msr(vm, 0, x))
 
-#define GUEST_ASSERT_EQ(a, b) do {				\
-	__typeof(a) _a = (a);					\
-	__typeof(b) _b = (b);					\
-	if (_a != _b)						\
-                ucall(UCALL_ABORT, 4,				\
-                        "Failed guest assert: "			\
-                        #a " == " #b, __LINE__, _a, _b);	\
-  } while(0)
-
 static void guest_code(void)
 {
 	u64 val = 0;
Signature · 1 line
2.31.1.527.g47e6f16901-goog

How about modify __GUEST_ASSERT so we can reuse it instead, like below? (I also took the opportunity to remove the unnecessary () within the comma separated statements.)

Thanks,
drew

-#define __GUEST_ASSERT(_condition, _nargs, _args...) do {      \
-       if (!(_condition))                                      \
-               ucall(UCALL_ABORT, 2 + _nargs,                  \
-                       "Failed guest assert: "                 \
-                       #_condition, __LINE__, _args);          \
+#define __GUEST_ASSERT(_condition, _condstr, _nargs, _args...) do {    \
+       if (!(_condition))                                              \
+               ucall(UCALL_ABORT, 2 + _nargs,                          \
+                       "Failed guest assert: "                         \
+                       _condstr, __LINE__, _args);                     \

} while (0)

#define GUEST_ASSERT(_condition) \
- __GUEST_ASSERT((_condition), 0, 0)
+ __GUEST_ASSERT(_condition, #_condition, 0, 0)

#define GUEST_ASSERT_1(_condition, arg1) \
- __GUEST_ASSERT((_condition), 1, (arg1))
+ __GUEST_ASSERT(_condition, #_condition, 1, arg1)

#define GUEST_ASSERT_2(_condition, arg1, arg2) \
- __GUEST_ASSERT((_condition), 2, (arg1), (arg2))
+ __GUEST_ASSERT(_condition, #_condition, 2, arg1, arg2)

#define GUEST_ASSERT_3(_condition, arg1, arg2, arg3) \
- __GUEST_ASSERT((_condition), 3, (arg1), (arg2), (arg3))
+ __GUEST_ASSERT(_condition, #_condition, 3, arg1, arg2, arg3)

#define GUEST_ASSERT_4(_condition, arg1, arg2, arg3, arg4) \
- __GUEST_ASSERT((_condition), 4, (arg1), (arg2), (arg3), (arg4))
-

-#define GUEST_ASSERT_EQ(a, b) do {                             \
-       __typeof(a) _a = (a);                                   \
-       __typeof(b) _b = (b);                                   \
-       if (_a != _b)                                           \
-               ucall(UCALL_ABORT, 4,                           \
-                       "Failed guest assert: "                 \
-                       #a " == " #b, __LINE__, _a, _b);        \

-} while(0)
+ __GUEST_ASSERT(_condition, #_condition, 4, arg1, arg2, arg3, arg4)
+
+#define GUEST_ASSERT_EQ(a, b) __GUEST_ASSERT((a) == (b), #a " == " #b, 2, a, b)

#9 lore raw

PATCH v2 4/5 KVM: selftests: Add exception handling support for aarch64

In reply to Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

The exception handling code is heavily inspired on kvm-unit-tests.

---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
tools/testing/selftests/kvm/Makefile +1 -1
index 4e548d7ab0ab..618c5903f478 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/processor.h +78 -0

94 lines not on this page · raw message

tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +130 -0

130 lines not on this page · raw message

tools/testing/selftests/kvm/lib/aarch64/processor.c +124 -0

142 lines not on this page · raw message

Signature · 1 line
2.31.1.527.g47e6f16901-goog
maintainer
#10 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 152 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

The exception handling code is heavily inspired on kvm-unit-tests.

---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
tools/testing/selftests/kvm/Makefile +1 -1
index 4e548d7ab0ab..618c5903f478 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/processor.h +78 -0
index b7fa0c8551db..40aae31b4afc 100644
@@ -8,6 +8,7 @@
 #define SELFTEST_KVM_PROCESSOR_H
 
 #include "kvm_util.h"
+#include <linux/stringify.h>
 
 
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
@@ -18,6 +19,7 @@
 #define MAIR_EL1	3, 0, 10, 2, 0
 #define TTBR0_EL1	3, 0,  2, 0, 0
 #define SCTLR_EL1	3, 0,  1, 0, 0
+#define VBAR_EL1	3, 0, 12, 0, 0
 
 /*
  * Default MAIR
@@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
 void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
 			      struct kvm_vcpu_init *init, void *guest_code);
 
+struct ex_regs {
+	u64 regs[31];
+	u64 sp;
+	u64 pc;
+	u64 pstate;
+};
+
+#define VECTOR_NUM	16
+
+enum {
+	VECTOR_SYNC_CURRENT_SP0,
+	VECTOR_IRQ_CURRENT_SP0,
+	VECTOR_FIQ_CURRENT_SP0,
+	VECTOR_ERROR_CURRENT_SP0,
+
+	VECTOR_SYNC_CURRENT,
+	VECTOR_IRQ_CURRENT,
+	VECTOR_FIQ_CURRENT,
+	VECTOR_ERROR_CURRENT,
+
+	VECTOR_SYNC_LOWER_64,
+	VECTOR_IRQ_LOWER_64,
+	VECTOR_FIQ_LOWER_64,
+	VECTOR_ERROR_LOWER_64,
+
+	VECTOR_SYNC_LOWER_32,
+	VECTOR_IRQ_LOWER_32,
+	VECTOR_FIQ_LOWER_32,
+	VECTOR_ERROR_LOWER_32,
+};
+
+#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
+			   (v) == VECTOR_SYNC_CURRENT     || \
+			   (v) == VECTOR_SYNC_LOWER_64    || \
+			   (v) == VECTOR_SYNC_LOWER_32)
+
+/* Some common EC (Exception classes) */
+#define ESR_EC_ILLEGAL_INS	0x0e
+#define ESR_EC_SVC64		0x15
+#define ESR_EC_IABORT_CURRENT	0x21
+#define ESR_EC_DABORT_CURRENT	0x25
+#define ESR_EC_SERROR		0x2f
+#define ESR_EC_HW_BP_CURRENT	0x31
+#define ESR_EC_SSTEP_CURRENT	0x33
+#define ESR_EC_WP_CURRENT	0x35
+#define ESR_EC_BRK_INS		0x3C
+
+#define ESR_EC_NUM		64
+
+#define ESR_EC_SHIFT		26
+#define ESR_EC_MASK		(ESR_EC_NUM - 1)
+
+void vm_init_descriptor_tables(struct kvm_vm *vm);
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
+
+typedef void(*handler_fn)(struct ex_regs *);
+void vm_install_exception_handler(struct kvm_vm *vm,
+		int vector, int ec, handler_fn handler);
+void vm_install_vector_handler(struct kvm_vm *vm,
+		int vector, handler_fn handler);
+
+#define SPSR_D          (1 << 9)
+#define SPSR_SS         (1 << 21)
+
+#define write_sysreg(reg, val)						  \
+({									  \
+	u64 __val = (u64)(val);						  \
+	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
+})
+
+#define read_sysreg(reg)						  \
+({	u64 val;							  \
+	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
+	val;								  \
+})
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +12 -0
new file mode 100644
index 000000000000..8a560021892b
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+.macro save_registers, vector
+	add	sp, sp, #-16 * 17
+
+	stp	x0, x1, [sp, #16 * 0]
+	stp	x2, x3, [sp, #16 * 1]
+	stp	x4, x5, [sp, #16 * 2]
+	stp	x6, x7, [sp, #16 * 3]
+	stp	x8, x9, [sp, #16 * 4]
+	stp	x10, x11, [sp, #16 * 5]
+	stp	x12, x13, [sp, #16 * 6]
+	stp	x14, x15, [sp, #16 * 7]

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 22

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+	stp	x16, x17, [sp, #16 * 8]
+	stp	x18, x19, [sp, #16 * 9]
+	stp	x20, x21, [sp, #16 * 10]
+	stp	x22, x23, [sp, #16 * 11]
+	stp	x24, x25, [sp, #16 * 12]
+	stp	x26, x27, [sp, #16 * 13]
+	stp	x28, x29, [sp, #16 * 14]
+
+	.if \vector >= 8
+	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you always run in handler mode. Therefore, saving/restoring it is only overhead. If an exception handler wants to introspect it, it is already available in the relevant system register.

Or did you have something else in mind for it?

Quoted 223 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +108 -0
+	.else
+	/*
+	 * This stores sp_el1 into ex_regs.sp so exception handlers can
+	 * "look" at it. It will _not_ be used to restore the sp_el1 on
+	 * return from the exception so handlers can not update it.
+	 */
+	mov	x1, sp
+	.endif
+	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
+
+	mrs	x1, elr_el1
+	mrs	x2, spsr_el1
+	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+.endm
+
+.macro restore_registers, vector
+	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+	msr	elr_el1, x1
+	msr	spsr_el1, x2
+
+	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
+	.if \vector >= 8
+	msr	sp_el0, x1
+	.endif
+
+	ldp	x28, x29, [sp, #16 * 14]
+	ldp	x26, x27, [sp, #16 * 13]
+	ldp	x24, x25, [sp, #16 * 12]
+	ldp	x22, x23, [sp, #16 * 11]
+	ldp	x20, x21, [sp, #16 * 10]
+	ldp	x18, x19, [sp, #16 * 9]
+	ldp	x16, x17, [sp, #16 * 8]
+	ldp	x14, x15, [sp, #16 * 7]
+	ldp	x12, x13, [sp, #16 * 6]
+	ldp	x10, x11, [sp, #16 * 5]
+	ldp	x8, x9, [sp, #16 * 4]
+	ldp	x6, x7, [sp, #16 * 3]
+	ldp	x4, x5, [sp, #16 * 2]
+	ldp	x2, x3, [sp, #16 * 1]
+	ldp	x0, x1, [sp, #16 * 0]
+
+	add	sp, sp, #16 * 17
+
+	eret
+.endm
+
+.pushsection ".entry.text", "ax"
+.balign 0x800
+.global vectors
+vectors:
+.popsection
+
+.set	vector, 0
+
+/*
+ * Build an exception handler for vector and append a jump to it into
+ * vectors (while making sure that it's 0x80 aligned).
+ */
+.macro HANDLER, label
+handler_\()\label:
+	save_registers vector
+	mov	x0, sp
+	mov	x1, #vector
+	bl	route_exception
+	restore_registers vector
+
+.pushsection ".entry.text", "ax"
+.balign 0x80
+	b	handler_\()\label
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+.macro HANDLER_INVALID
+.pushsection ".entry.text", "ax"
+.balign 0x80
+/* This will abort so no need to save and restore registers. */
+	mov	x0, #vector
+	b	kvm_exit_unexpected_vector
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+/*
+ * Caution: be sure to not add anything between the declaration of vectors
+ * above and these macro calls that will build the vectors table below it.
+ */
+	HANDLER_INVALID                         // Synchronous EL1t
+	HANDLER_INVALID                         // IRQ EL1t
+	HANDLER_INVALID                         // FIQ EL1t
+	HANDLER_INVALID                         // Error EL1t
+
+	HANDLER	el1h_sync                       // Synchronous EL1h
+	HANDLER	el1h_irq                        // IRQ EL1h
+	HANDLER el1h_fiq                        // FIQ EL1h
+	HANDLER	el1h_error                      // Error EL1h
+
+	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
+	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
+	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
+	HANDLER	el0_error_64                    // Error 64-bit EL0
+
+	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
+	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
+	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
+	HANDLER	el0_error_32                    // Error 32-bit EL0
tools/testing/selftests/kvm/lib/aarch64/processor.c +91 -0
index cee92d477dc0..25be71ec88be 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/compiler.h>
+#include <assert.h>
 
 #include "kvm_util.h"
 #include "../kvm_util_internal.h"
@@ -14,6 +15,8 @@
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
 #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
 
+vm_vaddr_t exception_handlers;
+
 static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
 {
 	return (v + vm->page_size) & ~(vm->page_size - 1);
@@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
 	va_end(ap);
 }
 
+void kvm_exit_unexpected_vector(int vector)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
+}
+
+void kvm_exit_unexpected_exception(int vector, uint64_t ec)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
+}
+
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
+	struct ucall uc;
+
+	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
+		return;
+
+	if (uc.args[2]) /* valid_ec */ {
+		assert(VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
+			uc.args[0], uc.args[1]);
+	} else {
+		assert(!VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx)",
+			uc.args[0]);
+	}
+}
+
+/*
+ * This exception handling code was heavily inspired on kvm-unit-tests. There
+ * is a set of default vector handlers stored in vector_handlers. These default
+ * vector handlers call user-installed handlers stored in exception_handlers.
+ * Synchronous handlers are indexed by (vector, ec), and irq handlers by
+ * (vector, ec=0).
+ */
+
+typedef void(*vector_fn)(struct ex_regs *, int vector);
+
+struct handlers {
+	vector_fn vector_handlers[VECTOR_NUM];
+	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
+};
+
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
+{
+	extern char vectors;
+
+	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
+}
+
+void default_sync_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+	uint64_t esr = read_sysreg(esr_el1);
+	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
+
+	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][ec])
+		handlers->exception_handlers[vector][ec](regs);
+	else
+		kvm_exit_unexpected_exception(vector, ec);
+}
+
+void default_irq_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][0])
+		handlers->exception_handlers[vector][0](regs);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void route_exception(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	if (handlers && handlers->vector_handlers[vector])
+		handlers->vector_handlers[vector](regs, vector);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void vm_init_descriptor_tables(struct kvm_vm *vm)
+{

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 439

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -0
+	struct handlers *handlers;
+
+	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
+			vm->page_size, 0, 0);
+
+	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
+	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;

How about FIQ, Error? Although they are unlikely, they are valid exceptions.

Quoted 14 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +14 -0
+
+	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
+}
+
+void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	assert(ec < ESR_EC_NUM);
+	handlers->exception_handlers[vector][ec] = handler;
+}

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 463

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -0
+
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(!VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	handlers->exception_handlers[vector][0] = handler;
 }

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#11 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 170 lines · Marc Zyngier
Quoted 152 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

The exception handling code is heavily inspired on kvm-unit-tests.

---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
tools/testing/selftests/kvm/Makefile +1 -1
index 4e548d7ab0ab..618c5903f478 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/processor.h +78 -0
index b7fa0c8551db..40aae31b4afc 100644
@@ -8,6 +8,7 @@
 #define SELFTEST_KVM_PROCESSOR_H
 
 #include "kvm_util.h"
+#include <linux/stringify.h>
 
 
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
@@ -18,6 +19,7 @@
 #define MAIR_EL1	3, 0, 10, 2, 0
 #define TTBR0_EL1	3, 0,  2, 0, 0
 #define SCTLR_EL1	3, 0,  1, 0, 0
+#define VBAR_EL1	3, 0, 12, 0, 0
 
 /*
  * Default MAIR
@@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
 void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
 			      struct kvm_vcpu_init *init, void *guest_code);
 
+struct ex_regs {
+	u64 regs[31];
+	u64 sp;
+	u64 pc;
+	u64 pstate;
+};
+
+#define VECTOR_NUM	16
+
+enum {
+	VECTOR_SYNC_CURRENT_SP0,
+	VECTOR_IRQ_CURRENT_SP0,
+	VECTOR_FIQ_CURRENT_SP0,
+	VECTOR_ERROR_CURRENT_SP0,
+
+	VECTOR_SYNC_CURRENT,
+	VECTOR_IRQ_CURRENT,
+	VECTOR_FIQ_CURRENT,
+	VECTOR_ERROR_CURRENT,
+
+	VECTOR_SYNC_LOWER_64,
+	VECTOR_IRQ_LOWER_64,
+	VECTOR_FIQ_LOWER_64,
+	VECTOR_ERROR_LOWER_64,
+
+	VECTOR_SYNC_LOWER_32,
+	VECTOR_IRQ_LOWER_32,
+	VECTOR_FIQ_LOWER_32,
+	VECTOR_ERROR_LOWER_32,
+};
+
+#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
+			   (v) == VECTOR_SYNC_CURRENT     || \
+			   (v) == VECTOR_SYNC_LOWER_64    || \
+			   (v) == VECTOR_SYNC_LOWER_32)
+
+/* Some common EC (Exception classes) */
+#define ESR_EC_ILLEGAL_INS	0x0e
+#define ESR_EC_SVC64		0x15
+#define ESR_EC_IABORT_CURRENT	0x21
+#define ESR_EC_DABORT_CURRENT	0x25
+#define ESR_EC_SERROR		0x2f
+#define ESR_EC_HW_BP_CURRENT	0x31
+#define ESR_EC_SSTEP_CURRENT	0x33
+#define ESR_EC_WP_CURRENT	0x35
+#define ESR_EC_BRK_INS		0x3C
+
+#define ESR_EC_NUM		64
+
+#define ESR_EC_SHIFT		26
+#define ESR_EC_MASK		(ESR_EC_NUM - 1)
+
+void vm_init_descriptor_tables(struct kvm_vm *vm);
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
+
+typedef void(*handler_fn)(struct ex_regs *);
+void vm_install_exception_handler(struct kvm_vm *vm,
+		int vector, int ec, handler_fn handler);
+void vm_install_vector_handler(struct kvm_vm *vm,
+		int vector, handler_fn handler);
+
+#define SPSR_D          (1 << 9)
+#define SPSR_SS         (1 << 21)
+
+#define write_sysreg(reg, val)						  \
+({									  \
+	u64 __val = (u64)(val);						  \
+	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
+})
+
+#define read_sysreg(reg)						  \
+({	u64 val;							  \
+	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
+	val;								  \
+})
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +12 -0
new file mode 100644
index 000000000000..8a560021892b
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+.macro save_registers, vector
+	add	sp, sp, #-16 * 17
+
+	stp	x0, x1, [sp, #16 * 0]
+	stp	x2, x3, [sp, #16 * 1]
+	stp	x4, x5, [sp, #16 * 2]
+	stp	x6, x7, [sp, #16 * 3]
+	stp	x8, x9, [sp, #16 * 4]
+	stp	x10, x11, [sp, #16 * 5]
+	stp	x12, x13, [sp, #16 * 6]
+	stp	x14, x15, [sp, #16 * 7]

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 22

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+	stp	x16, x17, [sp, #16 * 8]
+	stp	x18, x19, [sp, #16 * 9]
+	stp	x20, x21, [sp, #16 * 10]
+	stp	x22, x23, [sp, #16 * 11]
+	stp	x24, x25, [sp, #16 * 12]
+	stp	x26, x27, [sp, #16 * 13]
+	stp	x28, x29, [sp, #16 * 14]
+
+	.if \vector >= 8
+	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you always run in handler mode. Therefore, saving/restoring it is only overhead. If an exception handler wants to introspect it, it is already available in the relevant system register.

Quoted 2 lines

Or did you have something else in mind for it?

Not really. The reason for saving sp_el0 in there was just for consistency, so that handlers for both el0 and el1 exceptions could get the sp at regs->sp.

Restoring sp_el0 might be too much. So, what do you think of this v3: we keep the saving of sp_el0 into regs->sp (to keep things the same between el0 and el1) and delete the restoring of sp_el0?

Thanks,
Ricardo

Quoted 268 lines
Quoted 223 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +108 -0
+	.else
+	/*
+	 * This stores sp_el1 into ex_regs.sp so exception handlers can
+	 * "look" at it. It will _not_ be used to restore the sp_el1 on
+	 * return from the exception so handlers can not update it.
+	 */
+	mov	x1, sp
+	.endif
+	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
+
+	mrs	x1, elr_el1
+	mrs	x2, spsr_el1
+	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+.endm
+
+.macro restore_registers, vector
+	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+	msr	elr_el1, x1
+	msr	spsr_el1, x2
+
+	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
+	.if \vector >= 8
+	msr	sp_el0, x1
+	.endif
+
+	ldp	x28, x29, [sp, #16 * 14]
+	ldp	x26, x27, [sp, #16 * 13]
+	ldp	x24, x25, [sp, #16 * 12]
+	ldp	x22, x23, [sp, #16 * 11]
+	ldp	x20, x21, [sp, #16 * 10]
+	ldp	x18, x19, [sp, #16 * 9]
+	ldp	x16, x17, [sp, #16 * 8]
+	ldp	x14, x15, [sp, #16 * 7]
+	ldp	x12, x13, [sp, #16 * 6]
+	ldp	x10, x11, [sp, #16 * 5]
+	ldp	x8, x9, [sp, #16 * 4]
+	ldp	x6, x7, [sp, #16 * 3]
+	ldp	x4, x5, [sp, #16 * 2]
+	ldp	x2, x3, [sp, #16 * 1]
+	ldp	x0, x1, [sp, #16 * 0]
+
+	add	sp, sp, #16 * 17
+
+	eret
+.endm
+
+.pushsection ".entry.text", "ax"
+.balign 0x800
+.global vectors
+vectors:
+.popsection
+
+.set	vector, 0
+
+/*
+ * Build an exception handler for vector and append a jump to it into
+ * vectors (while making sure that it's 0x80 aligned).
+ */
+.macro HANDLER, label
+handler_\()\label:
+	save_registers vector
+	mov	x0, sp
+	mov	x1, #vector
+	bl	route_exception
+	restore_registers vector
+
+.pushsection ".entry.text", "ax"
+.balign 0x80
+	b	handler_\()\label
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+.macro HANDLER_INVALID
+.pushsection ".entry.text", "ax"
+.balign 0x80
+/* This will abort so no need to save and restore registers. */
+	mov	x0, #vector
+	b	kvm_exit_unexpected_vector
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+/*
+ * Caution: be sure to not add anything between the declaration of vectors
+ * above and these macro calls that will build the vectors table below it.
+ */
+	HANDLER_INVALID                         // Synchronous EL1t
+	HANDLER_INVALID                         // IRQ EL1t
+	HANDLER_INVALID                         // FIQ EL1t
+	HANDLER_INVALID                         // Error EL1t
+
+	HANDLER	el1h_sync                       // Synchronous EL1h
+	HANDLER	el1h_irq                        // IRQ EL1h
+	HANDLER el1h_fiq                        // FIQ EL1h
+	HANDLER	el1h_error                      // Error EL1h
+
+	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
+	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
+	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
+	HANDLER	el0_error_64                    // Error 64-bit EL0
+
+	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
+	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
+	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
+	HANDLER	el0_error_32                    // Error 32-bit EL0
tools/testing/selftests/kvm/lib/aarch64/processor.c +91 -0
index cee92d477dc0..25be71ec88be 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/compiler.h>
+#include <assert.h>
 
 #include "kvm_util.h"
 #include "../kvm_util_internal.h"
@@ -14,6 +15,8 @@
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
 #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
 
+vm_vaddr_t exception_handlers;
+
 static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
 {
 	return (v + vm->page_size) & ~(vm->page_size - 1);
@@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
 	va_end(ap);
 }
 
+void kvm_exit_unexpected_vector(int vector)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
+}
+
+void kvm_exit_unexpected_exception(int vector, uint64_t ec)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
+}
+
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
+	struct ucall uc;
+
+	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
+		return;
+
+	if (uc.args[2]) /* valid_ec */ {
+		assert(VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
+			uc.args[0], uc.args[1]);
+	} else {
+		assert(!VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx)",
+			uc.args[0]);
+	}
+}
+
+/*
+ * This exception handling code was heavily inspired on kvm-unit-tests. There
+ * is a set of default vector handlers stored in vector_handlers. These default
+ * vector handlers call user-installed handlers stored in exception_handlers.
+ * Synchronous handlers are indexed by (vector, ec), and irq handlers by
+ * (vector, ec=0).
+ */
+
+typedef void(*vector_fn)(struct ex_regs *, int vector);
+
+struct handlers {
+	vector_fn vector_handlers[VECTOR_NUM];
+	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
+};
+
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
+{
+	extern char vectors;
+
+	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
+}
+
+void default_sync_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+	uint64_t esr = read_sysreg(esr_el1);
+	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
+
+	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][ec])
+		handlers->exception_handlers[vector][ec](regs);
+	else
+		kvm_exit_unexpected_exception(vector, ec);
+}
+
+void default_irq_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][0])
+		handlers->exception_handlers[vector][0](regs);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void route_exception(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	if (handlers && handlers->vector_handlers[vector])
+		handlers->vector_handlers[vector](regs, vector);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void vm_init_descriptor_tables(struct kvm_vm *vm)
+{

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 439

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -0
+	struct handlers *handlers;
+
+	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
+			vm->page_size, 0, 0);
+
+	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
+	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;

How about FIQ, Error? Although they are unlikely, they are valid exceptions.

Quoted 14 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +14 -0
+
+	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
+}
+
+void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	assert(ec < ESR_EC_NUM);
+	handlers->exception_handlers[vector][ec] = handler;
+}

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 463

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -0
+
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(!VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	handlers->exception_handlers[vector][0] = handler;
 }

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#12 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Hi Ricardo,

Quoted 12 lines · Ricardo Koller
Quoted 11 lines · Marc Zyngier
Quoted 9 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The exception handling code is heavily inspired on kvm-unit-tests.

running the test on 5.12 I get

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794

I guess it is not an expected result. Any known bug waiting on the list?

Thanks

Eric

Quoted 441 lines
Quoted 158 lines
Quoted 142 lines
---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
tools/testing/selftests/kvm/Makefile +1 -1
index 4e548d7ab0ab..618c5903f478 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/processor.h +78 -0
index b7fa0c8551db..40aae31b4afc 100644
@@ -8,6 +8,7 @@
 #define SELFTEST_KVM_PROCESSOR_H
 
 #include "kvm_util.h"
+#include <linux/stringify.h>
 
 
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
@@ -18,6 +19,7 @@
 #define MAIR_EL1	3, 0, 10, 2, 0
 #define TTBR0_EL1	3, 0,  2, 0, 0
 #define SCTLR_EL1	3, 0,  1, 0, 0
+#define VBAR_EL1	3, 0, 12, 0, 0
 
 /*
  * Default MAIR
@@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
 void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
 			      struct kvm_vcpu_init *init, void *guest_code);
 
+struct ex_regs {
+	u64 regs[31];
+	u64 sp;
+	u64 pc;
+	u64 pstate;
+};
+
+#define VECTOR_NUM	16
+
+enum {
+	VECTOR_SYNC_CURRENT_SP0,
+	VECTOR_IRQ_CURRENT_SP0,
+	VECTOR_FIQ_CURRENT_SP0,
+	VECTOR_ERROR_CURRENT_SP0,
+
+	VECTOR_SYNC_CURRENT,
+	VECTOR_IRQ_CURRENT,
+	VECTOR_FIQ_CURRENT,
+	VECTOR_ERROR_CURRENT,
+
+	VECTOR_SYNC_LOWER_64,
+	VECTOR_IRQ_LOWER_64,
+	VECTOR_FIQ_LOWER_64,
+	VECTOR_ERROR_LOWER_64,
+
+	VECTOR_SYNC_LOWER_32,
+	VECTOR_IRQ_LOWER_32,
+	VECTOR_FIQ_LOWER_32,
+	VECTOR_ERROR_LOWER_32,
+};
+
+#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
+			   (v) == VECTOR_SYNC_CURRENT     || \
+			   (v) == VECTOR_SYNC_LOWER_64    || \
+			   (v) == VECTOR_SYNC_LOWER_32)
+
+/* Some common EC (Exception classes) */
+#define ESR_EC_ILLEGAL_INS	0x0e
+#define ESR_EC_SVC64		0x15
+#define ESR_EC_IABORT_CURRENT	0x21
+#define ESR_EC_DABORT_CURRENT	0x25
+#define ESR_EC_SERROR		0x2f
+#define ESR_EC_HW_BP_CURRENT	0x31
+#define ESR_EC_SSTEP_CURRENT	0x33
+#define ESR_EC_WP_CURRENT	0x35
+#define ESR_EC_BRK_INS		0x3C
+
+#define ESR_EC_NUM		64
+
+#define ESR_EC_SHIFT		26
+#define ESR_EC_MASK		(ESR_EC_NUM - 1)
+
+void vm_init_descriptor_tables(struct kvm_vm *vm);
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
+
+typedef void(*handler_fn)(struct ex_regs *);
+void vm_install_exception_handler(struct kvm_vm *vm,
+		int vector, int ec, handler_fn handler);
+void vm_install_vector_handler(struct kvm_vm *vm,
+		int vector, handler_fn handler);
+
+#define SPSR_D          (1 << 9)
+#define SPSR_SS         (1 << 21)
+
+#define write_sysreg(reg, val)						  \
+({									  \
+	u64 __val = (u64)(val);						  \
+	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
+})
+
+#define read_sysreg(reg)						  \
+({	u64 val;							  \
+	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
+	val;								  \
+})
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +12 -0
new file mode 100644
index 000000000000..8a560021892b
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+.macro save_registers, vector
+	add	sp, sp, #-16 * 17
+
+	stp	x0, x1, [sp, #16 * 0]
+	stp	x2, x3, [sp, #16 * 1]
+	stp	x4, x5, [sp, #16 * 2]
+	stp	x6, x7, [sp, #16 * 3]
+	stp	x8, x9, [sp, #16 * 4]
+	stp	x10, x11, [sp, #16 * 5]
+	stp	x12, x13, [sp, #16 * 6]
+	stp	x14, x15, [sp, #16 * 7]

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 22

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+	stp	x16, x17, [sp, #16 * 8]
+	stp	x18, x19, [sp, #16 * 9]
+	stp	x20, x21, [sp, #16 * 10]
+	stp	x22, x23, [sp, #16 * 11]
+	stp	x24, x25, [sp, #16 * 12]
+	stp	x26, x27, [sp, #16 * 13]
+	stp	x28, x29, [sp, #16 * 14]
+
+	.if \vector >= 8
+	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you always run in handler mode. Therefore, saving/restoring it is only overhead. If an exception handler wants to introspect it, it is already available in the relevant system register.

Quoted 2 lines

Or did you have something else in mind for it?

Not really. The reason for saving sp_el0 in there was just for consistency, so that handlers for both el0 and el1 exceptions could get the sp at regs->sp.

Restoring sp_el0 might be too much. So, what do you think of this v3: we keep the saving of sp_el0 into regs->sp (to keep things the same between el0 and el1) and delete the restoring of sp_el0?

Thanks,
Ricardo

Quoted 268 lines
Quoted 223 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +108 -0
+	.else
+	/*
+	 * This stores sp_el1 into ex_regs.sp so exception handlers can
+	 * "look" at it. It will _not_ be used to restore the sp_el1 on
+	 * return from the exception so handlers can not update it.
+	 */
+	mov	x1, sp
+	.endif
+	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
+
+	mrs	x1, elr_el1
+	mrs	x2, spsr_el1
+	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+.endm
+
+.macro restore_registers, vector
+	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+	msr	elr_el1, x1
+	msr	spsr_el1, x2
+
+	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
+	.if \vector >= 8
+	msr	sp_el0, x1
+	.endif
+
+	ldp	x28, x29, [sp, #16 * 14]
+	ldp	x26, x27, [sp, #16 * 13]
+	ldp	x24, x25, [sp, #16 * 12]
+	ldp	x22, x23, [sp, #16 * 11]
+	ldp	x20, x21, [sp, #16 * 10]
+	ldp	x18, x19, [sp, #16 * 9]
+	ldp	x16, x17, [sp, #16 * 8]
+	ldp	x14, x15, [sp, #16 * 7]
+	ldp	x12, x13, [sp, #16 * 6]
+	ldp	x10, x11, [sp, #16 * 5]
+	ldp	x8, x9, [sp, #16 * 4]
+	ldp	x6, x7, [sp, #16 * 3]
+	ldp	x4, x5, [sp, #16 * 2]
+	ldp	x2, x3, [sp, #16 * 1]
+	ldp	x0, x1, [sp, #16 * 0]
+
+	add	sp, sp, #16 * 17
+
+	eret
+.endm
+
+.pushsection ".entry.text", "ax"
+.balign 0x800
+.global vectors
+vectors:
+.popsection
+
+.set	vector, 0
+
+/*
+ * Build an exception handler for vector and append a jump to it into
+ * vectors (while making sure that it's 0x80 aligned).
+ */
+.macro HANDLER, label
+handler_\()\label:
+	save_registers vector
+	mov	x0, sp
+	mov	x1, #vector
+	bl	route_exception
+	restore_registers vector
+
+.pushsection ".entry.text", "ax"
+.balign 0x80
+	b	handler_\()\label
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+.macro HANDLER_INVALID
+.pushsection ".entry.text", "ax"
+.balign 0x80
+/* This will abort so no need to save and restore registers. */
+	mov	x0, #vector
+	b	kvm_exit_unexpected_vector
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+/*
+ * Caution: be sure to not add anything between the declaration of vectors
+ * above and these macro calls that will build the vectors table below it.
+ */
+	HANDLER_INVALID                         // Synchronous EL1t
+	HANDLER_INVALID                         // IRQ EL1t
+	HANDLER_INVALID                         // FIQ EL1t
+	HANDLER_INVALID                         // Error EL1t
+
+	HANDLER	el1h_sync                       // Synchronous EL1h
+	HANDLER	el1h_irq                        // IRQ EL1h
+	HANDLER el1h_fiq                        // FIQ EL1h
+	HANDLER	el1h_error                      // Error EL1h
+
+	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
+	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
+	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
+	HANDLER	el0_error_64                    // Error 64-bit EL0
+
+	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
+	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
+	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
+	HANDLER	el0_error_32                    // Error 32-bit EL0
tools/testing/selftests/kvm/lib/aarch64/processor.c +91 -0
index cee92d477dc0..25be71ec88be 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/compiler.h>
+#include <assert.h>
 
 #include "kvm_util.h"
 #include "../kvm_util_internal.h"
@@ -14,6 +15,8 @@
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
 #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
 
+vm_vaddr_t exception_handlers;
+
 static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
 {
 	return (v + vm->page_size) & ~(vm->page_size - 1);
@@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
 	va_end(ap);
 }
 
+void kvm_exit_unexpected_vector(int vector)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
+}
+
+void kvm_exit_unexpected_exception(int vector, uint64_t ec)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
+}
+
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
+	struct ucall uc;
+
+	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
+		return;
+
+	if (uc.args[2]) /* valid_ec */ {
+		assert(VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
+			uc.args[0], uc.args[1]);
+	} else {
+		assert(!VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx)",
+			uc.args[0]);
+	}
+}
+
+/*
+ * This exception handling code was heavily inspired on kvm-unit-tests. There
+ * is a set of default vector handlers stored in vector_handlers. These default
+ * vector handlers call user-installed handlers stored in exception_handlers.
+ * Synchronous handlers are indexed by (vector, ec), and irq handlers by
+ * (vector, ec=0).
+ */
+
+typedef void(*vector_fn)(struct ex_regs *, int vector);
+
+struct handlers {
+	vector_fn vector_handlers[VECTOR_NUM];
+	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
+};
+
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
+{
+	extern char vectors;
+
+	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
+}
+
+void default_sync_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+	uint64_t esr = read_sysreg(esr_el1);
+	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
+
+	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][ec])
+		handlers->exception_handlers[vector][ec](regs);
+	else
+		kvm_exit_unexpected_exception(vector, ec);
+}
+
+void default_irq_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][0])
+		handlers->exception_handlers[vector][0](regs);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void route_exception(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	if (handlers && handlers->vector_handlers[vector])
+		handlers->vector_handlers[vector](regs, vector);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void vm_init_descriptor_tables(struct kvm_vm *vm)
+{

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 439

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -0
+	struct handlers *handlers;
+
+	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
+			vm->page_size, 0, 0);
+
+	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
+	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;

How about FIQ, Error? Although they are unlikely, they are valid exceptions.

Quoted 14 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +14 -0
+
+	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
+}
+
+void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	assert(ec < ESR_EC_NUM);
+	handlers->exception_handlers[vector][ec] = handler;
+}

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 463

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -0
+
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(!VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	handlers->exception_handlers[vector][0] = handler;
 }

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#13 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 2 lines · Auger Eric

Hi Ricardo,

Hi Eric,

Thank you very much for the test.

Quoted 28 lines · Ricardo Koller
Quoted 12 lines · Marc Zyngier
Quoted 11 lines
Quoted 9 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The exception handling code is heavily inspired on kvm-unit-tests.

running the test on 5.12 I get

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794
Quoted 2 lines

I guess it is not an expected result. Any known bug waiting on the list?

Not expected. That should work, or at least abort early because there is no HW breakpoints support.

I'm trying to reproduce the failure; can you help me with some questions, please?

- does your setup have support for hardware breakpoints? Can you try a 'dmesg | grep break'? I'm looking for something like 'hw-breakpoint:
found ...'. If there is no such line it's very likely that the check for "debug_ver >= 6" is not enough and the test should check for "num_breakpoints > 0".
- does it fail consistently (every single attempt)?

Thanks!
Ricardo

Quoted 448 lines

Thanks

Eric

Quoted 441 lines
Quoted 158 lines
Quoted 142 lines
---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
tools/testing/selftests/kvm/Makefile +1 -1
index 4e548d7ab0ab..618c5903f478 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/processor.h +78 -0
index b7fa0c8551db..40aae31b4afc 100644
@@ -8,6 +8,7 @@
 #define SELFTEST_KVM_PROCESSOR_H
 
 #include "kvm_util.h"
+#include <linux/stringify.h>
 
 
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
@@ -18,6 +19,7 @@
 #define MAIR_EL1	3, 0, 10, 2, 0
 #define TTBR0_EL1	3, 0,  2, 0, 0
 #define SCTLR_EL1	3, 0,  1, 0, 0
+#define VBAR_EL1	3, 0, 12, 0, 0
 
 /*
  * Default MAIR
@@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
 void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
 			      struct kvm_vcpu_init *init, void *guest_code);
 
+struct ex_regs {
+	u64 regs[31];
+	u64 sp;
+	u64 pc;
+	u64 pstate;
+};
+
+#define VECTOR_NUM	16
+
+enum {
+	VECTOR_SYNC_CURRENT_SP0,
+	VECTOR_IRQ_CURRENT_SP0,
+	VECTOR_FIQ_CURRENT_SP0,
+	VECTOR_ERROR_CURRENT_SP0,
+
+	VECTOR_SYNC_CURRENT,
+	VECTOR_IRQ_CURRENT,
+	VECTOR_FIQ_CURRENT,
+	VECTOR_ERROR_CURRENT,
+
+	VECTOR_SYNC_LOWER_64,
+	VECTOR_IRQ_LOWER_64,
+	VECTOR_FIQ_LOWER_64,
+	VECTOR_ERROR_LOWER_64,
+
+	VECTOR_SYNC_LOWER_32,
+	VECTOR_IRQ_LOWER_32,
+	VECTOR_FIQ_LOWER_32,
+	VECTOR_ERROR_LOWER_32,
+};
+
+#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
+			   (v) == VECTOR_SYNC_CURRENT     || \
+			   (v) == VECTOR_SYNC_LOWER_64    || \
+			   (v) == VECTOR_SYNC_LOWER_32)
+
+/* Some common EC (Exception classes) */
+#define ESR_EC_ILLEGAL_INS	0x0e
+#define ESR_EC_SVC64		0x15
+#define ESR_EC_IABORT_CURRENT	0x21
+#define ESR_EC_DABORT_CURRENT	0x25
+#define ESR_EC_SERROR		0x2f
+#define ESR_EC_HW_BP_CURRENT	0x31
+#define ESR_EC_SSTEP_CURRENT	0x33
+#define ESR_EC_WP_CURRENT	0x35
+#define ESR_EC_BRK_INS		0x3C
+
+#define ESR_EC_NUM		64
+
+#define ESR_EC_SHIFT		26
+#define ESR_EC_MASK		(ESR_EC_NUM - 1)
+
+void vm_init_descriptor_tables(struct kvm_vm *vm);
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
+
+typedef void(*handler_fn)(struct ex_regs *);
+void vm_install_exception_handler(struct kvm_vm *vm,
+		int vector, int ec, handler_fn handler);
+void vm_install_vector_handler(struct kvm_vm *vm,
+		int vector, handler_fn handler);
+
+#define SPSR_D          (1 << 9)
+#define SPSR_SS         (1 << 21)
+
+#define write_sysreg(reg, val)						  \
+({									  \
+	u64 __val = (u64)(val);						  \
+	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
+})
+
+#define read_sysreg(reg)						  \
+({	u64 val;							  \
+	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
+	val;								  \
+})
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +12 -0
new file mode 100644
index 000000000000..8a560021892b
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+.macro save_registers, vector
+	add	sp, sp, #-16 * 17
+
+	stp	x0, x1, [sp, #16 * 0]
+	stp	x2, x3, [sp, #16 * 1]
+	stp	x4, x5, [sp, #16 * 2]
+	stp	x6, x7, [sp, #16 * 3]
+	stp	x8, x9, [sp, #16 * 4]
+	stp	x10, x11, [sp, #16 * 5]
+	stp	x12, x13, [sp, #16 * 6]
+	stp	x14, x15, [sp, #16 * 7]

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 22

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+	stp	x16, x17, [sp, #16 * 8]
+	stp	x18, x19, [sp, #16 * 9]
+	stp	x20, x21, [sp, #16 * 10]
+	stp	x22, x23, [sp, #16 * 11]
+	stp	x24, x25, [sp, #16 * 12]
+	stp	x26, x27, [sp, #16 * 13]
+	stp	x28, x29, [sp, #16 * 14]
+
+	.if \vector >= 8
+	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you always run in handler mode. Therefore, saving/restoring it is only overhead. If an exception handler wants to introspect it, it is already available in the relevant system register.

Quoted 2 lines

Or did you have something else in mind for it?

Not really. The reason for saving sp_el0 in there was just for consistency, so that handlers for both el0 and el1 exceptions could get the sp at regs->sp.

Restoring sp_el0 might be too much. So, what do you think of this v3: we keep the saving of sp_el0 into regs->sp (to keep things the same between el0 and el1) and delete the restoring of sp_el0?

Thanks,
Ricardo

Quoted 268 lines
Quoted 223 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +108 -0
+	.else
+	/*
+	 * This stores sp_el1 into ex_regs.sp so exception handlers can
+	 * "look" at it. It will _not_ be used to restore the sp_el1 on
+	 * return from the exception so handlers can not update it.
+	 */
+	mov	x1, sp
+	.endif
+	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
+
+	mrs	x1, elr_el1
+	mrs	x2, spsr_el1
+	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+.endm
+
+.macro restore_registers, vector
+	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+	msr	elr_el1, x1
+	msr	spsr_el1, x2
+
+	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
+	.if \vector >= 8
+	msr	sp_el0, x1
+	.endif
+
+	ldp	x28, x29, [sp, #16 * 14]
+	ldp	x26, x27, [sp, #16 * 13]
+	ldp	x24, x25, [sp, #16 * 12]
+	ldp	x22, x23, [sp, #16 * 11]
+	ldp	x20, x21, [sp, #16 * 10]
+	ldp	x18, x19, [sp, #16 * 9]
+	ldp	x16, x17, [sp, #16 * 8]
+	ldp	x14, x15, [sp, #16 * 7]
+	ldp	x12, x13, [sp, #16 * 6]
+	ldp	x10, x11, [sp, #16 * 5]
+	ldp	x8, x9, [sp, #16 * 4]
+	ldp	x6, x7, [sp, #16 * 3]
+	ldp	x4, x5, [sp, #16 * 2]
+	ldp	x2, x3, [sp, #16 * 1]
+	ldp	x0, x1, [sp, #16 * 0]
+
+	add	sp, sp, #16 * 17
+
+	eret
+.endm
+
+.pushsection ".entry.text", "ax"
+.balign 0x800
+.global vectors
+vectors:
+.popsection
+
+.set	vector, 0
+
+/*
+ * Build an exception handler for vector and append a jump to it into
+ * vectors (while making sure that it's 0x80 aligned).
+ */
+.macro HANDLER, label
+handler_\()\label:
+	save_registers vector
+	mov	x0, sp
+	mov	x1, #vector
+	bl	route_exception
+	restore_registers vector
+
+.pushsection ".entry.text", "ax"
+.balign 0x80
+	b	handler_\()\label
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+.macro HANDLER_INVALID
+.pushsection ".entry.text", "ax"
+.balign 0x80
+/* This will abort so no need to save and restore registers. */
+	mov	x0, #vector
+	b	kvm_exit_unexpected_vector
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+/*
+ * Caution: be sure to not add anything between the declaration of vectors
+ * above and these macro calls that will build the vectors table below it.
+ */
+	HANDLER_INVALID                         // Synchronous EL1t
+	HANDLER_INVALID                         // IRQ EL1t
+	HANDLER_INVALID                         // FIQ EL1t
+	HANDLER_INVALID                         // Error EL1t
+
+	HANDLER	el1h_sync                       // Synchronous EL1h
+	HANDLER	el1h_irq                        // IRQ EL1h
+	HANDLER el1h_fiq                        // FIQ EL1h
+	HANDLER	el1h_error                      // Error EL1h
+
+	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
+	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
+	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
+	HANDLER	el0_error_64                    // Error 64-bit EL0
+
+	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
+	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
+	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
+	HANDLER	el0_error_32                    // Error 32-bit EL0
tools/testing/selftests/kvm/lib/aarch64/processor.c +91 -0
index cee92d477dc0..25be71ec88be 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/compiler.h>
+#include <assert.h>
 
 #include "kvm_util.h"
 #include "../kvm_util_internal.h"
@@ -14,6 +15,8 @@
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
 #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
 
+vm_vaddr_t exception_handlers;
+
 static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
 {
 	return (v + vm->page_size) & ~(vm->page_size - 1);
@@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
 	va_end(ap);
 }
 
+void kvm_exit_unexpected_vector(int vector)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
+}
+
+void kvm_exit_unexpected_exception(int vector, uint64_t ec)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
+}
+
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
+	struct ucall uc;
+
+	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
+		return;
+
+	if (uc.args[2]) /* valid_ec */ {
+		assert(VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
+			uc.args[0], uc.args[1]);
+	} else {
+		assert(!VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx)",
+			uc.args[0]);
+	}
+}
+
+/*
+ * This exception handling code was heavily inspired on kvm-unit-tests. There
+ * is a set of default vector handlers stored in vector_handlers. These default
+ * vector handlers call user-installed handlers stored in exception_handlers.
+ * Synchronous handlers are indexed by (vector, ec), and irq handlers by
+ * (vector, ec=0).
+ */
+
+typedef void(*vector_fn)(struct ex_regs *, int vector);
+
+struct handlers {
+	vector_fn vector_handlers[VECTOR_NUM];
+	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
+};
+
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
+{
+	extern char vectors;
+
+	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
+}
+
+void default_sync_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+	uint64_t esr = read_sysreg(esr_el1);
+	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
+
+	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][ec])
+		handlers->exception_handlers[vector][ec](regs);
+	else
+		kvm_exit_unexpected_exception(vector, ec);
+}
+
+void default_irq_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][0])
+		handlers->exception_handlers[vector][0](regs);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void route_exception(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	if (handlers && handlers->vector_handlers[vector])
+		handlers->vector_handlers[vector](regs, vector);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void vm_init_descriptor_tables(struct kvm_vm *vm)
+{

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 439

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -0
+	struct handlers *handlers;
+
+	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
+			vm->page_size, 0, 0);
+
+	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
+	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;

How about FIQ, Error? Although they are unlikely, they are valid exceptions.

Quoted 14 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +14 -0
+
+	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
+}
+
+void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	assert(ec < ESR_EC_NUM);
+	handlers->exception_handlers[vector][ec] = handler;
+}

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 463

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -0
+
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(!VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	handlers->exception_handlers[vector][0] = handler;
 }

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#14 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Hi Ricardo,

Quoted 36 lines · Ricardo Koller
Quoted 2 lines · Auger Eric

Hi Ricardo,

Hi Eric,

Thank you very much for the test.

Quoted 28 lines · Ricardo Koller
Quoted 12 lines · Marc Zyngier
Quoted 11 lines
Quoted 9 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The exception handling code is heavily inspired on kvm-unit-tests.

running the test on 5.12 I get

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794
Quoted 8 lines
Quoted 2 lines

I guess it is not an expected result. Any known bug waiting on the list?

Not expected. That should work, or at least abort early because there is no HW breakpoints support.

I'm trying to reproduce the failure; can you help me with some questions, please?

sure, please find the answers below.

Quoted 6 lines

- does your setup have support for hardware breakpoints? Can you try a 'dmesg | grep break'? I'm looking for something like 'hw-breakpoint:
found ...'. If there is no such line it's very likely that the check for "debug_ver >= 6" is not enough and the test should check for "num_breakpoints > 0".

[   25.640418] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
Quoted 1 line

- does it fail consistently (every single attempt)?

yes it does.

I will try to find some time to investigate too

Thanks

Eric

Quoted 453 lines

Thanks!
Ricardo

Quoted 448 lines

Thanks

Eric

Quoted 441 lines
Quoted 158 lines
Quoted 142 lines
---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
tools/testing/selftests/kvm/Makefile +1 -1
index 4e548d7ab0ab..618c5903f478 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/processor.h +78 -0
index b7fa0c8551db..40aae31b4afc 100644
@@ -8,6 +8,7 @@
 #define SELFTEST_KVM_PROCESSOR_H
 
 #include "kvm_util.h"
+#include <linux/stringify.h>
 
 
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
@@ -18,6 +19,7 @@
 #define MAIR_EL1	3, 0, 10, 2, 0
 #define TTBR0_EL1	3, 0,  2, 0, 0
 #define SCTLR_EL1	3, 0,  1, 0, 0
+#define VBAR_EL1	3, 0, 12, 0, 0
 
 /*
  * Default MAIR
@@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
 void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
 			      struct kvm_vcpu_init *init, void *guest_code);
 
+struct ex_regs {
+	u64 regs[31];
+	u64 sp;
+	u64 pc;
+	u64 pstate;
+};
+
+#define VECTOR_NUM	16
+
+enum {
+	VECTOR_SYNC_CURRENT_SP0,
+	VECTOR_IRQ_CURRENT_SP0,
+	VECTOR_FIQ_CURRENT_SP0,
+	VECTOR_ERROR_CURRENT_SP0,
+
+	VECTOR_SYNC_CURRENT,
+	VECTOR_IRQ_CURRENT,
+	VECTOR_FIQ_CURRENT,
+	VECTOR_ERROR_CURRENT,
+
+	VECTOR_SYNC_LOWER_64,
+	VECTOR_IRQ_LOWER_64,
+	VECTOR_FIQ_LOWER_64,
+	VECTOR_ERROR_LOWER_64,
+
+	VECTOR_SYNC_LOWER_32,
+	VECTOR_IRQ_LOWER_32,
+	VECTOR_FIQ_LOWER_32,
+	VECTOR_ERROR_LOWER_32,
+};
+
+#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
+			   (v) == VECTOR_SYNC_CURRENT     || \
+			   (v) == VECTOR_SYNC_LOWER_64    || \
+			   (v) == VECTOR_SYNC_LOWER_32)
+
+/* Some common EC (Exception classes) */
+#define ESR_EC_ILLEGAL_INS	0x0e
+#define ESR_EC_SVC64		0x15
+#define ESR_EC_IABORT_CURRENT	0x21
+#define ESR_EC_DABORT_CURRENT	0x25
+#define ESR_EC_SERROR		0x2f
+#define ESR_EC_HW_BP_CURRENT	0x31
+#define ESR_EC_SSTEP_CURRENT	0x33
+#define ESR_EC_WP_CURRENT	0x35
+#define ESR_EC_BRK_INS		0x3C
+
+#define ESR_EC_NUM		64
+
+#define ESR_EC_SHIFT		26
+#define ESR_EC_MASK		(ESR_EC_NUM - 1)
+
+void vm_init_descriptor_tables(struct kvm_vm *vm);
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
+
+typedef void(*handler_fn)(struct ex_regs *);
+void vm_install_exception_handler(struct kvm_vm *vm,
+		int vector, int ec, handler_fn handler);
+void vm_install_vector_handler(struct kvm_vm *vm,
+		int vector, handler_fn handler);
+
+#define SPSR_D          (1 << 9)
+#define SPSR_SS         (1 << 21)
+
+#define write_sysreg(reg, val)						  \
+({									  \
+	u64 __val = (u64)(val);						  \
+	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
+})
+
+#define read_sysreg(reg)						  \
+({	u64 val;							  \
+	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
+	val;								  \
+})
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +12 -0
new file mode 100644
index 000000000000..8a560021892b
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+.macro save_registers, vector
+	add	sp, sp, #-16 * 17
+
+	stp	x0, x1, [sp, #16 * 0]
+	stp	x2, x3, [sp, #16 * 1]
+	stp	x4, x5, [sp, #16 * 2]
+	stp	x6, x7, [sp, #16 * 3]
+	stp	x8, x9, [sp, #16 * 4]
+	stp	x10, x11, [sp, #16 * 5]
+	stp	x12, x13, [sp, #16 * 6]
+	stp	x14, x15, [sp, #16 * 7]

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 22

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+	stp	x16, x17, [sp, #16 * 8]
+	stp	x18, x19, [sp, #16 * 9]
+	stp	x20, x21, [sp, #16 * 10]
+	stp	x22, x23, [sp, #16 * 11]
+	stp	x24, x25, [sp, #16 * 12]
+	stp	x26, x27, [sp, #16 * 13]
+	stp	x28, x29, [sp, #16 * 14]
+
+	.if \vector >= 8
+	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you always run in handler mode. Therefore, saving/restoring it is only overhead. If an exception handler wants to introspect it, it is already available in the relevant system register.

Quoted 2 lines

Or did you have something else in mind for it?

Not really. The reason for saving sp_el0 in there was just for consistency, so that handlers for both el0 and el1 exceptions could get the sp at regs->sp.

Restoring sp_el0 might be too much. So, what do you think of this v3: we keep the saving of sp_el0 into regs->sp (to keep things the same between el0 and el1) and delete the restoring of sp_el0?

Thanks,
Ricardo

Quoted 268 lines
Quoted 223 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +108 -0
+	.else
+	/*
+	 * This stores sp_el1 into ex_regs.sp so exception handlers can
+	 * "look" at it. It will _not_ be used to restore the sp_el1 on
+	 * return from the exception so handlers can not update it.
+	 */
+	mov	x1, sp
+	.endif
+	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
+
+	mrs	x1, elr_el1
+	mrs	x2, spsr_el1
+	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+.endm
+
+.macro restore_registers, vector
+	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+	msr	elr_el1, x1
+	msr	spsr_el1, x2
+
+	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
+	.if \vector >= 8
+	msr	sp_el0, x1
+	.endif
+
+	ldp	x28, x29, [sp, #16 * 14]
+	ldp	x26, x27, [sp, #16 * 13]
+	ldp	x24, x25, [sp, #16 * 12]
+	ldp	x22, x23, [sp, #16 * 11]
+	ldp	x20, x21, [sp, #16 * 10]
+	ldp	x18, x19, [sp, #16 * 9]
+	ldp	x16, x17, [sp, #16 * 8]
+	ldp	x14, x15, [sp, #16 * 7]
+	ldp	x12, x13, [sp, #16 * 6]
+	ldp	x10, x11, [sp, #16 * 5]
+	ldp	x8, x9, [sp, #16 * 4]
+	ldp	x6, x7, [sp, #16 * 3]
+	ldp	x4, x5, [sp, #16 * 2]
+	ldp	x2, x3, [sp, #16 * 1]
+	ldp	x0, x1, [sp, #16 * 0]
+
+	add	sp, sp, #16 * 17
+
+	eret
+.endm
+
+.pushsection ".entry.text", "ax"
+.balign 0x800
+.global vectors
+vectors:
+.popsection
+
+.set	vector, 0
+
+/*
+ * Build an exception handler for vector and append a jump to it into
+ * vectors (while making sure that it's 0x80 aligned).
+ */
+.macro HANDLER, label
+handler_\()\label:
+	save_registers vector
+	mov	x0, sp
+	mov	x1, #vector
+	bl	route_exception
+	restore_registers vector
+
+.pushsection ".entry.text", "ax"
+.balign 0x80
+	b	handler_\()\label
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+.macro HANDLER_INVALID
+.pushsection ".entry.text", "ax"
+.balign 0x80
+/* This will abort so no need to save and restore registers. */
+	mov	x0, #vector
+	b	kvm_exit_unexpected_vector
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+/*
+ * Caution: be sure to not add anything between the declaration of vectors
+ * above and these macro calls that will build the vectors table below it.
+ */
+	HANDLER_INVALID                         // Synchronous EL1t
+	HANDLER_INVALID                         // IRQ EL1t
+	HANDLER_INVALID                         // FIQ EL1t
+	HANDLER_INVALID                         // Error EL1t
+
+	HANDLER	el1h_sync                       // Synchronous EL1h
+	HANDLER	el1h_irq                        // IRQ EL1h
+	HANDLER el1h_fiq                        // FIQ EL1h
+	HANDLER	el1h_error                      // Error EL1h
+
+	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
+	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
+	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
+	HANDLER	el0_error_64                    // Error 64-bit EL0
+
+	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
+	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
+	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
+	HANDLER	el0_error_32                    // Error 32-bit EL0
tools/testing/selftests/kvm/lib/aarch64/processor.c +91 -0
index cee92d477dc0..25be71ec88be 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/compiler.h>
+#include <assert.h>
 
 #include "kvm_util.h"
 #include "../kvm_util_internal.h"
@@ -14,6 +15,8 @@
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
 #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
 
+vm_vaddr_t exception_handlers;
+
 static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
 {
 	return (v + vm->page_size) & ~(vm->page_size - 1);
@@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
 	va_end(ap);
 }
 
+void kvm_exit_unexpected_vector(int vector)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
+}
+
+void kvm_exit_unexpected_exception(int vector, uint64_t ec)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
+}
+
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
+	struct ucall uc;
+
+	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
+		return;
+
+	if (uc.args[2]) /* valid_ec */ {
+		assert(VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
+			uc.args[0], uc.args[1]);
+	} else {
+		assert(!VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx)",
+			uc.args[0]);
+	}
+}
+
+/*
+ * This exception handling code was heavily inspired on kvm-unit-tests. There
+ * is a set of default vector handlers stored in vector_handlers. These default
+ * vector handlers call user-installed handlers stored in exception_handlers.
+ * Synchronous handlers are indexed by (vector, ec), and irq handlers by
+ * (vector, ec=0).
+ */
+
+typedef void(*vector_fn)(struct ex_regs *, int vector);
+
+struct handlers {
+	vector_fn vector_handlers[VECTOR_NUM];
+	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
+};
+
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
+{
+	extern char vectors;
+
+	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
+}
+
+void default_sync_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+	uint64_t esr = read_sysreg(esr_el1);
+	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
+
+	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][ec])
+		handlers->exception_handlers[vector][ec](regs);
+	else
+		kvm_exit_unexpected_exception(vector, ec);
+}
+
+void default_irq_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][0])
+		handlers->exception_handlers[vector][0](regs);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void route_exception(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	if (handlers && handlers->vector_handlers[vector])
+		handlers->vector_handlers[vector](regs, vector);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void vm_init_descriptor_tables(struct kvm_vm *vm)
+{

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 439

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -0
+	struct handlers *handlers;
+
+	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
+			vm->page_size, 0, 0);
+
+	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
+	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;

How about FIQ, Error? Although they are unlikely, they are valid exceptions.

Quoted 14 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +14 -0
+
+	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
+}
+
+void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	assert(ec < ESR_EC_NUM);
+	handlers->exception_handlers[vector][ec] = handler;
+}

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 463

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -0
+
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(!VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	handlers->exception_handlers[vector][0] = handler;
 }

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#15 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 54 lines · Auger Eric

Hi Ricardo,

Quoted 36 lines · Ricardo Koller
Quoted 2 lines · Auger Eric

Hi Ricardo,

Hi Eric,

Thank you very much for the test.

Quoted 28 lines · Ricardo Koller
Quoted 12 lines · Marc Zyngier
Quoted 11 lines
Quoted 9 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The exception handling code is heavily inspired on kvm-unit-tests.

running the test on 5.12 I get

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794
Quoted 8 lines
Quoted 2 lines

I guess it is not an expected result. Any known bug waiting on the list?

Not expected. That should work, or at least abort early because there is no HW breakpoints support.

I'm trying to reproduce the failure; can you help me with some questions, please?

sure, please find the answers below.

Quoted 6 lines

- does your setup have support for hardware breakpoints? Can you try a 'dmesg | grep break'? I'm looking for something like 'hw-breakpoint:
found ...'. If there is no such line it's very likely that the check for "debug_ver >= 6" is not enough and the test should check for "num_breakpoints > 0".

Quoted 5 lines
[   25.640418] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
Quoted 1 line

- does it fail consistently (every single attempt)?

yes it does.

I will try to find some time to investigate too

Thank you very much Eric.

Quoted 458 lines

Thanks

Eric

Quoted 453 lines

Thanks!
Ricardo

Quoted 448 lines

Thanks

Eric

Quoted 441 lines
Quoted 158 lines
Quoted 142 lines
---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
tools/testing/selftests/kvm/Makefile +1 -1
index 4e548d7ab0ab..618c5903f478 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/processor.h +78 -0
index b7fa0c8551db..40aae31b4afc 100644
@@ -8,6 +8,7 @@
 #define SELFTEST_KVM_PROCESSOR_H
 
 #include "kvm_util.h"
+#include <linux/stringify.h>
 
 
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
@@ -18,6 +19,7 @@
 #define MAIR_EL1	3, 0, 10, 2, 0
 #define TTBR0_EL1	3, 0,  2, 0, 0
 #define SCTLR_EL1	3, 0,  1, 0, 0
+#define VBAR_EL1	3, 0, 12, 0, 0
 
 /*
  * Default MAIR
@@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
 void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
 			      struct kvm_vcpu_init *init, void *guest_code);
 
+struct ex_regs {
+	u64 regs[31];
+	u64 sp;
+	u64 pc;
+	u64 pstate;
+};
+
+#define VECTOR_NUM	16
+
+enum {
+	VECTOR_SYNC_CURRENT_SP0,
+	VECTOR_IRQ_CURRENT_SP0,
+	VECTOR_FIQ_CURRENT_SP0,
+	VECTOR_ERROR_CURRENT_SP0,
+
+	VECTOR_SYNC_CURRENT,
+	VECTOR_IRQ_CURRENT,
+	VECTOR_FIQ_CURRENT,
+	VECTOR_ERROR_CURRENT,
+
+	VECTOR_SYNC_LOWER_64,
+	VECTOR_IRQ_LOWER_64,
+	VECTOR_FIQ_LOWER_64,
+	VECTOR_ERROR_LOWER_64,
+
+	VECTOR_SYNC_LOWER_32,
+	VECTOR_IRQ_LOWER_32,
+	VECTOR_FIQ_LOWER_32,
+	VECTOR_ERROR_LOWER_32,
+};
+
+#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
+			   (v) == VECTOR_SYNC_CURRENT     || \
+			   (v) == VECTOR_SYNC_LOWER_64    || \
+			   (v) == VECTOR_SYNC_LOWER_32)
+
+/* Some common EC (Exception classes) */
+#define ESR_EC_ILLEGAL_INS	0x0e
+#define ESR_EC_SVC64		0x15
+#define ESR_EC_IABORT_CURRENT	0x21
+#define ESR_EC_DABORT_CURRENT	0x25
+#define ESR_EC_SERROR		0x2f
+#define ESR_EC_HW_BP_CURRENT	0x31
+#define ESR_EC_SSTEP_CURRENT	0x33
+#define ESR_EC_WP_CURRENT	0x35
+#define ESR_EC_BRK_INS		0x3C
+
+#define ESR_EC_NUM		64
+
+#define ESR_EC_SHIFT		26
+#define ESR_EC_MASK		(ESR_EC_NUM - 1)
+
+void vm_init_descriptor_tables(struct kvm_vm *vm);
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
+
+typedef void(*handler_fn)(struct ex_regs *);
+void vm_install_exception_handler(struct kvm_vm *vm,
+		int vector, int ec, handler_fn handler);
+void vm_install_vector_handler(struct kvm_vm *vm,
+		int vector, handler_fn handler);
+
+#define SPSR_D          (1 << 9)
+#define SPSR_SS         (1 << 21)
+
+#define write_sysreg(reg, val)						  \
+({									  \
+	u64 __val = (u64)(val);						  \
+	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
+})
+
+#define read_sysreg(reg)						  \
+({	u64 val;							  \
+	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
+	val;								  \
+})
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +12 -0
new file mode 100644
index 000000000000..8a560021892b
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+.macro save_registers, vector
+	add	sp, sp, #-16 * 17
+
+	stp	x0, x1, [sp, #16 * 0]
+	stp	x2, x3, [sp, #16 * 1]
+	stp	x4, x5, [sp, #16 * 2]
+	stp	x6, x7, [sp, #16 * 3]
+	stp	x8, x9, [sp, #16 * 4]
+	stp	x10, x11, [sp, #16 * 5]
+	stp	x12, x13, [sp, #16 * 6]
+	stp	x14, x15, [sp, #16 * 7]

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 22

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+	stp	x16, x17, [sp, #16 * 8]
+	stp	x18, x19, [sp, #16 * 9]
+	stp	x20, x21, [sp, #16 * 10]
+	stp	x22, x23, [sp, #16 * 11]
+	stp	x24, x25, [sp, #16 * 12]
+	stp	x26, x27, [sp, #16 * 13]
+	stp	x28, x29, [sp, #16 * 14]
+
+	.if \vector >= 8
+	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you always run in handler mode. Therefore, saving/restoring it is only overhead. If an exception handler wants to introspect it, it is already available in the relevant system register.

Quoted 2 lines

Or did you have something else in mind for it?

Not really. The reason for saving sp_el0 in there was just for consistency, so that handlers for both el0 and el1 exceptions could get the sp at regs->sp.

Restoring sp_el0 might be too much. So, what do you think of this v3: we keep the saving of sp_el0 into regs->sp (to keep things the same between el0 and el1) and delete the restoring of sp_el0?

Thanks,
Ricardo

Quoted 268 lines
Quoted 223 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +108 -0
+	.else
+	/*
+	 * This stores sp_el1 into ex_regs.sp so exception handlers can
+	 * "look" at it. It will _not_ be used to restore the sp_el1 on
+	 * return from the exception so handlers can not update it.
+	 */
+	mov	x1, sp
+	.endif
+	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
+
+	mrs	x1, elr_el1
+	mrs	x2, spsr_el1
+	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+.endm
+
+.macro restore_registers, vector
+	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+	msr	elr_el1, x1
+	msr	spsr_el1, x2
+
+	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
+	.if \vector >= 8
+	msr	sp_el0, x1
+	.endif
+
+	ldp	x28, x29, [sp, #16 * 14]
+	ldp	x26, x27, [sp, #16 * 13]
+	ldp	x24, x25, [sp, #16 * 12]
+	ldp	x22, x23, [sp, #16 * 11]
+	ldp	x20, x21, [sp, #16 * 10]
+	ldp	x18, x19, [sp, #16 * 9]
+	ldp	x16, x17, [sp, #16 * 8]
+	ldp	x14, x15, [sp, #16 * 7]
+	ldp	x12, x13, [sp, #16 * 6]
+	ldp	x10, x11, [sp, #16 * 5]
+	ldp	x8, x9, [sp, #16 * 4]
+	ldp	x6, x7, [sp, #16 * 3]
+	ldp	x4, x5, [sp, #16 * 2]
+	ldp	x2, x3, [sp, #16 * 1]
+	ldp	x0, x1, [sp, #16 * 0]
+
+	add	sp, sp, #16 * 17
+
+	eret
+.endm
+
+.pushsection ".entry.text", "ax"
+.balign 0x800
+.global vectors
+vectors:
+.popsection
+
+.set	vector, 0
+
+/*
+ * Build an exception handler for vector and append a jump to it into
+ * vectors (while making sure that it's 0x80 aligned).
+ */
+.macro HANDLER, label
+handler_\()\label:
+	save_registers vector
+	mov	x0, sp
+	mov	x1, #vector
+	bl	route_exception
+	restore_registers vector
+
+.pushsection ".entry.text", "ax"
+.balign 0x80
+	b	handler_\()\label
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+.macro HANDLER_INVALID
+.pushsection ".entry.text", "ax"
+.balign 0x80
+/* This will abort so no need to save and restore registers. */
+	mov	x0, #vector
+	b	kvm_exit_unexpected_vector
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+/*
+ * Caution: be sure to not add anything between the declaration of vectors
+ * above and these macro calls that will build the vectors table below it.
+ */
+	HANDLER_INVALID                         // Synchronous EL1t
+	HANDLER_INVALID                         // IRQ EL1t
+	HANDLER_INVALID                         // FIQ EL1t
+	HANDLER_INVALID                         // Error EL1t
+
+	HANDLER	el1h_sync                       // Synchronous EL1h
+	HANDLER	el1h_irq                        // IRQ EL1h
+	HANDLER el1h_fiq                        // FIQ EL1h
+	HANDLER	el1h_error                      // Error EL1h
+
+	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
+	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
+	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
+	HANDLER	el0_error_64                    // Error 64-bit EL0
+
+	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
+	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
+	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
+	HANDLER	el0_error_32                    // Error 32-bit EL0
tools/testing/selftests/kvm/lib/aarch64/processor.c +91 -0
index cee92d477dc0..25be71ec88be 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/compiler.h>
+#include <assert.h>
 
 #include "kvm_util.h"
 #include "../kvm_util_internal.h"
@@ -14,6 +15,8 @@
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
 #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
 
+vm_vaddr_t exception_handlers;
+
 static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
 {
 	return (v + vm->page_size) & ~(vm->page_size - 1);
@@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
 	va_end(ap);
 }
 
+void kvm_exit_unexpected_vector(int vector)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
+}
+
+void kvm_exit_unexpected_exception(int vector, uint64_t ec)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
+}
+
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
+	struct ucall uc;
+
+	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
+		return;
+
+	if (uc.args[2]) /* valid_ec */ {
+		assert(VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
+			uc.args[0], uc.args[1]);
+	} else {
+		assert(!VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx)",
+			uc.args[0]);
+	}
+}
+
+/*
+ * This exception handling code was heavily inspired on kvm-unit-tests. There
+ * is a set of default vector handlers stored in vector_handlers. These default
+ * vector handlers call user-installed handlers stored in exception_handlers.
+ * Synchronous handlers are indexed by (vector, ec), and irq handlers by
+ * (vector, ec=0).
+ */
+
+typedef void(*vector_fn)(struct ex_regs *, int vector);
+
+struct handlers {
+	vector_fn vector_handlers[VECTOR_NUM];
+	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
+};
+
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
+{
+	extern char vectors;
+
+	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
+}
+
+void default_sync_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+	uint64_t esr = read_sysreg(esr_el1);
+	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
+
+	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][ec])
+		handlers->exception_handlers[vector][ec](regs);
+	else
+		kvm_exit_unexpected_exception(vector, ec);
+}
+
+void default_irq_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][0])
+		handlers->exception_handlers[vector][0](regs);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void route_exception(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	if (handlers && handlers->vector_handlers[vector])
+		handlers->vector_handlers[vector](regs, vector);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void vm_init_descriptor_tables(struct kvm_vm *vm)
+{

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 439

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -0
+	struct handlers *handlers;
+
+	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
+			vm->page_size, 0, 0);
+
+	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
+	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;

How about FIQ, Error? Although they are unlikely, they are valid exceptions.

Quoted 14 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +14 -0
+
+	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
+}
+
+void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	assert(ec < ESR_EC_NUM);
+	handlers->exception_handlers[vector][ec] = handler;
+}

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 463

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -0
+
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(!VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	handlers->exception_handlers[vector][0] = handler;
 }

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#16 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

In reply to Auger Eric
Quoted 28 lines · Auger Eric

Hi Ricardo,

Quoted 25 lines · Ricardo Koller
Quoted 2 lines · Auger Eric

Hi Ricardo,

Hi Eric,

Thank you very much for the test.

Quoted 17 lines · Ricardo Koller
Quoted 12 lines · Marc Zyngier
Quoted 11 lines
Quoted 9 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The exception handling code is heavily inspired on kvm-unit-tests.

running the test on 5.12 I get

Hi Eric,

I'm able to reproduce the failure you are seeing on 5.6, specifically with kernels older than this commit:

4942dc6638b0 KVM: arm64: Write arch.mdcr_el2 changes since last vcpu_load on VHE

but not yet on v5.12. Could you share the commit of the kernel you are testing, please?

Thanks!
Ricardo

Quoted 489 lines
Quoted 11 lines
Quoted 11 lines

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794
Quoted 8 lines
Quoted 2 lines

I guess it is not an expected result. Any known bug waiting on the list?

Not expected. That should work, or at least abort early because there is no HW breakpoints support.

I'm trying to reproduce the failure; can you help me with some questions, please?

sure, please find the answers below.

Quoted 6 lines

- does your setup have support for hardware breakpoints? Can you try a 'dmesg | grep break'? I'm looking for something like 'hw-breakpoint:
found ...'. If there is no such line it's very likely that the check for "debug_ver >= 6" is not enough and the test should check for "num_breakpoints > 0".

[   25.640418] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
Quoted 1 line

- does it fail consistently (every single attempt)?

yes it does.

I will try to find some time to investigate too

Thanks

Eric

Quoted 453 lines

Thanks!
Ricardo

Quoted 448 lines

Thanks

Eric

Quoted 441 lines
Quoted 158 lines
Quoted 142 lines
---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
tools/testing/selftests/kvm/Makefile +1 -1
index 4e548d7ab0ab..618c5903f478 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/processor.h +78 -0
index b7fa0c8551db..40aae31b4afc 100644
@@ -8,6 +8,7 @@
 #define SELFTEST_KVM_PROCESSOR_H
 
 #include "kvm_util.h"
+#include <linux/stringify.h>
 
 
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
@@ -18,6 +19,7 @@
 #define MAIR_EL1	3, 0, 10, 2, 0
 #define TTBR0_EL1	3, 0,  2, 0, 0
 #define SCTLR_EL1	3, 0,  1, 0, 0
+#define VBAR_EL1	3, 0, 12, 0, 0
 
 /*
  * Default MAIR
@@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
 void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
 			      struct kvm_vcpu_init *init, void *guest_code);
 
+struct ex_regs {
+	u64 regs[31];
+	u64 sp;
+	u64 pc;
+	u64 pstate;
+};
+
+#define VECTOR_NUM	16
+
+enum {
+	VECTOR_SYNC_CURRENT_SP0,
+	VECTOR_IRQ_CURRENT_SP0,
+	VECTOR_FIQ_CURRENT_SP0,
+	VECTOR_ERROR_CURRENT_SP0,
+
+	VECTOR_SYNC_CURRENT,
+	VECTOR_IRQ_CURRENT,
+	VECTOR_FIQ_CURRENT,
+	VECTOR_ERROR_CURRENT,
+
+	VECTOR_SYNC_LOWER_64,
+	VECTOR_IRQ_LOWER_64,
+	VECTOR_FIQ_LOWER_64,
+	VECTOR_ERROR_LOWER_64,
+
+	VECTOR_SYNC_LOWER_32,
+	VECTOR_IRQ_LOWER_32,
+	VECTOR_FIQ_LOWER_32,
+	VECTOR_ERROR_LOWER_32,
+};
+
+#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
+			   (v) == VECTOR_SYNC_CURRENT     || \
+			   (v) == VECTOR_SYNC_LOWER_64    || \
+			   (v) == VECTOR_SYNC_LOWER_32)
+
+/* Some common EC (Exception classes) */
+#define ESR_EC_ILLEGAL_INS	0x0e
+#define ESR_EC_SVC64		0x15
+#define ESR_EC_IABORT_CURRENT	0x21
+#define ESR_EC_DABORT_CURRENT	0x25
+#define ESR_EC_SERROR		0x2f
+#define ESR_EC_HW_BP_CURRENT	0x31
+#define ESR_EC_SSTEP_CURRENT	0x33
+#define ESR_EC_WP_CURRENT	0x35
+#define ESR_EC_BRK_INS		0x3C
+
+#define ESR_EC_NUM		64
+
+#define ESR_EC_SHIFT		26
+#define ESR_EC_MASK		(ESR_EC_NUM - 1)
+
+void vm_init_descriptor_tables(struct kvm_vm *vm);
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
+
+typedef void(*handler_fn)(struct ex_regs *);
+void vm_install_exception_handler(struct kvm_vm *vm,
+		int vector, int ec, handler_fn handler);
+void vm_install_vector_handler(struct kvm_vm *vm,
+		int vector, handler_fn handler);
+
+#define SPSR_D          (1 << 9)
+#define SPSR_SS         (1 << 21)
+
+#define write_sysreg(reg, val)						  \
+({									  \
+	u64 __val = (u64)(val);						  \
+	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
+})
+
+#define read_sysreg(reg)						  \
+({	u64 val;							  \
+	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
+	val;								  \
+})
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +12 -0
new file mode 100644
index 000000000000..8a560021892b
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+.macro save_registers, vector
+	add	sp, sp, #-16 * 17
+
+	stp	x0, x1, [sp, #16 * 0]
+	stp	x2, x3, [sp, #16 * 1]
+	stp	x4, x5, [sp, #16 * 2]
+	stp	x6, x7, [sp, #16 * 3]
+	stp	x8, x9, [sp, #16 * 4]
+	stp	x10, x11, [sp, #16 * 5]
+	stp	x12, x13, [sp, #16 * 6]
+	stp	x14, x15, [sp, #16 * 7]

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 22

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+	stp	x16, x17, [sp, #16 * 8]
+	stp	x18, x19, [sp, #16 * 9]
+	stp	x20, x21, [sp, #16 * 10]
+	stp	x22, x23, [sp, #16 * 11]
+	stp	x24, x25, [sp, #16 * 12]
+	stp	x26, x27, [sp, #16 * 13]
+	stp	x28, x29, [sp, #16 * 14]
+
+	.if \vector >= 8
+	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you always run in handler mode. Therefore, saving/restoring it is only overhead. If an exception handler wants to introspect it, it is already available in the relevant system register.

Quoted 2 lines

Or did you have something else in mind for it?

Not really. The reason for saving sp_el0 in there was just for consistency, so that handlers for both el0 and el1 exceptions could get the sp at regs->sp.

Restoring sp_el0 might be too much. So, what do you think of this v3: we keep the saving of sp_el0 into regs->sp (to keep things the same between el0 and el1) and delete the restoring of sp_el0?

Thanks,
Ricardo

Quoted 268 lines
Quoted 223 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +108 -0
+	.else
+	/*
+	 * This stores sp_el1 into ex_regs.sp so exception handlers can
+	 * "look" at it. It will _not_ be used to restore the sp_el1 on
+	 * return from the exception so handlers can not update it.
+	 */
+	mov	x1, sp
+	.endif
+	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
+
+	mrs	x1, elr_el1
+	mrs	x2, spsr_el1
+	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+.endm
+
+.macro restore_registers, vector
+	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+	msr	elr_el1, x1
+	msr	spsr_el1, x2
+
+	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
+	.if \vector >= 8
+	msr	sp_el0, x1
+	.endif
+
+	ldp	x28, x29, [sp, #16 * 14]
+	ldp	x26, x27, [sp, #16 * 13]
+	ldp	x24, x25, [sp, #16 * 12]
+	ldp	x22, x23, [sp, #16 * 11]
+	ldp	x20, x21, [sp, #16 * 10]
+	ldp	x18, x19, [sp, #16 * 9]
+	ldp	x16, x17, [sp, #16 * 8]
+	ldp	x14, x15, [sp, #16 * 7]
+	ldp	x12, x13, [sp, #16 * 6]
+	ldp	x10, x11, [sp, #16 * 5]
+	ldp	x8, x9, [sp, #16 * 4]
+	ldp	x6, x7, [sp, #16 * 3]
+	ldp	x4, x5, [sp, #16 * 2]
+	ldp	x2, x3, [sp, #16 * 1]
+	ldp	x0, x1, [sp, #16 * 0]
+
+	add	sp, sp, #16 * 17
+
+	eret
+.endm
+
+.pushsection ".entry.text", "ax"
+.balign 0x800
+.global vectors
+vectors:
+.popsection
+
+.set	vector, 0
+
+/*
+ * Build an exception handler for vector and append a jump to it into
+ * vectors (while making sure that it's 0x80 aligned).
+ */
+.macro HANDLER, label
+handler_\()\label:
+	save_registers vector
+	mov	x0, sp
+	mov	x1, #vector
+	bl	route_exception
+	restore_registers vector
+
+.pushsection ".entry.text", "ax"
+.balign 0x80
+	b	handler_\()\label
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+.macro HANDLER_INVALID
+.pushsection ".entry.text", "ax"
+.balign 0x80
+/* This will abort so no need to save and restore registers. */
+	mov	x0, #vector
+	b	kvm_exit_unexpected_vector
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+/*
+ * Caution: be sure to not add anything between the declaration of vectors
+ * above and these macro calls that will build the vectors table below it.
+ */
+	HANDLER_INVALID                         // Synchronous EL1t
+	HANDLER_INVALID                         // IRQ EL1t
+	HANDLER_INVALID                         // FIQ EL1t
+	HANDLER_INVALID                         // Error EL1t
+
+	HANDLER	el1h_sync                       // Synchronous EL1h
+	HANDLER	el1h_irq                        // IRQ EL1h
+	HANDLER el1h_fiq                        // FIQ EL1h
+	HANDLER	el1h_error                      // Error EL1h
+
+	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
+	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
+	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
+	HANDLER	el0_error_64                    // Error 64-bit EL0
+
+	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
+	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
+	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
+	HANDLER	el0_error_32                    // Error 32-bit EL0
tools/testing/selftests/kvm/lib/aarch64/processor.c +91 -0
index cee92d477dc0..25be71ec88be 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/compiler.h>
+#include <assert.h>
 
 #include "kvm_util.h"
 #include "../kvm_util_internal.h"
@@ -14,6 +15,8 @@
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
 #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
 
+vm_vaddr_t exception_handlers;
+
 static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
 {
 	return (v + vm->page_size) & ~(vm->page_size - 1);
@@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
 	va_end(ap);
 }
 
+void kvm_exit_unexpected_vector(int vector)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
+}
+
+void kvm_exit_unexpected_exception(int vector, uint64_t ec)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
+}
+
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
+	struct ucall uc;
+
+	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
+		return;
+
+	if (uc.args[2]) /* valid_ec */ {
+		assert(VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
+			uc.args[0], uc.args[1]);
+	} else {
+		assert(!VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx)",
+			uc.args[0]);
+	}
+}
+
+/*
+ * This exception handling code was heavily inspired on kvm-unit-tests. There
+ * is a set of default vector handlers stored in vector_handlers. These default
+ * vector handlers call user-installed handlers stored in exception_handlers.
+ * Synchronous handlers are indexed by (vector, ec), and irq handlers by
+ * (vector, ec=0).
+ */
+
+typedef void(*vector_fn)(struct ex_regs *, int vector);
+
+struct handlers {
+	vector_fn vector_handlers[VECTOR_NUM];
+	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
+};
+
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
+{
+	extern char vectors;
+
+	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
+}
+
+void default_sync_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+	uint64_t esr = read_sysreg(esr_el1);
+	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
+
+	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][ec])
+		handlers->exception_handlers[vector][ec](regs);
+	else
+		kvm_exit_unexpected_exception(vector, ec);
+}
+
+void default_irq_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][0])
+		handlers->exception_handlers[vector][0](regs);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void route_exception(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	if (handlers && handlers->vector_handlers[vector])
+		handlers->vector_handlers[vector](regs, vector);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void vm_init_descriptor_tables(struct kvm_vm *vm)
+{

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 439

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -0
+	struct handlers *handlers;
+
+	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
+			vm->page_size, 0, 0);
+
+	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
+	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;

How about FIQ, Error? Although they are unlikely, they are valid exceptions.

Quoted 14 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +14 -0
+
+	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
+}
+
+void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	assert(ec < ESR_EC_NUM);
+	handlers->exception_handlers[vector][ec] = handler;
+}

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 463

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -0
+
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(!VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	handlers->exception_handlers[vector][0] = handler;
 }

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#17 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Hi Ricardo,

Quoted 32 lines · Ricardo Koller
Quoted 28 lines · Auger Eric

Hi Ricardo,

Quoted 25 lines · Ricardo Koller
Quoted 2 lines · Auger Eric

Hi Ricardo,

Hi Eric,

Thank you very much for the test.

Quoted 17 lines · Ricardo Koller
Quoted 12 lines · Marc Zyngier
Quoted 11 lines
Quoted 9 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The exception handling code is heavily inspired on kvm-unit-tests.

running the test on 5.12 I get

Hi Eric,

Quoted 7 lines

I'm able to reproduce the failure you are seeing on 5.6, specifically with kernels older than this commit:

4942dc6638b0 KVM: arm64: Write arch.mdcr_el2 changes since last vcpu_load on VHE

but not yet on v5.12. Could you share the commit of the kernel you are testing, please?

my host is a 5.12 kernel (8404c9fbc84b)

Thanks

Eric

Quoted 494 lines

Thanks!
Ricardo

Quoted 489 lines
Quoted 11 lines
Quoted 11 lines

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794
Quoted 8 lines
Quoted 2 lines

I guess it is not an expected result. Any known bug waiting on the list?

Not expected. That should work, or at least abort early because there is no HW breakpoints support.

I'm trying to reproduce the failure; can you help me with some questions, please?

sure, please find the answers below.

Quoted 6 lines

- does your setup have support for hardware breakpoints? Can you try a 'dmesg | grep break'? I'm looking for something like 'hw-breakpoint:
found ...'. If there is no such line it's very likely that the check for "debug_ver >= 6" is not enough and the test should check for "num_breakpoints > 0".

[   25.640418] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
Quoted 1 line

- does it fail consistently (every single attempt)?

yes it does.

I will try to find some time to investigate too

Thanks

Eric

Quoted 453 lines

Thanks!
Ricardo

Quoted 448 lines

Thanks

Eric

Quoted 441 lines
Quoted 158 lines
Quoted 142 lines
---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
tools/testing/selftests/kvm/Makefile +1 -1
index 4e548d7ab0ab..618c5903f478 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/processor.h +78 -0
index b7fa0c8551db..40aae31b4afc 100644
@@ -8,6 +8,7 @@
 #define SELFTEST_KVM_PROCESSOR_H
 
 #include "kvm_util.h"
+#include <linux/stringify.h>
 
 
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
@@ -18,6 +19,7 @@
 #define MAIR_EL1	3, 0, 10, 2, 0
 #define TTBR0_EL1	3, 0,  2, 0, 0
 #define SCTLR_EL1	3, 0,  1, 0, 0
+#define VBAR_EL1	3, 0, 12, 0, 0
 
 /*
  * Default MAIR
@@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
 void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
 			      struct kvm_vcpu_init *init, void *guest_code);
 
+struct ex_regs {
+	u64 regs[31];
+	u64 sp;
+	u64 pc;
+	u64 pstate;
+};
+
+#define VECTOR_NUM	16
+
+enum {
+	VECTOR_SYNC_CURRENT_SP0,
+	VECTOR_IRQ_CURRENT_SP0,
+	VECTOR_FIQ_CURRENT_SP0,
+	VECTOR_ERROR_CURRENT_SP0,
+
+	VECTOR_SYNC_CURRENT,
+	VECTOR_IRQ_CURRENT,
+	VECTOR_FIQ_CURRENT,
+	VECTOR_ERROR_CURRENT,
+
+	VECTOR_SYNC_LOWER_64,
+	VECTOR_IRQ_LOWER_64,
+	VECTOR_FIQ_LOWER_64,
+	VECTOR_ERROR_LOWER_64,
+
+	VECTOR_SYNC_LOWER_32,
+	VECTOR_IRQ_LOWER_32,
+	VECTOR_FIQ_LOWER_32,
+	VECTOR_ERROR_LOWER_32,
+};
+
+#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
+			   (v) == VECTOR_SYNC_CURRENT     || \
+			   (v) == VECTOR_SYNC_LOWER_64    || \
+			   (v) == VECTOR_SYNC_LOWER_32)
+
+/* Some common EC (Exception classes) */
+#define ESR_EC_ILLEGAL_INS	0x0e
+#define ESR_EC_SVC64		0x15
+#define ESR_EC_IABORT_CURRENT	0x21
+#define ESR_EC_DABORT_CURRENT	0x25
+#define ESR_EC_SERROR		0x2f
+#define ESR_EC_HW_BP_CURRENT	0x31
+#define ESR_EC_SSTEP_CURRENT	0x33
+#define ESR_EC_WP_CURRENT	0x35
+#define ESR_EC_BRK_INS		0x3C
+
+#define ESR_EC_NUM		64
+
+#define ESR_EC_SHIFT		26
+#define ESR_EC_MASK		(ESR_EC_NUM - 1)
+
+void vm_init_descriptor_tables(struct kvm_vm *vm);
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
+
+typedef void(*handler_fn)(struct ex_regs *);
+void vm_install_exception_handler(struct kvm_vm *vm,
+		int vector, int ec, handler_fn handler);
+void vm_install_vector_handler(struct kvm_vm *vm,
+		int vector, handler_fn handler);
+
+#define SPSR_D          (1 << 9)
+#define SPSR_SS         (1 << 21)
+
+#define write_sysreg(reg, val)						  \
+({									  \
+	u64 __val = (u64)(val);						  \
+	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
+})
+
+#define read_sysreg(reg)						  \
+({	u64 val;							  \
+	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
+	val;								  \
+})
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +12 -0
new file mode 100644
index 000000000000..8a560021892b
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+.macro save_registers, vector
+	add	sp, sp, #-16 * 17
+
+	stp	x0, x1, [sp, #16 * 0]
+	stp	x2, x3, [sp, #16 * 1]
+	stp	x4, x5, [sp, #16 * 2]
+	stp	x6, x7, [sp, #16 * 3]
+	stp	x8, x9, [sp, #16 * 4]
+	stp	x10, x11, [sp, #16 * 5]
+	stp	x12, x13, [sp, #16 * 6]
+	stp	x14, x15, [sp, #16 * 7]

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 22

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+	stp	x16, x17, [sp, #16 * 8]
+	stp	x18, x19, [sp, #16 * 9]
+	stp	x20, x21, [sp, #16 * 10]
+	stp	x22, x23, [sp, #16 * 11]
+	stp	x24, x25, [sp, #16 * 12]
+	stp	x26, x27, [sp, #16 * 13]
+	stp	x28, x29, [sp, #16 * 14]
+
+	.if \vector >= 8
+	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you always run in handler mode. Therefore, saving/restoring it is only overhead. If an exception handler wants to introspect it, it is already available in the relevant system register.

Quoted 2 lines

Or did you have something else in mind for it?

Not really. The reason for saving sp_el0 in there was just for consistency, so that handlers for both el0 and el1 exceptions could get the sp at regs->sp.

Restoring sp_el0 might be too much. So, what do you think of this v3: we keep the saving of sp_el0 into regs->sp (to keep things the same between el0 and el1) and delete the restoring of sp_el0?

Thanks,
Ricardo

Quoted 268 lines
Quoted 223 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +108 -0
+	.else
+	/*
+	 * This stores sp_el1 into ex_regs.sp so exception handlers can
+	 * "look" at it. It will _not_ be used to restore the sp_el1 on
+	 * return from the exception so handlers can not update it.
+	 */
+	mov	x1, sp
+	.endif
+	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
+
+	mrs	x1, elr_el1
+	mrs	x2, spsr_el1
+	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+.endm
+
+.macro restore_registers, vector
+	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+	msr	elr_el1, x1
+	msr	spsr_el1, x2
+
+	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
+	.if \vector >= 8
+	msr	sp_el0, x1
+	.endif
+
+	ldp	x28, x29, [sp, #16 * 14]
+	ldp	x26, x27, [sp, #16 * 13]
+	ldp	x24, x25, [sp, #16 * 12]
+	ldp	x22, x23, [sp, #16 * 11]
+	ldp	x20, x21, [sp, #16 * 10]
+	ldp	x18, x19, [sp, #16 * 9]
+	ldp	x16, x17, [sp, #16 * 8]
+	ldp	x14, x15, [sp, #16 * 7]
+	ldp	x12, x13, [sp, #16 * 6]
+	ldp	x10, x11, [sp, #16 * 5]
+	ldp	x8, x9, [sp, #16 * 4]
+	ldp	x6, x7, [sp, #16 * 3]
+	ldp	x4, x5, [sp, #16 * 2]
+	ldp	x2, x3, [sp, #16 * 1]
+	ldp	x0, x1, [sp, #16 * 0]
+
+	add	sp, sp, #16 * 17
+
+	eret
+.endm
+
+.pushsection ".entry.text", "ax"
+.balign 0x800
+.global vectors
+vectors:
+.popsection
+
+.set	vector, 0
+
+/*
+ * Build an exception handler for vector and append a jump to it into
+ * vectors (while making sure that it's 0x80 aligned).
+ */
+.macro HANDLER, label
+handler_\()\label:
+	save_registers vector
+	mov	x0, sp
+	mov	x1, #vector
+	bl	route_exception
+	restore_registers vector
+
+.pushsection ".entry.text", "ax"
+.balign 0x80
+	b	handler_\()\label
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+.macro HANDLER_INVALID
+.pushsection ".entry.text", "ax"
+.balign 0x80
+/* This will abort so no need to save and restore registers. */
+	mov	x0, #vector
+	b	kvm_exit_unexpected_vector
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+/*
+ * Caution: be sure to not add anything between the declaration of vectors
+ * above and these macro calls that will build the vectors table below it.
+ */
+	HANDLER_INVALID                         // Synchronous EL1t
+	HANDLER_INVALID                         // IRQ EL1t
+	HANDLER_INVALID                         // FIQ EL1t
+	HANDLER_INVALID                         // Error EL1t
+
+	HANDLER	el1h_sync                       // Synchronous EL1h
+	HANDLER	el1h_irq                        // IRQ EL1h
+	HANDLER el1h_fiq                        // FIQ EL1h
+	HANDLER	el1h_error                      // Error EL1h
+
+	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
+	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
+	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
+	HANDLER	el0_error_64                    // Error 64-bit EL0
+
+	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
+	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
+	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
+	HANDLER	el0_error_32                    // Error 32-bit EL0
tools/testing/selftests/kvm/lib/aarch64/processor.c +91 -0
index cee92d477dc0..25be71ec88be 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/compiler.h>
+#include <assert.h>
 
 #include "kvm_util.h"
 #include "../kvm_util_internal.h"
@@ -14,6 +15,8 @@
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
 #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
 
+vm_vaddr_t exception_handlers;
+
 static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
 {
 	return (v + vm->page_size) & ~(vm->page_size - 1);
@@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
 	va_end(ap);
 }
 
+void kvm_exit_unexpected_vector(int vector)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
+}
+
+void kvm_exit_unexpected_exception(int vector, uint64_t ec)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
+}
+
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
+	struct ucall uc;
+
+	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
+		return;
+
+	if (uc.args[2]) /* valid_ec */ {
+		assert(VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
+			uc.args[0], uc.args[1]);
+	} else {
+		assert(!VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx)",
+			uc.args[0]);
+	}
+}
+
+/*
+ * This exception handling code was heavily inspired on kvm-unit-tests. There
+ * is a set of default vector handlers stored in vector_handlers. These default
+ * vector handlers call user-installed handlers stored in exception_handlers.
+ * Synchronous handlers are indexed by (vector, ec), and irq handlers by
+ * (vector, ec=0).
+ */
+
+typedef void(*vector_fn)(struct ex_regs *, int vector);
+
+struct handlers {
+	vector_fn vector_handlers[VECTOR_NUM];
+	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
+};
+
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
+{
+	extern char vectors;
+
+	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
+}
+
+void default_sync_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+	uint64_t esr = read_sysreg(esr_el1);
+	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
+
+	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][ec])
+		handlers->exception_handlers[vector][ec](regs);
+	else
+		kvm_exit_unexpected_exception(vector, ec);
+}
+
+void default_irq_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][0])
+		handlers->exception_handlers[vector][0](regs);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void route_exception(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	if (handlers && handlers->vector_handlers[vector])
+		handlers->vector_handlers[vector](regs, vector);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void vm_init_descriptor_tables(struct kvm_vm *vm)
+{

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 439

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -0
+	struct handlers *handlers;
+
+	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
+			vm->page_size, 0, 0);
+
+	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
+	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;

How about FIQ, Error? Although they are unlikely, they are valid exceptions.

Quoted 14 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +14 -0
+
+	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
+}
+
+void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	assert(ec < ESR_EC_NUM);
+	handlers->exception_handlers[vector][ec] = handler;
+}

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 463

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +9 -0
+
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(!VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	handlers->exception_handlers[vector][0] = handler;
 }

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#18 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 38 lines · Auger Eric

Hi Ricardo,

Quoted 35 lines · Ricardo Koller
Quoted 28 lines · Auger Eric

Hi Ricardo,

Quoted 25 lines · Ricardo Koller
Quoted 2 lines · Auger Eric

Hi Ricardo,

Hi Eric,

Thank you very much for the test.

Quoted 17 lines · Ricardo Koller
Quoted 12 lines · Marc Zyngier
Quoted 11 lines
Quoted 9 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The exception handling code is heavily inspired on kvm-unit-tests.

running the test on 5.12 I get

Hi Eric,

I'm able to reproduce the failure you are seeing on 5.6, specifically with kernels older than this commit:

Quoted 7 lines
Quoted 5 lines

4942dc6638b0 KVM: arm64: Write arch.mdcr_el2 changes since last
vcpu_load on VHE

but not yet on v5.12. Could you share the commit of the kernel you are testing, please?

my host is a 5.12 kernel (8404c9fbc84b)

Time to compare notes then. What HW are you using? Running VHE or not?

Thanks,

        M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#19 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Hi,

Quoted 46 lines · Marc Zyngier
Quoted 45 lines · Auger Eric

Hi Ricardo,

Quoted 42 lines · Ricardo Koller
Quoted 32 lines · Auger Eric

Hi Ricardo,

Quoted 29 lines · Ricardo Koller
Quoted 2 lines · Auger Eric

Hi Ricardo,

Hi Eric,

Thank you very much for the test.

Quoted 21 lines · Ricardo Koller
Quoted 16 lines · Marc Zyngier
Quoted 15 lines
Quoted 13 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64
selftests. The exception handling defaults to an unhandled-exception
handler which aborts the test, just like x86. These handlers can be
overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a
previous commit, UCALL_UNHANDLED.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The exception handling code is heavily inspired on kvm-unit-tests.

running the test on 5.12 I get

Hi Eric,

I'm able to reproduce the failure you are seeing on 5.6, specifically with kernels older than this commit:

  4942dc6638b0 KVM: arm64: Write arch.mdcr_el2 changes since last vcpu_load on VHE

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

but not yet on v5.12. Could you share the commit of the kernel you are testing, please?

my host is a 5.12 kernel (8404c9fbc84b)

Time to compare notes then. What HW are you using? Running VHE or not?

VHE yes. Cavium Sabre system.

Thanks

Eric

Quoted 4 lines

Thanks,

        M.

#20 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 49 lines · Auger Eric

Hi,

Quoted 46 lines · Marc Zyngier
Quoted 45 lines · Auger Eric

Hi Ricardo,

Quoted 42 lines · Ricardo Koller
Quoted 32 lines · Auger Eric

Hi Ricardo,

Quoted 29 lines · Ricardo Koller
Quoted 2 lines · Auger Eric

Hi Ricardo,

Hi Eric,

Thank you very much for the test.

Quoted 21 lines · Ricardo Koller
Quoted 16 lines · Marc Zyngier
Quoted 15 lines
Quoted 13 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64
selftests. The exception handling defaults to an unhandled-exception
handler which aborts the test, just like x86. These handlers can be
overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a
previous commit, UCALL_UNHANDLED.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The exception handling code is heavily inspired on kvm-unit-tests.

running the test on 5.12 I get

Hi Eric,

I'm able to reproduce the failure you are seeing on 5.6, specifically with kernels older than this commit:

  4942dc6638b0 KVM: arm64: Write arch.mdcr_el2 changes since last vcpu_load on VHE

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

but not yet on v5.12. Could you share the commit of the kernel you are testing, please?

my host is a 5.12 kernel (8404c9fbc84b)

Time to compare notes then. What HW are you using? Running VHE or not?

VHE yes. Cavium Sabre system.

On my side. VHE (v5.12) on both QEMU (5.2.0 cpu=max) and Ampere Altra.

Quoted 8 lines

Thanks

Eric

Quoted 4 lines

Thanks,

        M.

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

reviewer
#21 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

In reply to Auger Eric

Hi Eric,

Quoted 11 lines · Auger Eric

running the test on 5.12 I get

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794

FYI I can also reproduce it on my VHE box. And Drew's suggestion [*] seemed to work for me. Is the ISB a requirement of architecture?

[*] https://lore.kernel.org/kvm/[email protected]/

Thanks,
Zenghui

maintainer
#22 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 15 lines · Zenghui Yu

Hi Eric,

Quoted 11 lines · Auger Eric

running the test on 5.12 I get

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794
Quoted 2 lines

FYI I can also reproduce it on my VHE box. And Drew's suggestion [*] seemed to work for me. Is the ISB a requirement of architecture?

Very much so. Given that there is no context synchronisation (such as ERET or an interrupt) in this code, the CPU is perfectly allowed to delay the system register effect as long as it can.

        M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#23 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 16 lines · Marc Zyngier
Quoted 15 lines · Zenghui Yu

Hi Eric,

Quoted 11 lines · Auger Eric

running the test on 5.12 I get

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794
Quoted 10 lines
Quoted 2 lines

FYI I can also reproduce it on my VHE box. And Drew's suggestion [*] seemed to work for me. Is the ISB a requirement of architecture?

Very much so. Given that there is no context synchronisation (such as ERET or an interrupt) in this code, the CPU is perfectly allowed to delay the system register effect as long as it can.

        M.
Signature · 1 line
Jazz is not dead. It just smells funny...

Thank you very much Eric, Zenghui, Marc, and Andrew (for the ISB suggestion)!

As per Zenghui test, will send a V3 that includes the missing ISBs. Hopefully that will fix the issue for Eric as well. It's very interesting that the CPU seems to _always_ reorder those instructions.

Thanks!
Ricardo
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

maintainer
#24 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 28 lines · Ricardo Koller
Quoted 19 lines · Marc Zyngier
Quoted 15 lines · Zenghui Yu

Hi Eric,

Quoted 11 lines · Auger Eric

running the test on 5.12 I get

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794
Quoted 2 lines

FYI I can also reproduce it on my VHE box. And Drew's suggestion [*] seemed to work for me. Is the ISB a requirement of architecture?

Quoted 7 lines

Very much so. Given that there is no context synchronisation (such as ERET or an interrupt) in this code, the CPU is perfectly allowed to delay the system register effect as long as it can.

        M.

--
Jazz is not dead. It just smells funny...

Quoted 6 lines

Thank you very much Eric, Zenghui, Marc, and Andrew (for the ISB suggestion)!

As per Zenghui test, will send a V3 that includes the missing ISBs. Hopefully that will fix the issue for Eric as well. It's very interesting that the CPU seems to _always_ reorder those instructions.

I suspect that because hitting the debug registers can be a costly operation (it mobilises a lot of resources in the CPU), there is a strong incentive to let it slide until there is an actual mandate to commit the resource.

It also means that SW can issue a bunch of these without too much overhead, and only pay the cost *once*.

Your N1 CPU seems to be less aggressive on this. Implement choice, I'd say (it probably is more aggressive than TX2 on other things). Also, QEMU will almost always hide these problems, due to the nature of TCG.

Thanks,

         M.
Signature · 5 lines
Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
#25 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 44 lines · Marc Zyngier
Quoted 28 lines · Ricardo Koller
Quoted 19 lines · Marc Zyngier
Quoted 15 lines · Zenghui Yu

Hi Eric,

Quoted 11 lines · Auger Eric

running the test on 5.12 I get

==== Test Assertion Failure ====
aarch64/debug-exceptions.c:232: false pid=6477 tid=6477 errno=4 - Interrupted system call

     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794
Quoted 2 lines

FYI I can also reproduce it on my VHE box. And Drew's suggestion [*] seemed to work for me. Is the ISB a requirement of architecture?

Quoted 7 lines

Very much so. Given that there is no context synchronisation (such as ERET or an interrupt) in this code, the CPU is perfectly allowed to delay the system register effect as long as it can.

        M.

--
Jazz is not dead. It just smells funny...

Quoted 6 lines

Thank you very much Eric, Zenghui, Marc, and Andrew (for the ISB suggestion)!

As per Zenghui test, will send a V3 that includes the missing ISBs. Hopefully that will fix the issue for Eric as well. It's very interesting that the CPU seems to _always_ reorder those instructions.

I suspect that because hitting the debug registers can be a costly operation (it mobilises a lot of resources in the CPU), there is a strong incentive to let it slide until there is an actual mandate to commit the resource.

It also means that SW can issue a bunch of these without too much overhead, and only pay the cost *once*.

Quoted 10 lines

Your N1 CPU seems to be less aggressive on this. Implement choice, I'd say (it probably is more aggressive than TX2 on other things). Also, QEMU will almost always hide these problems, due to the nature of TCG.

Thanks,

         M.
Signature · 1 line
Jazz is not dead. It just smells funny...

Thank you, this is very informative.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

maintainer
#26 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

In reply to Ricardo Koller
Quoted 4 lines · Ricardo Koller
Quoted 2 lines · Marc Zyngier

On Sat, 01 May 2021 00:24:06 +0100,
Ricardo Koller <[email protected]> wrote:

[...]

Quoted 8 lines
Quoted 8 lines

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 22

Quoted 2 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +2 -0
+	.if \vector >= 8
+	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you always run in handler mode. Therefore, saving/restoring it is only overhead. If an exception handler wants to introspect it, it is already available in the relevant system register.

Quoted 6 lines
Quoted 2 lines

Or did you have something else in mind for it?

Not really. The reason for saving sp_el0 in there was just for consistency, so that handlers for both el0 and el1 exceptions could get the sp at regs->sp.

We already have sp_el0 consistency by virtue of having it stored in in a sysreg.

Quoted 3 lines

Restoring sp_el0 might be too much. So, what do you think of this
v3: we keep the saving of sp_el0 into regs->sp (to keep things the same between el0 and el1) and delete the restoring of sp_el0?

To me, the whole purpose of saving some some context is to allow the exception handling code to run C code and introspect the interrupted state. But saving things that are not affected by the context change seems a bit pointless.

One thing I'd like to see though is to save sp_el1 as it was at the point of the exception (because that is meaningful to get the exception context -- think of an unaligned EL1 stack for example), which means correcting the value that gets saved.

So I would suggest to *only* save sp_el1, to always save it (irrespective of the exception coming from EL0 or EL1), and to save a retro-corrected value so that the handler can directly know where the previous stack pointer was.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#27 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

Quoted 36 lines · Marc Zyngier
Quoted 4 lines · Ricardo Koller
Quoted 2 lines · Marc Zyngier

On Sat, 01 May 2021 00:24:06 +0100,
Ricardo Koller <[email protected]> wrote:

[...]

Quoted 8 lines
Quoted 8 lines

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 22

Quoted 2 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +2 -0
+	.if \vector >= 8
+	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you always run in handler mode. Therefore, saving/restoring it is only overhead. If an exception handler wants to introspect it, it is already available in the relevant system register.

Quoted 6 lines
Quoted 2 lines

Or did you have something else in mind for it?

Not really. The reason for saving sp_el0 in there was just for consistency, so that handlers for both el0 and el1 exceptions could get the sp at regs->sp.

We already have sp_el0 consistency by virtue of having it stored in in a sysreg.

Quoted 3 lines

Restoring sp_el0 might be too much. So, what do you think of this
v3: we keep the saving of sp_el0 into regs->sp (to keep things the same between el0 and el1) and delete the restoring of sp_el0?

To me, the whole purpose of saving some some context is to allow the exception handling code to run C code and introspect the interrupted state. But saving things that are not affected by the context change seems a bit pointless.

Quoted 4 lines

One thing I'd like to see though is to save sp_el1 as it was at the point of the exception (because that is meaningful to get the exception context -- think of an unaligned EL1 stack for example), which means correcting the value that gets saved.

Got it. Replacing:
mov x1, sp
with:

	add     x1, sp, #16 * 17
Quoted 5 lines

So I would suggest to *only* save sp_el1, to always save it (irrespective of the exception coming from EL0 or EL1), and to save a retro-corrected value so that the handler can directly know where the previous stack pointer was.

Sounds good, will send a V3 accordingly.

Thanks!
Ricardo

Quoted 7 lines

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#28 lore raw

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

In reply to Ricardo Koller
Quoted 92 lines · Ricardo Koller

Add the infrastructure needed to enable exception handling in aarch64 selftests. The exception handling defaults to an unhandled-exception handler which aborts the test, just like x86. These handlers can be overridden by calling vm_install_vector_handler(vector) or vm_install_exception_handler(vector, ec). The unhandled exception reporting from the guest is done using the ucall type introduced in a previous commit, UCALL_UNHANDLED.

The exception handling code is heavily inspired on kvm-unit-tests.

---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
tools/testing/selftests/kvm/Makefile +1 -1
index 4e548d7ab0ab..618c5903f478 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/processor.h +38 -0
index b7fa0c8551db..40aae31b4afc 100644
@@ -8,6 +8,7 @@
 #define SELFTEST_KVM_PROCESSOR_H
 
 #include "kvm_util.h"
+#include <linux/stringify.h>
 
 
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
@@ -18,6 +19,7 @@
 #define MAIR_EL1	3, 0, 10, 2, 0
 #define TTBR0_EL1	3, 0,  2, 0, 0
 #define SCTLR_EL1	3, 0,  1, 0, 0
+#define VBAR_EL1	3, 0, 12, 0, 0
 
 /*
  * Default MAIR
@@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
 void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
 			      struct kvm_vcpu_init *init, void *guest_code);
 
+struct ex_regs {
+	u64 regs[31];
+	u64 sp;
+	u64 pc;
+	u64 pstate;
+};
+
+#define VECTOR_NUM	16
+
+enum {
+	VECTOR_SYNC_CURRENT_SP0,
+	VECTOR_IRQ_CURRENT_SP0,
+	VECTOR_FIQ_CURRENT_SP0,
+	VECTOR_ERROR_CURRENT_SP0,
+
+	VECTOR_SYNC_CURRENT,
+	VECTOR_IRQ_CURRENT,
+	VECTOR_FIQ_CURRENT,
+	VECTOR_ERROR_CURRENT,
+
+	VECTOR_SYNC_LOWER_64,
+	VECTOR_IRQ_LOWER_64,
+	VECTOR_FIQ_LOWER_64,
+	VECTOR_ERROR_LOWER_64,
+
+	VECTOR_SYNC_LOWER_32,
+	VECTOR_IRQ_LOWER_32,
+	VECTOR_FIQ_LOWER_32,
+	VECTOR_ERROR_LOWER_32,
+};
+
+#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
+			   (v) == VECTOR_SYNC_CURRENT     || \
+			   (v) == VECTOR_SYNC_LOWER_64    || \
+			   (v) == VECTOR_SYNC_LOWER_32)
+

tools/testing/selftests/kvm/include/aarch64/processor.h void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini · line 106

Quoted 10 lines
tools/testing/selftests/kvm/include/aarch64/processor.h +10 -0
+/* Some common EC (Exception classes) */
+#define ESR_EC_ILLEGAL_INS	0x0e
+#define ESR_EC_SVC64		0x15
+#define ESR_EC_IABORT_CURRENT	0x21
+#define ESR_EC_DABORT_CURRENT	0x25
+#define ESR_EC_SERROR		0x2f
+#define ESR_EC_HW_BP_CURRENT	0x31
+#define ESR_EC_SSTEP_CURRENT	0x33
+#define ESR_EC_WP_CURRENT	0x35
+#define ESR_EC_BRK_INS		0x3C

Let's leave it to the specific unit tests to define the above EC's as they are needed. The unit tests can also then decide if the defines belong in the common header or in the unit test itself.

Quoted 7 lines
tools/testing/selftests/kvm/include/aarch64/processor.h +7 -0
+
+#define ESR_EC_NUM		64
+
+#define ESR_EC_SHIFT		26
+#define ESR_EC_MASK		(ESR_EC_NUM - 1)
+
+void vm_init_descriptor_tables(struct kvm_vm *vm);

tools/testing/selftests/kvm/include/aarch64/processor.h void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini · line 123

Quoted 10 lines
tools/testing/selftests/kvm/include/aarch64/processor.h +10 -0
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
+
+typedef void(*handler_fn)(struct ex_regs *);
+void vm_install_exception_handler(struct kvm_vm *vm,
+		int vector, int ec, handler_fn handler);
+void vm_install_vector_handler(struct kvm_vm *vm,
+		int vector, handler_fn handler);
+
+#define SPSR_D          (1 << 9)
+#define SPSR_SS         (1 << 21)

I think these two SPSR defines belong directly in the debug unit test, rather than the common header.

Quoted 92 lines
tools/testing/selftests/kvm/include/aarch64/processor.h +13 -0
+
+#define write_sysreg(reg, val)						  \
+({									  \
+	u64 __val = (u64)(val);						  \
+	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
+})
+
+#define read_sysreg(reg)						  \
+({	u64 val;							  \
+	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
+	val;								  \
+})
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +72 -0
new file mode 100644
index 000000000000..8a560021892b
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+.macro save_registers, vector
+	add	sp, sp, #-16 * 17
+
+	stp	x0, x1, [sp, #16 * 0]
+	stp	x2, x3, [sp, #16 * 1]
+	stp	x4, x5, [sp, #16 * 2]
+	stp	x6, x7, [sp, #16 * 3]
+	stp	x8, x9, [sp, #16 * 4]
+	stp	x10, x11, [sp, #16 * 5]
+	stp	x12, x13, [sp, #16 * 6]
+	stp	x14, x15, [sp, #16 * 7]
+	stp	x16, x17, [sp, #16 * 8]
+	stp	x18, x19, [sp, #16 * 9]
+	stp	x20, x21, [sp, #16 * 10]
+	stp	x22, x23, [sp, #16 * 11]
+	stp	x24, x25, [sp, #16 * 12]
+	stp	x26, x27, [sp, #16 * 13]
+	stp	x28, x29, [sp, #16 * 14]
+
+	.if \vector >= 8
+	mrs	x1, sp_el0
+	.else
+	/*
+	 * This stores sp_el1 into ex_regs.sp so exception handlers can
+	 * "look" at it. It will _not_ be used to restore the sp_el1 on
+	 * return from the exception so handlers can not update it.
+	 */
+	mov	x1, sp
+	.endif
+	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
+
+	mrs	x1, elr_el1
+	mrs	x2, spsr_el1
+	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+.endm
+
+.macro restore_registers, vector
+	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+	msr	elr_el1, x1
+	msr	spsr_el1, x2
+
+	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
+	.if \vector >= 8
+	msr	sp_el0, x1
+	.endif
+
+	ldp	x28, x29, [sp, #16 * 14]
+	ldp	x26, x27, [sp, #16 * 13]
+	ldp	x24, x25, [sp, #16 * 12]
+	ldp	x22, x23, [sp, #16 * 11]
+	ldp	x20, x21, [sp, #16 * 10]
+	ldp	x18, x19, [sp, #16 * 9]
+	ldp	x16, x17, [sp, #16 * 8]
+	ldp	x14, x15, [sp, #16 * 7]
+	ldp	x12, x13, [sp, #16 * 6]
+	ldp	x10, x11, [sp, #16 * 5]
+	ldp	x8, x9, [sp, #16 * 4]
+	ldp	x6, x7, [sp, #16 * 3]
+	ldp	x4, x5, [sp, #16 * 2]
+	ldp	x2, x3, [sp, #16 * 1]
+	ldp	x0, x1, [sp, #16 * 0]
+
+	add	sp, sp, #16 * 17
+
+	eret
+.endm
+
+.pushsection ".entry.text", "ax"
+.balign 0x800
+.global vectors
+vectors:

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 82

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+.popsection
+
+.set	vector, 0
+
+/*
+ * Build an exception handler for vector and append a jump to it into
+ * vectors (while making sure that it's 0x80 aligned).
+ */
+.macro HANDLER, label
+handler_\()\label:

I don't think we need the \(). Based on [1] it may be necessary *after* a macro argument to separate it from the following text. I don't think it serves any purpose before the argument though.

[1] https://sourceware.org/binutils/docs/as/Macro.html

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+	save_registers vector
+	mov	x0, sp
+	mov	x1, #vector
+	bl	route_exception
+	restore_registers vector
+
+.pushsection ".entry.text", "ax"
+.balign 0x80
+	b	handler_\()\label
+.popsection

tools/testing/selftests/kvm/lib/aarch64/handlers.S line 102

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +10 -0
+
+.set	vector, vector + 1
+.endm
+
+.macro HANDLER_INVALID
+.pushsection ".entry.text", "ax"
+.balign 0x80
+/* This will abort so no need to save and restore registers. */
+	mov	x0, #vector
+	b	kvm_exit_unexpected_vector

I'd put a 'b .' here out of paranoia.

Quoted 41 lines
tools/testing/selftests/kvm/lib/aarch64/handlers.S new file +28 -0
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+/*
+ * Caution: be sure to not add anything between the declaration of vectors
+ * above and these macro calls that will build the vectors table below it.
+ */
+	HANDLER_INVALID                         // Synchronous EL1t
+	HANDLER_INVALID                         // IRQ EL1t
+	HANDLER_INVALID                         // FIQ EL1t
+	HANDLER_INVALID                         // Error EL1t
+
+	HANDLER	el1h_sync                       // Synchronous EL1h
+	HANDLER	el1h_irq                        // IRQ EL1h
+	HANDLER el1h_fiq                        // FIQ EL1h
+	HANDLER	el1h_error                      // Error EL1h
+
+	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
+	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
+	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
+	HANDLER	el0_error_64                    // Error 64-bit EL0
+
+	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
+	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
+	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
+	HANDLER	el0_error_32                    // Error 32-bit EL0
tools/testing/selftests/kvm/lib/aarch64/processor.c +1 -0
index cee92d477dc0..25be71ec88be 100644
@@ -6,6 +6,7 @@
  */
 
 #include <linux/compiler.h>
+#include <assert.h>
 
 #include "kvm_util.h"
 #include "../kvm_util_internal.h"

tools/testing/selftests/kvm/lib/aarch64/processor.c line 18

Quoted 4 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +1 -0
@@ -14,6 +15,8 @@
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
 #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
 
+vm_vaddr_t exception_handlers;

static? Also some functions below could maybe be static.

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 342

Quoted 11 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +4 -0
+
 static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
 {
 	return (v + vm->page_size) & ~(vm->page_size - 1);
@@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
 	va_end(ap);
 }
 
+void kvm_exit_unexpected_vector(int vector)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);

Instead of the 'b . ' suggested above, there could be a 'while (1) ;' here.

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 347

Quoted 5 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +5 -0
+}
+
+void kvm_exit_unexpected_exception(int vector, uint64_t ec)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);

And also a 'while (1) ;' here.

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +8 -0
+}
+
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
+	struct ucall uc;
+
+	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
+		return;
+
+	if (uc.args[2]) /* valid_ec */ {

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 367

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -0
+		assert(VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
+			uc.args[0], uc.args[1]);
+	} else {
+		assert(!VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx)",
+			uc.args[0]);
+	}

Can use TEST_FAIL for the above to TEST_ASSERT(false, ...)

Quoted 64 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +64 -0
+}
+
+/*
+ * This exception handling code was heavily inspired on kvm-unit-tests. There
+ * is a set of default vector handlers stored in vector_handlers. These default
+ * vector handlers call user-installed handlers stored in exception_handlers.
+ * Synchronous handlers are indexed by (vector, ec), and irq handlers by
+ * (vector, ec=0).
+ */
+
+typedef void(*vector_fn)(struct ex_regs *, int vector);
+
+struct handlers {
+	vector_fn vector_handlers[VECTOR_NUM];
+	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
+};
+
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
+{
+	extern char vectors;
+
+	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
+}
+
+void default_sync_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+	uint64_t esr = read_sysreg(esr_el1);
+	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
+
+	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][ec])
+		handlers->exception_handlers[vector][ec](regs);
+	else
+		kvm_exit_unexpected_exception(vector, ec);
+}
+
+void default_irq_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][0])
+		handlers->exception_handlers[vector][0](regs);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void route_exception(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	if (handlers && handlers->vector_handlers[vector])
+		handlers->vector_handlers[vector](regs, vector);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void vm_init_descriptor_tables(struct kvm_vm *vm)
+{
+	struct handlers *handlers;
+

tools/testing/selftests/kvm/lib/aarch64/processor.c void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...) · line 441

Quoted 10 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +10 -0
+	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
+			vm->page_size, 0, 0);
+
+	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
+	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;
+
+	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;

I see this is derived from the x86 code, so I guess it's OK for now, but having the handlers be VM state instead of VCPU state means the VCPUs will have to always share the same handlers. I think we should make them per-vcpu instead.

Quoted 25 lines
tools/testing/selftests/kvm/lib/aarch64/processor.c +21 -0
+}
+
+void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	assert(ec < ESR_EC_NUM);
+	handlers->exception_handlers[vector][ec] = handler;
+}
+
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(!VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	handlers->exception_handlers[vector][0] = handler;
 }
Signature · 1 line
2.31.1.527.g47e6f16901-goog

Thanks,
drew

#29 lore raw

PATCH v2 5/5 KVM: selftests: Add aarch64/debug-exceptions test

In reply to Ricardo Koller

Covers fundamental tests for debug exceptions. The guest installs and handle its debug exceptions itself, without KVM_SET_GUEST_DEBUG.

---
 tools/testing/selftests/kvm/.gitignore        |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/aarch64/debug-exceptions.c  | 244 ++++++++++++++++++
 .../selftests/kvm/include/aarch64/processor.h |  14 +-
 4 files changed, 254 insertions(+), 6 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/aarch64/debug-exceptions.c
tools/testing/selftests/kvm/.gitignore +1 -0
index e65d5572aefc..f09ed908422b 100644
@@ -1,4 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
+/aarch64/debug-exceptions
 /aarch64/get-reg-list
 /aarch64/get-reg-list-sve
 /aarch64/vgic_init
tools/testing/selftests/kvm/Makefile +1 -0
index 618c5903f478..2f92442c0cc9 100644
@@ -73,6 +73,7 @@ TEST_GEN_PROGS_x86_64 += memslot_modification_stress_test
 TEST_GEN_PROGS_x86_64 += set_memory_region_test
 TEST_GEN_PROGS_x86_64 += steal_time
 
+TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve
 TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
tools/testing/selftests/kvm/aarch64/debug-exceptions.c new file +244 -0

244 lines not on this page · raw message

tools/testing/selftests/kvm/include/aarch64/processor.h +8 -6
index 40aae31b4afc..a3ebef8e88c7 100644
@@ -14,12 +14,14 @@
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
 			   KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
 
-#define CPACR_EL1	3, 0,  1, 0, 2
-#define TCR_EL1		3, 0,  2, 0, 2
-#define MAIR_EL1	3, 0, 10, 2, 0
-#define TTBR0_EL1	3, 0,  2, 0, 0
-#define SCTLR_EL1	3, 0,  1, 0, 0
-#define VBAR_EL1	3, 0, 12, 0, 0
+#define CPACR_EL1               3, 0,  1, 0, 2
+#define TCR_EL1                 3, 0,  2, 0, 2
+#define MAIR_EL1                3, 0, 10, 2, 0
+#define TTBR0_EL1               3, 0,  2, 0, 0
+#define SCTLR_EL1               3, 0,  1, 0, 0
+#define VBAR_EL1                3, 0, 12, 0, 0
+
+#define ID_AA64DFR0_EL1         3, 0,  0, 5, 0
 
 /*
  * Default MAIR
Signature · 1 line
2.31.1.527.g47e6f16901-goog
Reviewed-by on v2 5/5
#30 lore raw

Re: [PATCH v2 5/5] KVM: selftests: Add aarch64/debug-exceptions test

Quoted 66 lines · Ricardo Koller

Covers fundamental tests for debug exceptions. The guest installs and handle its debug exceptions itself, without KVM_SET_GUEST_DEBUG.

---
 tools/testing/selftests/kvm/.gitignore        |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/aarch64/debug-exceptions.c  | 244 ++++++++++++++++++
 .../selftests/kvm/include/aarch64/processor.h |  14 +-
 4 files changed, 254 insertions(+), 6 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/aarch64/debug-exceptions.c
tools/testing/selftests/kvm/.gitignore +1 -0
index e65d5572aefc..f09ed908422b 100644
@@ -1,4 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
+/aarch64/debug-exceptions
 /aarch64/get-reg-list
 /aarch64/get-reg-list-sve
 /aarch64/vgic_init
tools/testing/selftests/kvm/Makefile +1 -0
index 618c5903f478..2f92442c0cc9 100644
@@ -73,6 +73,7 @@ TEST_GEN_PROGS_x86_64 += memslot_modification_stress_test
 TEST_GEN_PROGS_x86_64 += set_memory_region_test
 TEST_GEN_PROGS_x86_64 += steal_time
 
+TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve
 TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
tools/testing/selftests/kvm/aarch64/debug-exceptions.c new file +26 -0
new file mode 100644
index 000000000000..87352ee09211
@@ -0,0 +1,244 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_util.h>
+#include <kvm_util.h>
+#include <processor.h>
+
+#define VCPU_ID 0
+
+#define MDSCR_KDE	(1 << 13)
+#define MDSCR_MDE	(1 << 15)
+#define MDSCR_SS	(1 << 0)
+
+#define DBGBCR_LEN8	(0xff << 5)
+#define DBGBCR_EXEC	(0x0 << 3)
+#define DBGBCR_EL1	(0x1 << 1)
+#define DBGBCR_E	(0x1 << 0)
+
+#define DBGWCR_LEN8	(0xff << 5)
+#define DBGWCR_RD	(0x1 << 3)
+#define DBGWCR_WR	(0x2 << 3)
+#define DBGWCR_EL1	(0x1 << 1)
+#define DBGWCR_E	(0x1 << 0)
+
+extern unsigned char sw_bp, hw_bp, bp_svc, bp_brk, hw_wp, ss_start;
+static volatile uint64_t sw_bp_addr, hw_bp_addr;
+static volatile uint64_t wp_addr, wp_data_addr;
+static volatile uint64_t svc_addr;

tools/testing/selftests/kvm/aarch64/debug-exceptions.c line 36

Quoted 10 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c new file +10 -0
+static volatile uint64_t ss_addr[4], ss_idx;
+#define  PC(v)  ((uint64_t)&(v))
+
+static void reset_debug_state(void)
+{
+	asm volatile("msr daifset, #8");
+
+	write_sysreg(osdlr_el1, 0);
+	write_sysreg(oslar_el1, 0);
+	asm volatile("isb" : : : "memory");

We may want to create an isb() macro in include/aarch64/processor.h

Quoted 14 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c new file +14 -0
+
+	write_sysreg(mdscr_el1, 0);
+	/* This test only uses the first bp and wp slot. */
+	write_sysreg(dbgbvr0_el1, 0);
+	write_sysreg(dbgbcr0_el1, 0);
+	write_sysreg(dbgwcr0_el1, 0);
+	write_sysreg(dbgwvr0_el1, 0);
+	asm volatile("isb" : : : "memory");
+}
+
+static void install_wp(uint64_t addr)
+{
+	uint32_t wcr;
+	uint32_t mdscr;

tools/testing/selftests/kvm/aarch64/debug-exceptions.c line 60

Quoted 10 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c new file +10 -0
+
+	wcr = DBGWCR_LEN8 | DBGWCR_RD | DBGWCR_WR | DBGWCR_EL1 | DBGWCR_E;
+	write_sysreg(dbgwcr0_el1, wcr);
+	write_sysreg(dbgwvr0_el1, addr);
+	asm volatile("isb" : : : "memory");
+
+	asm volatile("msr daifclr, #8");
+
+	mdscr = read_sysreg(mdscr_el1) | MDSCR_KDE | MDSCR_MDE;
+	write_sysreg(mdscr_el1, mdscr);

no isb?

Quoted 6 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c new file +6 -0
+}
+
+static void install_hw_bp(uint64_t addr)
+{
+	uint32_t bcr;
+	uint32_t mdscr;

tools/testing/selftests/kvm/aarch64/debug-exceptions.c line 76

Quoted 10 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c new file +10 -0
+
+	bcr = DBGBCR_LEN8 | DBGBCR_EXEC | DBGBCR_EL1 | DBGBCR_E;
+	write_sysreg(dbgbcr0_el1, bcr);
+	write_sysreg(dbgbvr0_el1, addr);
+	asm volatile("isb" : : : "memory");
+
+	asm volatile("msr daifclr, #8");
+
+	mdscr = read_sysreg(mdscr_el1) | MDSCR_KDE | MDSCR_MDE;
+	write_sysreg(mdscr_el1, mdscr);

no isb?

tools/testing/selftests/kvm/aarch64/debug-exceptions.c line 86

Quoted 10 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c new file +10 -0
+}
+
+static void install_ss(void)
+{
+	uint32_t mdscr;
+
+	asm volatile("msr daifclr, #8");
+
+	mdscr = read_sysreg(mdscr_el1) | MDSCR_KDE | MDSCR_SS;
+	write_sysreg(mdscr_el1, mdscr);

no isb?

Quoted 186 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c new file +158 -0
+}
+
+static volatile char write_data;
+
+static void guest_code(void)
+{
+	GUEST_SYNC(0);
+
+	/* Software-breakpoint */
+	asm volatile("sw_bp: brk #0");
+	GUEST_ASSERT_EQ(sw_bp_addr, PC(sw_bp));
+
+	GUEST_SYNC(1);
+
+	/* Hardware-breakpoint */
+	reset_debug_state();
+	install_hw_bp(PC(hw_bp));
+	asm volatile("hw_bp: nop");
+	GUEST_ASSERT_EQ(hw_bp_addr, PC(hw_bp));
+
+	GUEST_SYNC(2);
+
+	/* Hardware-breakpoint + svc */
+	reset_debug_state();
+	install_hw_bp(PC(bp_svc));
+	asm volatile("bp_svc: svc #0");
+	GUEST_ASSERT_EQ(hw_bp_addr, PC(bp_svc));
+	GUEST_ASSERT_EQ(svc_addr, PC(bp_svc) + 4);
+
+	GUEST_SYNC(3);
+
+	/* Hardware-breakpoint + software-breakpoint */
+	reset_debug_state();
+	install_hw_bp(PC(bp_brk));
+	asm volatile("bp_brk: brk #0");
+	GUEST_ASSERT_EQ(sw_bp_addr, PC(bp_brk));
+	GUEST_ASSERT_EQ(hw_bp_addr, PC(bp_brk));
+
+	GUEST_SYNC(4);
+
+	/* Watchpoint */
+	reset_debug_state();
+	install_wp(PC(write_data));
+	write_data = 'x';
+	GUEST_ASSERT_EQ(write_data, 'x');
+	GUEST_ASSERT_EQ(wp_data_addr, PC(write_data));
+
+	GUEST_SYNC(5);
+
+	/* Single-step */
+	reset_debug_state();
+	install_ss();
+	ss_idx = 0;
+	asm volatile("ss_start:\n"
+		     "mrs x0, esr_el1\n"
+		     "add x0, x0, #1\n"
+		     "msr daifset, #8\n"
+		     : : : "x0");
+	GUEST_ASSERT_EQ(ss_addr[0], PC(ss_start));
+	GUEST_ASSERT_EQ(ss_addr[1], PC(ss_start) + 4);
+	GUEST_ASSERT_EQ(ss_addr[2], PC(ss_start) + 8);
+
+	GUEST_DONE();
+}
+
+static void guest_sw_bp_handler(struct ex_regs *regs)
+{
+	sw_bp_addr = regs->pc;
+	regs->pc += 4;
+}
+
+static void guest_hw_bp_handler(struct ex_regs *regs)
+{
+	hw_bp_addr = regs->pc;
+	regs->pstate |= SPSR_D;
+}
+
+static void guest_wp_handler(struct ex_regs *regs)
+{
+	wp_data_addr = read_sysreg(far_el1);
+	wp_addr = regs->pc;
+	regs->pstate |= SPSR_D;
+}
+
+static void guest_ss_handler(struct ex_regs *regs)
+{
+	GUEST_ASSERT_1(ss_idx < 4, ss_idx);
+	ss_addr[ss_idx++] = regs->pc;
+	regs->pstate |= SPSR_SS;
+}
+
+static void guest_svc_handler(struct ex_regs *regs)
+{
+	svc_addr = regs->pc;
+}
+
+static int debug_version(struct kvm_vm *vm)
+{
+	uint64_t id_aa64dfr0;
+
+	get_reg(vm, VCPU_ID, ARM64_SYS_REG(ID_AA64DFR0_EL1), &id_aa64dfr0);
+	return id_aa64dfr0 & 0xf;
+}
+
+int main(int argc, char *argv[])
+{
+	struct kvm_vm *vm;
+	struct ucall uc;
+	int stage;
+
+	vm = vm_create_default(VCPU_ID, 0, guest_code);
+	ucall_init(vm, NULL);
+
+	vm_init_descriptor_tables(vm);
+	vcpu_init_descriptor_tables(vm, VCPU_ID);
+
+	if (debug_version(vm) < 6) {
+		print_skip("Armv8 debug architecture not supported.");
+		kvm_vm_free(vm);
+		exit(KSFT_SKIP);
+	}
+
+	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
+				ESR_EC_BRK_INS, guest_sw_bp_handler);
+	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
+				ESR_EC_HW_BP_CURRENT, guest_hw_bp_handler);
+	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
+				ESR_EC_WP_CURRENT, guest_wp_handler);
+	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
+				ESR_EC_SSTEP_CURRENT, guest_ss_handler);
+	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
+				ESR_EC_SVC64, guest_svc_handler);
+
+	for (stage = 0; stage < 7; stage++) {
+		vcpu_run(vm, VCPU_ID);
+
+		switch (get_ucall(vm, VCPU_ID, &uc)) {
+		case UCALL_SYNC:
+			TEST_ASSERT(uc.args[1] == stage,
+				"Stage %d: Unexpected sync ucall, got %lx",
+				stage, (ulong)uc.args[1]);
+			break;
+		case UCALL_ABORT:
+			TEST_FAIL("%s at %s:%ld\n\tvalues: %#lx, %#lx",
+				(const char *)uc.args[0],
+				__FILE__, uc.args[1], uc.args[2], uc.args[3]);
+			break;
+		case UCALL_DONE:
+			goto done;
+		default:
+			TEST_FAIL("Unknown ucall %lu", uc.cmd);
+		}
+	}
+
+done:
+	kvm_vm_free(vm);
+	return 0;
+}
tools/testing/selftests/kvm/include/aarch64/processor.h +8 -6
index 40aae31b4afc..a3ebef8e88c7 100644
@@ -14,12 +14,14 @@
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
 			   KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
 
-#define CPACR_EL1	3, 0,  1, 0, 2
-#define TCR_EL1		3, 0,  2, 0, 2
-#define MAIR_EL1	3, 0, 10, 2, 0
-#define TTBR0_EL1	3, 0,  2, 0, 0
-#define SCTLR_EL1	3, 0,  1, 0, 0
-#define VBAR_EL1	3, 0, 12, 0, 0
+#define CPACR_EL1               3, 0,  1, 0, 2
+#define TCR_EL1                 3, 0,  2, 0, 2
+#define MAIR_EL1                3, 0, 10, 2, 0
+#define TTBR0_EL1               3, 0,  2, 0, 0
+#define SCTLR_EL1               3, 0,  1, 0, 0
+#define VBAR_EL1                3, 0, 12, 0, 0
+
+#define ID_AA64DFR0_EL1         3, 0,  0, 5, 0
 
 /*
  * Default MAIR
Signature · 1 line
2.31.1.527.g47e6f16901-goog

Other than the potentially missing isb's it looks good to me.

Thanks,
drew

maintainer
#31 lore raw
In reply to Ricardo Koller
Quoted 10 lines · Ricardo Koller

Hi,

These patches add a debug exception test in aarch64 KVM selftests while also adding basic exception handling support.

The structure of the exception handling is based on its x86 counterpart. Tests use the same calls to initialize exception handling and both architectures allow tests to override the handler for a particular vector, or (vector, ec) for synchronous exceptions in the arm64 case.

Quoted 6 lines

The debug test is similar to x86_64/debug_regs, except that the x86 one controls the debugging from outside the VM. This proposed arm64 test controls and handles debug exceptions from the inside.

Thanks,
Ricardo

Marc, are you going to queue this in your tree?

Thanks,

Paolo

Quoted 42 lines

v1 -> v2:

Addressed comments from Andrew and Marc (thank you very much):
- rename vm_handle_exception in all tests.
- introduce UCALL_UNHANDLED in x86 first.
- move GUEST_ASSERT_EQ to common utils header.
- handle sync and other exceptions separately: use two tables (like kvm-unit-tests).
- add two separate functions for installing sync versus other exceptions
- changes in handlers.S: use the same layout as user_pt_regs, treat the EL1t vectors as invalid, refactor the vector table creation to not use manual numbering, add comments, remove LR from the stored registers.
- changes in debug-exceptions.c: remove unused headers, use the common GUEST_ASSERT_EQ, use vcpu_run instead of _vcpu_run.
- changes in processor.h: write_sysreg with support for xzr, replace EL1 with current in macro names, define ESR_EC_MASK as ESR_EC_NUM-1.

Ricardo Koller (5):
KVM: selftests: Rename vm_handle_exception
KVM: selftests: Introduce UCALL_UNHANDLED for unhandled vector
reporting
KVM: selftests: Move GUEST_ASSERT_EQ to utils header
KVM: selftests: Add exception handling support for aarch64
KVM: selftests: Add aarch64/debug-exceptions test

  tools/testing/selftests/kvm/.gitignore        |   1 +
  tools/testing/selftests/kvm/Makefile          |   3 +-
  .../selftests/kvm/aarch64/debug-exceptions.c  | 244 ++++++++++++++++++
  .../selftests/kvm/include/aarch64/processor.h |  90 ++++++-
  .../testing/selftests/kvm/include/kvm_util.h  |  10 +
  .../selftests/kvm/include/x86_64/processor.h  |   4 +-
  .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++
  .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++
  .../selftests/kvm/lib/x86_64/processor.c      |  19 +-
  .../selftests/kvm/x86_64/kvm_pv_test.c        |   2 +-
  .../selftests/kvm/x86_64/tsc_msrs_test.c      |   9 -
  .../kvm/x86_64/userspace_msr_exit_test.c      |   8 +-
  .../selftests/kvm/x86_64/xapic_ipi_test.c     |   2 +-
  13 files changed, 611 insertions(+), 35 deletions(-)

create mode 100644 tools/testing/selftests/kvm/aarch64/debug-exceptions.c create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S

maintainer
#32 lore raw
Quoted 12 lines · Paolo Bonzini
Quoted 10 lines · Ricardo Koller

Hi,

These patches add a debug exception test in aarch64 KVM selftests while also adding basic exception handling support.

The structure of the exception handling is based on its x86 counterpart. Tests use the same calls to initialize exception handling and both architectures allow tests to override the handler for a particular vector, or (vector, ec) for synchronous exceptions in the arm64 case.

Quoted 8 lines
Quoted 6 lines

The debug test is similar to x86_64/debug_regs, except that the x86 one controls the debugging from outside the VM. This proposed arm64 test controls and handles debug exceptions from the inside.

Thanks,
Ricardo

Marc, are you going to queue this in your tree?

That is my plan. I'm waiting for the fixes branch to be merged in Linus' tree to start queuing stuff on top.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.