Skip to contentSkip to conversation

KVM: arm64: selftests: Fixes for single-step test

PATCH In review Sean Christopherson 4 participants 11 messages Nov 17, 2022

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author reviewer
#0 lore raw

cover KVM: arm64: selftests: Fixes for single-step test

Marc,

I would like to route this through Paolo's tree/queue for 6.2 along with a big pile of other selftests updates. I am hoping to get the selftests pile queued sooner than later as there is a lot of active development in that area, and don't want to have the selftests be in a broken state. I'm going to send Paolo a pull request shortly, I'll Cc you (and others) to keep everyone in the loop and give a chance for objections.

Fix a typo and an imminenent not-technically-a-bug bug in the single-step test where executing an atomic sequence in the guest with single-step enable will hang the guest due to eret clearing the local exclusive monitor.

Sean Christopherson (2):
KVM: arm64: selftests: Disable single-step with correct KVM define
KVM: arm64: selftests: Disable single-step without relying on ucall()

 .../selftests/kvm/aarch64/debug-exceptions.c  | 28 ++++++++++---------
 1 file changed, 15 insertions(+), 13 deletions(-)

base-commit: d663b8a285986072428a6a145e5994bc275df994

Signature · 1 line
2.38.1.431.g37b22c650d-goog
author reviewer
#1 lore raw

PATCH 1/2 KVM: arm64: selftests: Disable single-step with correct KVM define

Disable single-step by setting debug.control to KVM_GUESTDBG_ENABLE, not to SINGLE_STEP_DISABLE. The latter is an arbitrary test enum that just happens to have the same value as KVM_GUESTDBG_ENABLE, and so effectively disables single-step debug.

No functional change intended.

Cc Reiji Watanabe <[email protected]>
Fixes b18e4d4aebdd ("KVM: arm64: selftests: Add a test case for KVM_GUESTDBG_SINGLESTEP")
---
 tools/testing/selftests/kvm/aarch64/debug-exceptions.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/kvm/aarch64/debug-exceptions.c +1 -1
index 947bd201435c..91f55b45dfc7 100644
@@ -369,7 +369,7 @@ void test_single_step_from_userspace(int test_cnt)
 						KVM_GUESTDBG_SINGLESTEP;
 				ss_enable = true;
 			} else {
-				debug.control = SINGLE_STEP_DISABLE;
+				debug.control = KVM_GUESTDBG_ENABLE;
 				ss_enable = false;
 			}
 
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] KVM: arm64: selftests: Disable single-step with correct KVM define

Quoted 10 lines · Sean Christopherson

Disable single-step by setting debug.control to KVM_GUESTDBG_ENABLE, not to SINGLE_STEP_DISABLE. The latter is an arbitrary test enum that just happens to have the same value as KVM_GUESTDBG_ENABLE, and so effectively disables single-step debug.

No functional change intended.

Cc Reiji Watanabe <[email protected]>
Fixes b18e4d4aebdd ("KVM: arm64: selftests: Add a test case for KVM_GUESTDBG_SINGLESTEP")

--
Thanks,
Oliver
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Reviewed-by on v1 1/2
#3 lore raw

Re: [PATCH 1/2] KVM: arm64: selftests: Disable single-step with correct KVM define

In reply to Sean Christopherson
Quoted 20 lines · Sean Christopherson

Disable single-step by setting debug.control to KVM_GUESTDBG_ENABLE, not to SINGLE_STEP_DISABLE. The latter is an arbitrary test enum that just happens to have the same value as KVM_GUESTDBG_ENABLE, and so effectively disables single-step debug.

No functional change intended.

Cc Reiji Watanabe <[email protected]>
Fixes b18e4d4aebdd ("KVM: arm64: selftests: Add a test case for KVM_GUESTDBG_SINGLESTEP")
---
 tools/testing/selftests/kvm/aarch64/debug-exceptions.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/kvm/aarch64/debug-exceptions.c +0 -0
index 947bd201435c..91f55b45dfc7 100644

tools/testing/selftests/kvm/aarch64/debug-exceptions.c void test_single_step_from_userspace(int test_cnt) · line 372

Quoted 5 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c +1 -1
@@ -369,7 +369,7 @@ void test_single_step_from_userspace(int test_cnt)
                                                KVM_GUESTDBG_SINGLESTEP;
                                ss_enable = true;
                        } else {
-                               debug.control = SINGLE_STEP_DISABLE;
+                               debug.control = KVM_GUESTDBG_ENABLE;

Maybe I originally wanted to set it to 0:) There is no issue with KVM_GUESTDBG_ENABLE at all, as KVM_GUESTDBG_SINGLESTEP is cleared with that.

Thank you for fixing this!

Thanks,
Reiji

Quoted 6 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c +0 -1
                                ss_enable = false;
                        }
 
--

2.38.1.431.g37b22c650d-goog

author reviewer
#4 lore raw

PATCH 2/2 KVM: arm64: selftests: Disable single-step without relying on ucall()

In reply to Sean Christopherson

Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit ucall() to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme for ucall() will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.

  40e57c:       c85f7c20        ldxr    x0, [x1]
  40e580:       aa100011        orr     x17, x0, x16
  40e584:       c80ffc31        stlxr   w15, x17, [x1]
  40e588:       35ffffaf        cbnz    w15, 40e57c <__aarch64_ldset8_sync+0x1c>

the guest will hang as the local exclusive monitor is reset by eret, i.e. the stlxr will always fail due to the VM-Exit for the debug exception.

---
 .../selftests/kvm/aarch64/debug-exceptions.c  | 28 ++++++++++---------
 1 file changed, 15 insertions(+), 13 deletions(-)
tools/testing/selftests/kvm/aarch64/debug-exceptions.c +15 -13
index 91f55b45dfc7..65cd7e4f07fa 100644
@@ -241,7 +241,6 @@ static void guest_svc_handler(struct ex_regs *regs)
 
 enum single_step_op {
 	SINGLE_STEP_ENABLE = 0,
-	SINGLE_STEP_DISABLE = 1,
 };
 
 static void guest_code_ss(int test_cnt)
@@ -258,7 +257,7 @@ static void guest_code_ss(int test_cnt)
 		GUEST_SYNC(SINGLE_STEP_ENABLE);
 
 		/*
-		 * The userspace will veriry that the pc is as expected during
+		 * The userspace will verify that the pc is as expected during
 		 * single step execution between iter_ss_begin and iter_ss_end.
 		 */
 		asm volatile("iter_ss_begin:nop\n");
@@ -268,11 +267,9 @@ static void guest_code_ss(int test_cnt)
 		bvr = read_sysreg(dbgbvr0_el1);
 		wvr = read_sysreg(dbgwvr0_el1);
 
+		/* Userspace disables Single Step when the end is nigh. */
 		asm volatile("iter_ss_end:\n");
 
-		/* Disable Single Step execution */
-		GUEST_SYNC(SINGLE_STEP_DISABLE);
-
 		GUEST_ASSERT(bvr == w_bvr);
 		GUEST_ASSERT(wvr == w_wvr);
 	}
@@ -364,15 +361,12 @@ void test_single_step_from_userspace(int test_cnt)
 			TEST_ASSERT(cmd == UCALL_SYNC,
 				    "Unexpected ucall cmd 0x%lx", cmd);
 
-			if (uc.args[1] == SINGLE_STEP_ENABLE) {
-				debug.control = KVM_GUESTDBG_ENABLE |
-						KVM_GUESTDBG_SINGLESTEP;
-				ss_enable = true;
-			} else {
-				debug.control = KVM_GUESTDBG_ENABLE;
-				ss_enable = false;
-			}
+			TEST_ASSERT(uc.args[1] == SINGLE_STEP_ENABLE,
+				    "Unexpected ucall action 0x%lx", uc.args[1]);
 
+			debug.control = KVM_GUESTDBG_ENABLE |
+					KVM_GUESTDBG_SINGLESTEP;
+			ss_enable = true;
 			vcpu_guest_debug_set(vcpu, &debug);
 			continue;
 		}
@@ -385,6 +379,14 @@ void test_single_step_from_userspace(int test_cnt)
 			    "Unexpected pc 0x%lx (expected 0x%lx)",
 			    pc, test_pc);
 
+		if ((pc + 4) == (uint64_t)&iter_ss_end) {
+			test_pc = 0;
+			debug.control = KVM_GUESTDBG_ENABLE;
+			ss_enable = false;
+			vcpu_guest_debug_set(vcpu, &debug);
+			continue;
+		}
+
 		/*
 		 * If the current pc is between iter_ss_bgin and
 		 * iter_ss_end, the pc for the next KVM_EXIT_DEBUG should
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Reviewed-by on v1 2/2
#5 lore raw

Re: [PATCH 2/2] KVM: arm64: selftests: Disable single-step without relying on ucall()

Quoted 5 lines · Sean Christopherson

Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit ucall() to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme for ucall() will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.

Off topic, but I didn't ask when we were discussing this issue. What is the atomic used for in the pool-based ucall implementation?

Quoted 8 lines
  40e57c:       c85f7c20        ldxr    x0, [x1]
  40e580:       aa100011        orr     x17, x0, x16
  40e584:       c80ffc31        stlxr   w15, x17, [x1]
  40e588:       35ffffaf        cbnz    w15, 40e57c <__aarch64_ldset8_sync+0x1c>

the guest will hang as the local exclusive monitor is reset by eret, i.e. the stlxr will always fail due to the VM-Exit for the debug exception.

... due to the debug exception taken to EL2.

What's a VM-Exit anyways? ;-)

Quoted 4 lines

With the commit message nit:

--
Thanks,
Oliver

author reviewer
#6 lore raw

Re: [PATCH 2/2] KVM: arm64: selftests: Disable single-step without relying on ucall()

Quoted 9 lines · Oliver Upton
Quoted 5 lines · Sean Christopherson

Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit ucall() to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme for ucall() will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.

Off topic, but I didn't ask when we were discussing this issue. What is the atomic used for in the pool-based ucall implementation?

To avoid having to plumb an "id" into the guest, vCPUs grab a ucall entry from the pool on a first-come first-serve basis, and then release the entry when the ucall is complete. The current implementation is a bitmap, e.g. every possible entry has a bit in the map, and vCPUs do an atomic bit-test-and-set to claim an entry.

Ugh. And there's a bug. Of course I notice it after sending the pull request. Depsite being defined in atomic.h, and despite clear_bit() being atomic in the kernel, tools' clear_bit() isn't actually atomic. Grr.

Doesn't cause problems because there are so few multi-vCPU selftests, but that needs to be fixed. Best thing would be to fix clear_bit() itself.

author reviewer
#7 lore raw

Re: [PATCH 2/2] KVM: arm64: selftests: Disable single-step without relying on ucall()

Quoted 17 lines · Sean Christopherson
Quoted 9 lines · Oliver Upton
Quoted 5 lines · Sean Christopherson

Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit ucall() to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme for ucall() will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.

Off topic, but I didn't ask when we were discussing this issue. What is the atomic used for in the pool-based ucall implementation?

To avoid having to plumb an "id" into the guest, vCPUs grab a ucall entry from the pool on a first-come first-serve basis, and then release the entry when the ucall is complete. The current implementation is a bitmap, e.g. every possible entry has a bit in the map, and vCPUs do an atomic bit-test-and-set to claim an entry.

Quoted 6 lines

Ugh. And there's a bug. Of course I notice it after sending the pull request. Depsite being defined in atomic.h, and despite clear_bit() being atomic in the kernel, tools' clear_bit() isn't actually atomic. Grr.

Doesn't cause problems because there are so few multi-vCPU selftests, but that needs to be fixed. Best thing would be to fix clear_bit() itself.

Ha! And I bet when clear_bit() is fixed, this test will start failing again because the ucall() to activate single-step needs to release the entry _after_ exiting to the host, i.e. single-step will be enabled across the atomic region again.

author reviewer
#8 lore raw

Re: [PATCH 2/2] KVM: arm64: selftests: Disable single-step without relying on ucall()

Quoted 22 lines · Sean Christopherson
Quoted 21 lines · Sean Christopherson
Quoted 9 lines · Oliver Upton
Quoted 5 lines · Sean Christopherson

Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit ucall() to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme for ucall() will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.

Off topic, but I didn't ask when we were discussing this issue. What is the atomic used for in the pool-based ucall implementation?

To avoid having to plumb an "id" into the guest, vCPUs grab a ucall entry from the pool on a first-come first-serve basis, and then release the entry when the ucall is complete. The current implementation is a bitmap, e.g. every possible entry has a bit in the map, and vCPUs do an atomic bit-test-and-set to claim an entry.

Ugh. And there's a bug. Of course I notice it after sending the pull request. Depsite being defined in atomic.h, and despite clear_bit() being atomic in the kernel, tools' clear_bit() isn't actually atomic. Grr.

Quoted 7 lines
Quoted 2 lines

Doesn't cause problems because there are so few multi-vCPU selftests, but that needs to be fixed. Best thing would be to fix clear_bit() itself.

Ha! And I bet when clear_bit() is fixed, this test will start failing again because the ucall() to activate single-step needs to release the entry _after_ exiting to the host, i.e. single-step will be enabled across the atomic region again.

LOL, yep. Test gets stuck in __aarch64_ldclr8_sync().

Reviewed-by on v1 2/2
#9 lore raw

Re: [PATCH 2/2] KVM: arm64: selftests: Disable single-step without relying on ucall()

In reply to Sean Christopherson
Quoted 12 lines · Sean Christopherson

Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit ucall() to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme for ucall() will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.

  40e57c:       c85f7c20        ldxr    x0, [x1]
  40e580:       aa100011        orr     x17, x0, x16
  40e584:       c80ffc31        stlxr   w15, x17, [x1]
  40e588:       35ffffaf        cbnz    w15, 40e57c <__aarch64_ldset8_sync+0x1c>
Quoted 8 lines

the guest will hang as the local exclusive monitor is reset by eret, i.e. the stlxr will always fail due to the VM-Exit for the debug exception.

maintainer Acked-by on the v1 series
#10 lore raw
In reply to Sean Christopherson
Quoted 18 lines · Sean Christopherson

Marc,

I would like to route this through Paolo's tree/queue for 6.2 along with a big pile of other selftests updates. I am hoping to get the selftests pile queued sooner than later as there is a lot of active development in that area, and don't want to have the selftests be in a broken state. I'm going to send Paolo a pull request shortly, I'll Cc you (and others) to keep everyone in the loop and give a chance for objections.

Fix a typo and an imminenent not-technically-a-bug bug in the single-step test where executing an atomic sequence in the guest with single-step enable will hang the guest due to eret clearing the local exclusive monitor.

Quoted 3 lines

Sean Christopherson (2):
KVM: arm64: selftests: Disable single-step with correct KVM define
KVM: arm64: selftests: Disable single-step without relying on ucall()

I'm obviously late to the party, but hey... For the record:

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.