KVM: arm64: selftests: Fixes for single-step test
Sean Christopherson
reviewercover KVM: arm64: selftests: Fixes for single-step test
Marc,
I would like to route this through Paolo's tree/queue for 6.2 along with a big pile of other selftests updates. I am hoping to get the selftests pile queued sooner than later as there is a lot of active development in that area, and don't want to have the selftests be in a broken state. I'm going to send Paolo a pull request shortly, I'll Cc you (and others) to keep everyone in the loop and give a chance for objections.
Fix a typo and an imminenent not-technically-a-bug bug in the single-step test where executing an atomic sequence in the guest with single-step enable will hang the guest due to eret clearing the local exclusive monitor.
Sean Christopherson (2):
KVM: arm64: selftests: Disable single-step with correct KVM define
KVM: arm64: selftests: Disable single-step without relying on ucall()
.../selftests/kvm/aarch64/debug-exceptions.c | 28 ++++++++++--------- 1 file changed, 15 insertions(+), 13 deletions(-)
base-commit: d663b8a285986072428a6a145e5994bc275df994
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH 1/2 KVM: arm64: selftests: Disable single-step with correct KVM define
Disable single-step by setting debug.control to KVM_GUESTDBG_ENABLE, not to SINGLE_STEP_DISABLE. The latter is an arbitrary test enum that just happens to have the same value as KVM_GUESTDBG_ENABLE, and so effectively disables single-step debug.
No functional change intended.
--- tools/testing/selftests/kvm/aarch64/debug-exceptions.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/kvm/aarch64/debug-exceptions.c +1 -1
@@ -369,7 +369,7 @@ void test_single_step_from_userspace(int test_cnt)KVM_GUESTDBG_SINGLESTEP ; ss_enable = true; }else { - debug.control =SINGLE_STEP_DISABLE ; + debug.control =KVM_GUESTDBG_ENABLE ; ss_enable = false; }
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Re: [PATCH 1/2] KVM: arm64: selftests: Disable single-step with correct KVM define
Quoted 10 lines · Sean Christopherson
Disable single-step by setting debug.control to
KVM_GUESTDBG_ENABLE, not toSINGLE_STEP_DISABLE. The latter is an arbitrary test enum that just happens to have the same value asKVM_GUESTDBG_ENABLE, and so effectively disables single-step debug.No functional change intended.
Cc Reiji Watanabe <[email protected]>Fixes b18e4d4aebdd ("KVM: arm64: selftests: Add a test case for KVM_GUESTDBG_SINGLESTEP")Signed-off-by Sean Christopherson <[email protected]>
--
Thanks,
Oliver
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 1/2] KVM: arm64: selftests: Disable single-step with correct KVM define
In reply to Sean ChristophersonQuoted 20 lines · Sean Christopherson
Disable single-step by setting debug.control to
KVM_GUESTDBG_ENABLE, not toSINGLE_STEP_DISABLE. The latter is an arbitrary test enum that just happens to have the same value asKVM_GUESTDBG_ENABLE, and so effectively disables single-step debug.No functional change intended.
Cc Reiji Watanabe <[email protected]>Fixes b18e4d4aebdd ("KVM: arm64: selftests: Add a test case for KVM_GUESTDBG_SINGLESTEP")Signed-off-by Sean Christopherson <[email protected]>--- tools/testing/selftests/kvm/aarch64/debug-exceptions.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/testing/selftests/kvm/aarch64/debug-exceptions.c +0
-0
tools/testing/selftests/kvm/aarch64/debug-exceptions.c void test_single_step_from_userspace(int test_cnt) · line 372
Quoted 5 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c +1
-1@@ -369,7 +369,7 @@ void test_single_step_from_userspace(int test_cnt)KVM_GUESTDBG_SINGLESTEP ; ss_enable = true; }else { - debug.control =SINGLE_STEP_DISABLE ; + debug.control =KVM_GUESTDBG_ENABLE ;
Maybe I originally wanted to set it to 0:) There is no issue with KVM_GUESTDBG_ENABLE at all, as KVM_GUESTDBG_SINGLESTEP is cleared with that.
Thank you for fixing this!
Thanks,
Reiji
Quoted 6 lines
tools/testing/selftests/kvm/aarch64/debug-exceptions.c +0
-1ss_enable = false; } --2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH 2/2 KVM: arm64: selftests: Disable single-step without relying on ucall()
In reply to Sean ChristophersonAutomatically disable single-step when the guest reaches the end of the verified section instead of using an explicit ucall() to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme for ucall() will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.
40e57c: c85f7c20 ldxr x0, [x1] 40e580: aa100011 orr x17, x0, x16 40e584: c80ffc31 stlxr w15, x17, [x1] 40e588: 35ffffaf cbnz w15, 40e57c <__aarch64_ldset8_sync+0x1c>
the guest will hang as the local exclusive monitor is reset by eret, i.e. the stlxr will always fail due to the VM-Exit for the debug exception.
--- .../selftests/kvm/aarch64/debug-exceptions.c | 28 ++++++++++--------- 1 file changed, 15 insertions(+), 13 deletions(-)
tools/testing/selftests/kvm/aarch64/debug-exceptions.c +15 -13
@@ -241,7 +241,6 @@ static void guest_svc_handler(struct ex_regs *regs)enum single_step_op {SINGLE_STEP_ENABLE =0 , -SINGLE_STEP_DISABLE =1 , };static void guest_code_ss (int test_cnt)
@@ -258,7 +257,7 @@ static void guest_code_ss(int test_cnt)GUEST_SYNC (SINGLE_STEP_ENABLE );/* -* The userspace will veriry that the pc is as expected during +* The userspace will verify that the pc is as expected during * single step execution between iter_ss_begin and iter_ss_end. */ asmvolatile ("iter_ss_begin:nop\n" );
@@ -268,11 +267,9 @@ static void guest_code_ss(int test_cnt) bvr =read_sysreg (dbgbvr0_el1); wvr =read_sysreg (dbgwvr0_el1); +/* Userspace disables Single Step when the end is nigh. */ asmvolatile ("iter_ss_end:\n" ); -/* Disable Single Step execution */ -GUEST_SYNC (SINGLE_STEP_DISABLE ); -GUEST_ASSERT (bvr == w_bvr);GUEST_ASSERT (wvr == w_wvr); }
@@ -364,15 +361,12 @@ void test_single_step_from_userspace(int test_cnt)TEST_ASSERT (cmd ==UCALL_SYNC ,"Unexpected ucall cmd 0x%lx" , cmd); -if (uc.args[1 ] ==SINGLE_STEP_ENABLE ) { - debug.control =KVM_GUESTDBG_ENABLE | -KVM_GUESTDBG_SINGLESTEP ; - ss_enable = true; - }else { - debug.control =KVM_GUESTDBG_ENABLE ; - ss_enable = false; - } +TEST_ASSERT (uc.args[1 ] ==SINGLE_STEP_ENABLE , +"Unexpected ucall action 0x%lx" , uc.args[1 ]); + debug.control =KVM_GUESTDBG_ENABLE | +KVM_GUESTDBG_SINGLESTEP ; + ss_enable = true;vcpu_guest_debug_set (vcpu, &debug);continue ; }
@@ -385,6 +379,14 @@ void test_single_step_from_userspace(int test_cnt)"Unexpected pc 0x%lx (expected 0x%lx)" , pc, test_pc); +if ((pc +4 ) == (uint64_t )&iter_ss_end) { + test_pc =0 ; + debug.control =KVM_GUESTDBG_ENABLE ; + ss_enable = false; +vcpu_guest_debug_set (vcpu, &debug); +continue ; + } +/* * If the current pc is between iter_ss_bgin and * iter_ss_end, the pc for the next KVM_EXIT_DEBUG should
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Re: [PATCH 2/2] KVM: arm64: selftests: Disable single-step without relying on ucall()
Quoted 5 lines · Sean Christopherson
Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit
ucall()to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme forucall()will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.
Off topic, but I didn't ask when we were discussing this issue. What is the atomic used for in the pool-based ucall implementation?
Quoted 8 lines
40e57c: c85f7c20 ldxr x0, [x1] 40e580: aa100011 orr x17, x0, x16 40e584: c80ffc31 stlxr w15, x17, [x1] 40e588: 35ffffaf cbnz w15, 40e57c <__aarch64_ldset8_sync+0x1c>the guest will hang as the local exclusive monitor is reset by eret, i.e. the stlxr will always fail due to the VM-Exit for the debug exception.
... due to the debug exception taken to EL2.
What's a VM-Exit anyways? ;-)
Quoted 4 lines
Cc Oliver Upton <[email protected]>Cc Marc Zyngier <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
With the commit message nit:
--
Thanks,
Oliver
Sean Christopherson
reviewerRe: [PATCH 2/2] KVM: arm64: selftests: Disable single-step without relying on ucall()
Quoted 9 lines · Oliver Upton
Quoted 5 lines · Sean Christopherson
Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit
ucall()to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme forucall()will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.Off topic, but I didn't ask when we were discussing this issue. What is the atomic used for in the pool-based ucall implementation?
To avoid having to plumb an "id" into the guest, vCPUs grab a ucall entry from the pool on a first-come first-serve basis, and then release the entry when the ucall is complete. The current implementation is a bitmap, e.g. every possible entry has a bit in the map, and vCPUs do an atomic bit-test-and-set to claim an entry.
Ugh. And there's a bug. Of course I notice it after sending the pull request. Depsite being defined in atomic.h, and despite clear_bit() being atomic in the kernel, tools' clear_bit() isn't actually atomic. Grr.
Doesn't cause problems because there are so few multi-vCPU selftests, but that needs to be fixed. Best thing would be to fix clear_bit() itself.
Sean Christopherson
reviewerRe: [PATCH 2/2] KVM: arm64: selftests: Disable single-step without relying on ucall()
Quoted 17 lines · Sean Christopherson
Quoted 9 lines · Oliver Upton
Quoted 5 lines · Sean Christopherson
Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit
ucall()to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme forucall()will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.Off topic, but I didn't ask when we were discussing this issue. What is the atomic used for in the pool-based ucall implementation?
To avoid having to plumb an "id" into the guest, vCPUs grab a ucall entry from the pool on a first-come first-serve basis, and then release the entry when the ucall is complete. The current implementation is a bitmap, e.g. every possible entry has a bit in the map, and vCPUs do an atomic bit-test-and-set to claim an entry.
Quoted 6 lines
Ugh. And there's a bug. Of course I notice it after sending the pull request. Depsite being defined in
atomic.h, and despiteclear_bit()being atomic in the kernel, tools'clear_bit()isn't actually atomic. Grr.Doesn't cause problems because there are so few multi-vCPU selftests, but that needs to be fixed. Best thing would be to fix
clear_bit()itself.
Ha! And I bet when clear_bit() is fixed, this test will start failing again because the ucall() to activate single-step needs to release the entry _after_ exiting to the host, i.e. single-step will be enabled across the atomic region again.
Sean Christopherson
reviewerRe: [PATCH 2/2] KVM: arm64: selftests: Disable single-step without relying on ucall()
Quoted 22 lines · Sean Christopherson
Quoted 21 lines · Sean Christopherson
Quoted 9 lines · Oliver Upton
Quoted 5 lines · Sean Christopherson
Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit
ucall()to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme forucall()will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.Off topic, but I didn't ask when we were discussing this issue. What is the atomic used for in the pool-based ucall implementation?
To avoid having to plumb an "id" into the guest, vCPUs grab a ucall entry from the pool on a first-come first-serve basis, and then release the entry when the ucall is complete. The current implementation is a bitmap, e.g. every possible entry has a bit in the map, and vCPUs do an atomic bit-test-and-set to claim an entry.
Ugh. And there's a bug. Of course I notice it after sending the pull request. Depsite being defined in
atomic.h, and despiteclear_bit()being atomic in the kernel, tools'clear_bit()isn't actually atomic. Grr.
Quoted 7 lines
Quoted 2 lines
Doesn't cause problems because there are so few multi-vCPU selftests, but that needs to be fixed. Best thing would be to fix
clear_bit()itself.Ha! And I bet when
clear_bit()is fixed, this test will start failing again because theucall()to activate single-step needs to release the entry _after_ exiting to the host, i.e. single-step will be enabled across the atomic region again.
LOL, yep. Test gets stuck in __aarch64_ldclr8_sync().
Re: [PATCH 2/2] KVM: arm64: selftests: Disable single-step without relying on ucall()
In reply to Sean ChristophersonQuoted 12 lines · Sean Christopherson
Automatically disable single-step when the guest reaches the end of the verified section instead of using an explicit
ucall()to ask userspace to disable single-step. An upcoming change to implement a pool-based scheme forucall()will add an atomic operation (bit test and set) in the guest ucall code, and if the compiler generate "old school" atomics, e.g.40e57c: c85f7c20 ldxr x0, [x1] 40e580: aa100011 orr x17, x0, x16 40e584: c80ffc31 stlxr w15, x17, [x1] 40e588: 35ffffaf cbnz w15, 40e57c <__aarch64_ldset8_sync+0x1c>
Quoted 8 lines
the guest will hang as the local exclusive monitor is reset by eret, i.e. the stlxr will always fail due to the VM-Exit for the debug exception.
Cc Oliver Upton <[email protected]>Cc Marc Zyngier <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 18 lines · Sean Christopherson
Marc,
I would like to route this through Paolo's tree/queue for 6.2 along with a big pile of other selftests updates. I am hoping to get the selftests pile queued sooner than later as there is a lot of active development in that area, and don't want to have the selftests be in a broken state. I'm going to send Paolo a pull request shortly, I'll Cc you (and others) to keep everyone in the loop and give a chance for objections.
Fix a typo and an imminenent not-technically-a-bug bug in the single-step test where executing an atomic sequence in the guest with single-step enable will hang the guest due to eret clearing the local exclusive monitor.
Quoted 3 lines
Sean Christopherson (2):
KVM: arm64: selftests: Disable single-step with correct KVM define
KVM: arm64: selftests: Disable single-step without relying onucall()
I'm obviously late to the party, but hey... For the record:
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.