RISC-V: KVM: Guest Debug Support - Software Breakpoint Part
Subsystems (4)
cover RISC-V: KVM: Guest Debug Support - Software Breakpoint Part
This series implements the ?KVM Guset Debug? feature on RISC-V. This is an existing feature which is already supported by some other arches. It allows us to debug a RISC-V KVM guest from GDB in host side.
As the first stage, the software breakpoints (ebreak instruction) is implemented. HW breakpoints support will come later after a synthetically consideration with the SBI debug trigger extension.
A selftest case was added in this series. Manual test was done on QEMU RISC-V hypervisor emulator. (add '-s' to enable the gdbserver in QEMU)
This series is based on Linux 6.8-rc2 and also available at:
https://github.com/Du-Chao/kvm-riscv/tree/guest_debug_sw
The matched QEMU is available at:
https://github.com/Du-Chao/qemu/tree/riscv_gd_sw
Changes from RFC->v1:
- Rebased on Linux 6.8-rc2.
- Merge PATCH1 and PATCH2 into one patch. (If Paolo's change
https://lore.kernel.org/kvm/20240131233056.10845-8-pbonzini at redhat.com/ is adopted, then we can keep 'arch/riscv/include/uapi/asm/kvm.h' untouched)
- kselftest case added.
RFC link:
https://lore.kernel.org/kvm/20231221095002.7404-1-duchao at eswincomputing.com
Chao Du (3):
RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()
RISC-V: KVM: Handle breakpoint exits for VCPU
RISC-V: KVM: selftests: Add breakpoints test support
arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu.c | 15 +++++- arch/riscv/kvm/vcpu_exit.c | 4 ++ arch/riscv/kvm/vm.c | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../testing/selftests/kvm/riscv/breakpoints.c | 49 +++++++++++++++++++ 6 files changed, 69 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/kvm/riscv/breakpoints.c
--
2.17.1
PATCH 1/3 RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()
kvm_vm_ioctl_check_extension(): Return 1 if KVM_CAP_SET_GUEST_DEBUG is being checked.
kvm_arch_vcpu_ioctl_set_guest_debug(): Update the guest_debug flags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.
--- arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu.c | 15 +++++++++++++-- arch/riscv/kvm/vm.c | 1 + 3 files changed, 15 insertions(+), 2 deletions(-)
arch/riscv/include/uapi/asm/kvm.h +1 -0
@@ -17,6 +17,7 @@#define __KVM_HAVE_IRQ_LINE#define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_GUEST_DEBUG#define KVM_COALESCED_MMIO_PAGE_OFFSET 1
arch/riscv/kvm/vcpu.c +13 -2
@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,int kvm_arch_vcpu_ioctl_set_guest_debug (struct kvm_vcpu *vcpu,struct kvm_guest_debug *dbg) { -/* TODO; To be implemented later. */ -return -EINVAL ; +if (dbg->control &KVM_GUESTDBG_ENABLE ) { +if (vcpu->guest_debug != dbg->control) { + vcpu->guest_debug = dbg->control; +csr_clear (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }else { +if (vcpu->guest_debug !=0 ) { + vcpu->guest_debug =0 ; +csr_set (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + } + +return 0 ; }static void kvm_riscv_vcpu_setup_config (struct kvm_vcpu *vcpu)
arch/riscv/kvm/vm.c +1 -0
@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_READONLY_MEM :case KVM_CAP_MP_STATE :case KVM_CAP_IMMEDIATE_EXIT : +case KVM_CAP_SET_GUEST_DEBUG : r =1 ;break ;case KVM_CAP_NR_VCPUS :
--
2.17.1
Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:
kvm_vm_ioctl_check_extension(): Return 1 ifKVM_CAP_SET_GUEST_DEBUGis being checked.
kvm_arch_vcpu_ioctl_set_guest_debug(): Update theguest_debugflags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.Signed-off-by Chao Du <[email protected]>--- arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu.c | 15 +++++++++++++-- arch/riscv/kvm/vm.c | 1 + 3 files changed, 15 insertions(+), 2 deletions(-)arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -17,6 +17,7 @@#define __KVM_HAVE_IRQ_LINE#define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_GUEST_DEBUG#define KVM_COALESCED_MMIO_PAGE_OFFSET 1arch/riscv/kvm/vcpu.c +1
-2@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,int kvm_arch_vcpu_ioctl_set_guest_debug (struct kvm_vcpu *vcpu,struct kvm_guest_debug *dbg) { -/* TODO; To be implemented later. */ -return -EINVAL ; +if (dbg->control &KVM_GUESTDBG_ENABLE ) {
arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488
Quoted 10 lines
arch/riscv/kvm/vcpu.c +10
-0+if (vcpu->guest_debug != dbg->control) { + vcpu->guest_debug = dbg->control; +csr_clear (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }else { +if (vcpu->guest_debug !=0 ) { + vcpu->guest_debug =0 ; +csr_set (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }
This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.
To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "struct kvm_vcpu_config" which is pre-initialized in kvm_riscv_vcpu_setup_config() without setting EXC_BREAKPOINT bit.
2) The kvm_arch_vcpu_ioctl_set_guest_debug() should only set/clear EXC_BREAKPOINT bit in "hedeleg" of "struct kvm_vcpu_config".
3) The kvm_riscv_vcpu_swap_in_guest_state() must write the HEDELEG csr before entering the Guest/VM.
Regards,
Anup
Quoted 25 lines
arch/riscv/kvm/vcpu.c +2
-0+ +return 0 ; }static void kvm_riscv_vcpu_setup_config (struct kvm_vcpu *vcpu)arch/riscv/kvm/vm.c +1
-1@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_READONLY_MEM :case KVM_CAP_MP_STATE :case KVM_CAP_IMMEDIATE_EXIT : +case KVM_CAP_SET_GUEST_DEBUG : r =1 ;break ;case KVM_CAP_NR_VCPUS : --2.17.1
--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()
Quoted 64 lines · Anup Patel
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:
kvm_vm_ioctl_check_extension(): Return 1 ifKVM_CAP_SET_GUEST_DEBUGis being checked.
kvm_arch_vcpu_ioctl_set_guest_debug(): Update theguest_debugflags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.Signed-off-by Chao Du <[email protected]>--- arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu.c | 15 +++++++++++++-- arch/riscv/kvm/vm.c | 1 + 3 files changed, 15 insertions(+), 2 deletions(-)arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -17,6 +17,7 @@#define __KVM_HAVE_IRQ_LINE#define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_GUEST_DEBUG#define KVM_COALESCED_MMIO_PAGE_OFFSET 1arch/riscv/kvm/vcpu.c +1
-2@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,int kvm_arch_vcpu_ioctl_set_guest_debug (struct kvm_vcpu *vcpu,struct kvm_guest_debug *dbg) { -/* TODO; To be implemented later. */ -return -EINVAL ; +if (dbg->control &KVM_GUESTDBG_ENABLE ) {arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488
Quoted 10 lines
arch/riscv/kvm/vcpu.c +10
-0+if (vcpu->guest_debug != dbg->control) { + vcpu->guest_debug = dbg->control; +csr_clear (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }else { +if (vcpu->guest_debug !=0 ) { + vcpu->guest_debug =0 ; +csr_set (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.
To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "structkvm_vcpu_config" which is pre-initialized inkvm_riscv_vcpu_setup_config()without settingEXC_BREAKPOINTbit.
2) Thekvm_arch_vcpu_ioctl_set_guest_debug()should only set/clearEXC_BREAKPOINTbit in "hedeleg" of "structkvm_vcpu_config".
3) Thekvm_riscv_vcpu_swap_in_guest_state()must write the HEDELEG csr before entering the Guest/VM.
Quoted 3 lines
Regards,
Anup
Thanks for the review and detailed suggestion.
Maybe we could make it a bit easier:
1) The kvm_arch_vcpu_ioctl_set_guest_debug() only update vcpu->guest_debug accordingly.
2) The kvm_riscv_vcpu_swap_in_guest_state() check vcpu->guest_debug and set/clear the HEDELEG csr accordingly.
Could you confirm if this is OK?
If yes, I will post another revision.
Regards,
Chao
Quoted 25 lines
Quoted 25 lines
arch/riscv/kvm/vcpu.c +2
-0+ +return 0 ; }static void kvm_riscv_vcpu_setup_config (struct kvm_vcpu *vcpu)arch/riscv/kvm/vm.c +1
-1@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_READONLY_MEM :case KVM_CAP_MP_STATE :case KVM_CAP_IMMEDIATE_EXIT : +case KVM_CAP_SET_GUEST_DEBUG : r =1 ;break ;case KVM_CAP_NR_VCPUS : --2.17.1
--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()
Quoted 70 lines · On Tue, Feb 20, 2024 at 8:31?AM Chao Du <[email protected]> wrote:
Quoted 64 lines · Anup Patel
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:
kvm_vm_ioctl_check_extension(): Return 1 ifKVM_CAP_SET_GUEST_DEBUGis being checked.
kvm_arch_vcpu_ioctl_set_guest_debug(): Update theguest_debugflags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.Signed-off-by Chao Du <[email protected]>--- arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu.c | 15 +++++++++++++-- arch/riscv/kvm/vm.c | 1 + 3 files changed, 15 insertions(+), 2 deletions(-)arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -17,6 +17,7 @@#define __KVM_HAVE_IRQ_LINE#define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_GUEST_DEBUG#define KVM_COALESCED_MMIO_PAGE_OFFSET 1arch/riscv/kvm/vcpu.c +1
-2@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,int kvm_arch_vcpu_ioctl_set_guest_debug (struct kvm_vcpu *vcpu,struct kvm_guest_debug *dbg) { -/* TODO; To be implemented later. */ -return -EINVAL ; +if (dbg->control &KVM_GUESTDBG_ENABLE ) {arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488
Quoted 10 lines
arch/riscv/kvm/vcpu.c +10
-0+if (vcpu->guest_debug != dbg->control) { + vcpu->guest_debug = dbg->control; +csr_clear (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }else { +if (vcpu->guest_debug !=0 ) { + vcpu->guest_debug =0 ; +csr_set (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.
To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "structkvm_vcpu_config" which is pre-initialized inkvm_riscv_vcpu_setup_config()without settingEXC_BREAKPOINTbit.
2) Thekvm_arch_vcpu_ioctl_set_guest_debug()should only set/clearEXC_BREAKPOINTbit in "hedeleg" of "structkvm_vcpu_config".
3) Thekvm_riscv_vcpu_swap_in_guest_state()must write the HEDELEG csr before entering the Guest/VM.Quoted 3 lines
Regards,
Anup
Quoted 8 lines
Thanks for the review and detailed suggestion.
Maybe we could make it a bit easier:
1) Thekvm_arch_vcpu_ioctl_set_guest_debug()only updatevcpu->guest_debugaccordingly.
2) Thekvm_riscv_vcpu_swap_in_guest_state()checkvcpu->guest_debugand set/clear the HEDELEG csr accordingly.Could you confirm if this is OK?
Your suggestion will work but it adds an additional "if ()" check in kvm_riscv_vcpu_swap_in_guest_state() which is in the hot path.
I am still leaning towards what I suggested.
Regards,
Anup
Quoted 34 lines
If yes, I will post another revision.
Regards,
ChaoQuoted 21 lines
Quoted 21 lines
arch/riscv/kvm/vcpu.c +2
-0+ +return 0 ; }static void kvm_riscv_vcpu_setup_config (struct kvm_vcpu *vcpu)arch/riscv/kvm/vm.c +1
-1@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_READONLY_MEM :case KVM_CAP_MP_STATE :case KVM_CAP_IMMEDIATE_EXIT : +case KVM_CAP_SET_GUEST_DEBUG : r =1 ;break ;case KVM_CAP_NR_VCPUS : --2.17.1
Quoted 4 lines
Quoted 4 lines
--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()
Quoted 79 lines · Anup Patel
Quoted 77 lines · On Tue, Feb 20, 2024 at 8:31?AM Chao Du <[email protected]> wrote:
Quoted 64 lines · Anup Patel
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:
kvm_vm_ioctl_check_extension(): Return 1 ifKVM_CAP_SET_GUEST_DEBUGis being checked.
kvm_arch_vcpu_ioctl_set_guest_debug(): Update theguest_debugflags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.Signed-off-by Chao Du <[email protected]>--- arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu.c | 15 +++++++++++++-- arch/riscv/kvm/vm.c | 1 + 3 files changed, 15 insertions(+), 2 deletions(-)arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -17,6 +17,7 @@#define __KVM_HAVE_IRQ_LINE#define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_GUEST_DEBUG#define KVM_COALESCED_MMIO_PAGE_OFFSET 1arch/riscv/kvm/vcpu.c +1
-2@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,int kvm_arch_vcpu_ioctl_set_guest_debug (struct kvm_vcpu *vcpu,struct kvm_guest_debug *dbg) { -/* TODO; To be implemented later. */ -return -EINVAL ; +if (dbg->control &KVM_GUESTDBG_ENABLE ) {arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488
Quoted 10 lines
arch/riscv/kvm/vcpu.c +10
-0+if (vcpu->guest_debug != dbg->control) { + vcpu->guest_debug = dbg->control; +csr_clear (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }else { +if (vcpu->guest_debug !=0 ) { + vcpu->guest_debug =0 ; +csr_set (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.
To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "structkvm_vcpu_config" which is pre-initialized inkvm_riscv_vcpu_setup_config()without settingEXC_BREAKPOINTbit.
2) Thekvm_arch_vcpu_ioctl_set_guest_debug()should only set/clearEXC_BREAKPOINTbit in "hedeleg" of "structkvm_vcpu_config".
3) Thekvm_riscv_vcpu_swap_in_guest_state()must write the HEDELEG csr before entering the Guest/VM.Quoted 3 lines
Regards,
AnupThanks for the review and detailed suggestion.
Maybe we could make it a bit easier:
1) Thekvm_arch_vcpu_ioctl_set_guest_debug()only updatevcpu->guest_debugaccordingly.
2) Thekvm_riscv_vcpu_swap_in_guest_state()checkvcpu->guest_debugand set/clear the HEDELEG csr accordingly.
Quoted 5 lines
Quoted 1 line
Could you confirm if this is OK?
Your suggestion will work but it adds an additional "if ()" check in
kvm_riscv_vcpu_swap_in_guest_state()which is in the hot path.
Yes, it makes sense.
I will prepare a V2 patch.
Thanks,
Chao
Quoted 39 lines
I am still leaning towards what I suggested.
Regards,
AnupQuoted 34 lines
If yes, I will post another revision.
Regards,
ChaoQuoted 21 lines
Quoted 21 lines
arch/riscv/kvm/vcpu.c +2
-0+ +return 0 ; }static void kvm_riscv_vcpu_setup_config (struct kvm_vcpu *vcpu)arch/riscv/kvm/vm.c +1
-1@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_READONLY_MEM :case KVM_CAP_MP_STATE :case KVM_CAP_IMMEDIATE_EXIT : +case KVM_CAP_SET_GUEST_DEBUG : r =1 ;break ;case KVM_CAP_NR_VCPUS : --2.17.1
Quoted 4 lines
Quoted 4 lines
--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()
Quoted 81 lines · On Wed, Feb 21, 2024 at 7:28?AM Chao Du <[email protected]> wrote:
Quoted 79 lines · Anup Patel
Quoted 77 lines · On Tue, Feb 20, 2024 at 8:31?AM Chao Du <[email protected]> wrote:
Quoted 64 lines · Anup Patel
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:
kvm_vm_ioctl_check_extension(): Return 1 ifKVM_CAP_SET_GUEST_DEBUGis being checked.
kvm_arch_vcpu_ioctl_set_guest_debug(): Update theguest_debugflags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.Signed-off-by Chao Du <[email protected]>--- arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu.c | 15 +++++++++++++-- arch/riscv/kvm/vm.c | 1 + 3 files changed, 15 insertions(+), 2 deletions(-)arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -17,6 +17,7 @@#define __KVM_HAVE_IRQ_LINE#define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_GUEST_DEBUG#define KVM_COALESCED_MMIO_PAGE_OFFSET 1arch/riscv/kvm/vcpu.c +1
-2@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,int kvm_arch_vcpu_ioctl_set_guest_debug (struct kvm_vcpu *vcpu,struct kvm_guest_debug *dbg) { -/* TODO; To be implemented later. */ -return -EINVAL ; +if (dbg->control &KVM_GUESTDBG_ENABLE ) {arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488
Quoted 10 lines
arch/riscv/kvm/vcpu.c +10
-0+if (vcpu->guest_debug != dbg->control) { + vcpu->guest_debug = dbg->control; +csr_clear (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }else { +if (vcpu->guest_debug !=0 ) { + vcpu->guest_debug =0 ; +csr_set (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.
To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "structkvm_vcpu_config" which is pre-initialized inkvm_riscv_vcpu_setup_config()without settingEXC_BREAKPOINTbit.
2) Thekvm_arch_vcpu_ioctl_set_guest_debug()should only set/clearEXC_BREAKPOINTbit in "hedeleg" of "structkvm_vcpu_config".
3) Thekvm_riscv_vcpu_swap_in_guest_state()must write the HEDELEG csr before entering the Guest/VM.Quoted 3 lines
Regards,
AnupThanks for the review and detailed suggestion.
Maybe we could make it a bit easier:
1) Thekvm_arch_vcpu_ioctl_set_guest_debug()only updatevcpu->guest_debugaccordingly.
2) Thekvm_riscv_vcpu_swap_in_guest_state()checkvcpu->guest_debugand set/clear the HEDELEG csr accordingly.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Could you confirm if this is OK?
Your suggestion will work but it adds an additional "if ()" check in
kvm_riscv_vcpu_swap_in_guest_state()which is in the hot path.Yes, it makes sense.
I will prepare a V2 patch.
I just realized that kvm_arch_vcpu_load() will be a much better place to add hedeleg CSR write instead of kvm_riscv_vcpu_swap_in_guest_state(). This way it will be only at the start hot-path (aka run-loop).
Regards,
Anup
Quoted 43 lines
Thanks,
ChaoQuoted 39 lines
I am still leaning towards what I suggested.
Regards,
AnupQuoted 34 lines
If yes, I will post another revision.
Regards,
ChaoQuoted 21 lines
Quoted 21 lines
arch/riscv/kvm/vcpu.c +2
-0+ +return 0 ; }static void kvm_riscv_vcpu_setup_config (struct kvm_vcpu *vcpu)arch/riscv/kvm/vm.c +1
-1@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_READONLY_MEM :case KVM_CAP_MP_STATE :case KVM_CAP_IMMEDIATE_EXIT : +case KVM_CAP_SET_GUEST_DEBUG : r =1 ;break ;case KVM_CAP_NR_VCPUS : --2.17.1
Quoted 4 lines
Quoted 4 lines
--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()
Quoted 89 lines · Anup Patel
Quoted 87 lines · On Wed, Feb 21, 2024 at 7:28?AM Chao Du <[email protected]> wrote:
Quoted 84 lines · Anup Patel
Quoted 70 lines · On Tue, Feb 20, 2024 at 8:31?AM Chao Du <[email protected]> wrote:
Quoted 64 lines · Anup Patel
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:
kvm_vm_ioctl_check_extension(): Return 1 ifKVM_CAP_SET_GUEST_DEBUGis being checked.
kvm_arch_vcpu_ioctl_set_guest_debug(): Update theguest_debugflags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.Signed-off-by Chao Du <[email protected]>--- arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu.c | 15 +++++++++++++-- arch/riscv/kvm/vm.c | 1 + 3 files changed, 15 insertions(+), 2 deletions(-)arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -17,6 +17,7 @@#define __KVM_HAVE_IRQ_LINE#define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_GUEST_DEBUG#define KVM_COALESCED_MMIO_PAGE_OFFSET 1arch/riscv/kvm/vcpu.c +1
-2@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,int kvm_arch_vcpu_ioctl_set_guest_debug (struct kvm_vcpu *vcpu,struct kvm_guest_debug *dbg) { -/* TODO; To be implemented later. */ -return -EINVAL ; +if (dbg->control &KVM_GUESTDBG_ENABLE ) {arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488
Quoted 10 lines
arch/riscv/kvm/vcpu.c +10
-0+if (vcpu->guest_debug != dbg->control) { + vcpu->guest_debug = dbg->control; +csr_clear (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }else { +if (vcpu->guest_debug !=0 ) { + vcpu->guest_debug =0 ; +csr_set (CSR_HEDELEG ,BIT (EXC_BREAKPOINT )); + } + }This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.
To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "structkvm_vcpu_config" which is pre-initialized inkvm_riscv_vcpu_setup_config()without settingEXC_BREAKPOINTbit.
2) Thekvm_arch_vcpu_ioctl_set_guest_debug()should only set/clearEXC_BREAKPOINTbit in "hedeleg" of "structkvm_vcpu_config".
3) Thekvm_riscv_vcpu_swap_in_guest_state()must write the HEDELEG csr before entering the Guest/VM.Quoted 3 lines
Regards,
AnupQuoted 8 lines
Thanks for the review and detailed suggestion.
Maybe we could make it a bit easier:
1) Thekvm_arch_vcpu_ioctl_set_guest_debug()only updatevcpu->guest_debugaccordingly.
2) Thekvm_riscv_vcpu_swap_in_guest_state()checkvcpu->guest_debugand set/clear the HEDELEG csr accordingly.Could you confirm if this is OK?
Your suggestion will work but it adds an additional "if ()" check in
kvm_riscv_vcpu_swap_in_guest_state()which is in the hot path.
Quoted 7 lines
Quoted 2 lines
Yes, it makes sense.
I will prepare a V2 patch.I just realized that
kvm_arch_vcpu_load()will be a much better place to add hedeleg CSR write instead ofkvm_riscv_vcpu_swap_in_guest_state(). This way it will be only at the start hot-path (aka run-loop).
OK, I'm also considering doing so.
Quoted 47 lines
Regards,
AnupQuoted 43 lines
Thanks,
ChaoQuoted 39 lines
I am still leaning towards what I suggested.
Regards,
AnupQuoted 34 lines
If yes, I will post another revision.
Regards,
ChaoQuoted 21 lines
Quoted 21 lines
arch/riscv/kvm/vcpu.c +2
-0+ +return 0 ; }static void kvm_riscv_vcpu_setup_config (struct kvm_vcpu *vcpu)arch/riscv/kvm/vm.c +1
-1@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_READONLY_MEM :case KVM_CAP_MP_STATE :case KVM_CAP_IMMEDIATE_EXIT : +case KVM_CAP_SET_GUEST_DEBUG : r =1 ;break ;case KVM_CAP_NR_VCPUS : --2.17.1
Quoted 4 lines
Quoted 4 lines
--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
PATCH 2/3 RISC-V: KVM: Handle breakpoint exits for VCPU
In reply to Chao DuExit to userspace for breakpoint traps. Set the exit_reason as KVM_EXIT_DEBUG before exit.
--- arch/riscv/kvm/vcpu_exit.c | 4 ++++ 1 file changed, 4 insertions(+)
arch/riscv/kvm/vcpu_exit.c +4 -0
@@ -204,6 +204,10 @@ int kvm_riscv_vcpu_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,if (vcpu->arch.guest_context.hstatus &HSTATUS_SPV ) ret =kvm_riscv_vcpu_sbi_ecall (vcpu, run);break ; +case EXC_BREAKPOINT : + run->exit_reason =KVM_EXIT_DEBUG ; + ret =0 ; +break ;default :break ; }
Signature · 1 line
2.17.1
Re: [PATCH v1 2/3] RISC-V: KVM: Handle breakpoint exits for VCPU
Quoted 5 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:
Exit to userspace for breakpoint traps. Set the
exit_reasonasKVM_EXIT_DEBUGbefore exit.Signed-off-by Chao Du <[email protected]>
Looks good to me.
Regards,
Anup
Quoted 22 lines
--- arch/riscv/kvm/vcpu_exit.c | 4 ++++ 1 file changed, 4 insertions(+)arch/riscv/kvm/vcpu_exit.c +4
-1@@ -204,6 +204,10 @@ int kvm_riscv_vcpu_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,if (vcpu->arch.guest_context.hstatus &HSTATUS_SPV ) ret =kvm_riscv_vcpu_sbi_ecall (vcpu, run);break ; +case EXC_BREAKPOINT : + run->exit_reason =KVM_EXIT_DEBUG ; + ret =0 ; +break ;default :break ; } --2.17.1
PATCH 3/3 RISC-V: KVM: selftests: Add breakpoints test support
In reply to Chao DuInitial support for RISC-V KVM breakpoint test. Check the exit reason and the PC when guest debug is enabled.
--- tools/testing/selftests/kvm/Makefile | 1 + .../testing/selftests/kvm/riscv/breakpoints.c | 49 +++++++++++++++++++ 2 files changed, 50 insertions(+) create mode 100644 tools/testing/selftests/kvm/riscv/breakpoints.c
tools/testing/selftests/kvm/Makefile +1 -0
@@ -184,6 +184,7 @@ TEST_GEN_PROGS_s390x += rseq_test TEST_GEN_PROGS_s390x += set_memory_region_test TEST_GEN_PROGS_s390x += kvm_binary_stats_test +TEST_GEN_PROGS_riscv += riscv/breakpoints TEST_GEN_PROGS_riscv += demand_paging_test TEST_GEN_PROGS_riscv += dirty_log_test TEST_GEN_PROGS_riscv += get-reg-list
tools/testing/selftests/kvm/riscv/breakpoints.c new file +49 -0
49 lines not on this page · raw message
--
2.17.1