Skip to contentSkip to conversation

RISC-V: KVM: Guest Debug Support - Software Breakpoint Part

PATCH Superseded Chao Du 3 participants 11 messages Feb 6, 2024 – Feb 21, 2024

Subsystems (4)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Chao Du

author
#0 lore raw

cover RISC-V: KVM: Guest Debug Support - Software Breakpoint Part

This series implements the ?KVM Guset Debug? feature on RISC-V. This is an existing feature which is already supported by some other arches. It allows us to debug a RISC-V KVM guest from GDB in host side.

As the first stage, the software breakpoints (ebreak instruction) is implemented. HW breakpoints support will come later after a synthetically consideration with the SBI debug trigger extension.

A selftest case was added in this series. Manual test was done on QEMU RISC-V hypervisor emulator. (add '-s' to enable the gdbserver in QEMU)

This series is based on Linux 6.8-rc2 and also available at:
https://github.com/Du-Chao/kvm-riscv/tree/guest_debug_sw

The matched QEMU is available at:
https://github.com/Du-Chao/qemu/tree/riscv_gd_sw

Changes from RFC->v1:
- Rebased on Linux 6.8-rc2.
- Merge PATCH1 and PATCH2 into one patch. (If Paolo's change
https://lore.kernel.org/kvm/20240131233056.10845-8-pbonzini at redhat.com/ is adopted, then we can keep 'arch/riscv/include/uapi/asm/kvm.h' untouched)
- kselftest case added.

RFC link:
https://lore.kernel.org/kvm/20231221095002.7404-1-duchao at eswincomputing.com

Chao Du (3):
RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()
RISC-V: KVM: Handle breakpoint exits for VCPU
RISC-V: KVM: selftests: Add breakpoints test support

 arch/riscv/include/uapi/asm/kvm.h             |  1 +
 arch/riscv/kvm/vcpu.c                         | 15 +++++-
 arch/riscv/kvm/vcpu_exit.c                    |  4 ++
 arch/riscv/kvm/vm.c                           |  1 +
 tools/testing/selftests/kvm/Makefile          |  1 +
 .../testing/selftests/kvm/riscv/breakpoints.c | 49 +++++++++++++++++++
 6 files changed, 69 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/riscv/breakpoints.c

--
2.17.1

author
#1 lore raw

PATCH 1/3 RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()

kvm_vm_ioctl_check_extension(): Return 1 if KVM_CAP_SET_GUEST_DEBUG is being checked.

kvm_arch_vcpu_ioctl_set_guest_debug(): Update the guest_debug flags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.

---
 arch/riscv/include/uapi/asm/kvm.h |  1 +
 arch/riscv/kvm/vcpu.c             | 15 +++++++++++++--
 arch/riscv/kvm/vm.c               |  1 +
 3 files changed, 15 insertions(+), 2 deletions(-)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index d6b7a5b95874..8890977836f0 100644
@@ -17,6 +17,7 @@
 
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_GUEST_DEBUG
 
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 
arch/riscv/kvm/vcpu.c +13 -2
index b5ca9f2e98ac..6cee974592ac 100644
@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
 int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
 					struct kvm_guest_debug *dbg)
 {
-	/* TODO; To be implemented later. */
-	return -EINVAL;
+	if (dbg->control & KVM_GUESTDBG_ENABLE) {
+		if (vcpu->guest_debug != dbg->control) {
+			vcpu->guest_debug = dbg->control;
+			csr_clear(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+		}
+	} else {
+		if (vcpu->guest_debug != 0) {
+			vcpu->guest_debug = 0;
+			csr_set(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+		}
+	}
+
+	return 0;
 }
 
 static void kvm_riscv_vcpu_setup_config(struct kvm_vcpu *vcpu)
arch/riscv/kvm/vm.c +1 -0
index ce58bc48e5b8..7396b8654f45 100644
@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_READONLY_MEM:
 	case KVM_CAP_MP_STATE:
 	case KVM_CAP_IMMEDIATE_EXIT:
+	case KVM_CAP_SET_GUEST_DEBUG:
 		r = 1;
 		break;
 	case KVM_CAP_NR_VCPUS:

--
2.17.1

#2 lore raw

Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()

Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:

kvm_vm_ioctl_check_extension(): Return 1 if KVM_CAP_SET_GUEST_DEBUG is being checked.

kvm_arch_vcpu_ioctl_set_guest_debug(): Update the guest_debug flags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.

---
 arch/riscv/include/uapi/asm/kvm.h |  1 +
 arch/riscv/kvm/vcpu.c             | 15 +++++++++++++--
 arch/riscv/kvm/vm.c               |  1 +
 3 files changed, 15 insertions(+), 2 deletions(-)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index d6b7a5b95874..8890977836f0 100644
@@ -17,6 +17,7 @@
 
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_GUEST_DEBUG
 
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 
arch/riscv/kvm/vcpu.c +1 -2
index b5ca9f2e98ac..6cee974592ac 100644
@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
 int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
                                        struct kvm_guest_debug *dbg)
 {
-       /* TODO; To be implemented later. */
-       return -EINVAL;
+       if (dbg->control & KVM_GUESTDBG_ENABLE) {

arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488

Quoted 10 lines
arch/riscv/kvm/vcpu.c +10 -0
+               if (vcpu->guest_debug != dbg->control) {
+                       vcpu->guest_debug = dbg->control;
+                       csr_clear(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       } else {
+               if (vcpu->guest_debug != 0) {
+                       vcpu->guest_debug = 0;
+                       csr_set(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       }

This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.

To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "struct kvm_vcpu_config" which is pre-initialized in kvm_riscv_vcpu_setup_config() without setting EXC_BREAKPOINT bit.
2) The kvm_arch_vcpu_ioctl_set_guest_debug() should only set/clear EXC_BREAKPOINT bit in "hedeleg" of "struct kvm_vcpu_config".
3) The kvm_riscv_vcpu_swap_in_guest_state() must write the HEDELEG csr before entering the Guest/VM.

Regards,
Anup

Quoted 25 lines
arch/riscv/kvm/vcpu.c +2 -0
+
+       return 0;
 }
 
 static void kvm_riscv_vcpu_setup_config(struct kvm_vcpu *vcpu)
arch/riscv/kvm/vm.c +1 -1
index ce58bc48e5b8..7396b8654f45 100644
@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
        case KVM_CAP_READONLY_MEM:
        case KVM_CAP_MP_STATE:
        case KVM_CAP_IMMEDIATE_EXIT:
+       case KVM_CAP_SET_GUEST_DEBUG:
                r = 1;
                break;
        case KVM_CAP_NR_VCPUS:
--

2.17.1

--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

author
#3 lore raw

Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()

Quoted 64 lines · Anup Patel
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:

kvm_vm_ioctl_check_extension(): Return 1 if KVM_CAP_SET_GUEST_DEBUG is being checked.

kvm_arch_vcpu_ioctl_set_guest_debug(): Update the guest_debug flags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.

---
 arch/riscv/include/uapi/asm/kvm.h |  1 +
 arch/riscv/kvm/vcpu.c             | 15 +++++++++++++--
 arch/riscv/kvm/vm.c               |  1 +
 3 files changed, 15 insertions(+), 2 deletions(-)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index d6b7a5b95874..8890977836f0 100644
@@ -17,6 +17,7 @@
 
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_GUEST_DEBUG
 
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 
arch/riscv/kvm/vcpu.c +1 -2
index b5ca9f2e98ac..6cee974592ac 100644
@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
 int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
                                        struct kvm_guest_debug *dbg)
 {
-       /* TODO; To be implemented later. */
-       return -EINVAL;
+       if (dbg->control & KVM_GUESTDBG_ENABLE) {

arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488

Quoted 10 lines
arch/riscv/kvm/vcpu.c +10 -0
+               if (vcpu->guest_debug != dbg->control) {
+                       vcpu->guest_debug = dbg->control;
+                       csr_clear(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       } else {
+               if (vcpu->guest_debug != 0) {
+                       vcpu->guest_debug = 0;
+                       csr_set(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       }

This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.

To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "struct kvm_vcpu_config" which is pre-initialized in kvm_riscv_vcpu_setup_config() without setting EXC_BREAKPOINT bit.
2) The kvm_arch_vcpu_ioctl_set_guest_debug() should only set/clear EXC_BREAKPOINT bit in "hedeleg" of "struct kvm_vcpu_config".
3) The kvm_riscv_vcpu_swap_in_guest_state() must write the HEDELEG csr before entering the Guest/VM.

Quoted 3 lines

Regards,
Anup

Thanks for the review and detailed suggestion.
Maybe we could make it a bit easier:
1) The kvm_arch_vcpu_ioctl_set_guest_debug() only update vcpu->guest_debug accordingly.
2) The kvm_riscv_vcpu_swap_in_guest_state() check vcpu->guest_debug and set/clear the HEDELEG csr accordingly.

Could you confirm if this is OK?
If yes, I will post another revision.

Regards,
Chao

Quoted 25 lines
Quoted 25 lines
arch/riscv/kvm/vcpu.c +2 -0
+
+       return 0;
 }
 
 static void kvm_riscv_vcpu_setup_config(struct kvm_vcpu *vcpu)
arch/riscv/kvm/vm.c +1 -1
index ce58bc48e5b8..7396b8654f45 100644
@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
        case KVM_CAP_READONLY_MEM:
        case KVM_CAP_MP_STATE:
        case KVM_CAP_IMMEDIATE_EXIT:
+       case KVM_CAP_SET_GUEST_DEBUG:
                r = 1;
                break;
        case KVM_CAP_NR_VCPUS:
--

2.17.1

--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

#4 lore raw

Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()

Quoted 70 lines · On Tue, Feb 20, 2024 at 8:31?AM Chao Du <[email protected]> wrote:
Quoted 64 lines · Anup Patel
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:

kvm_vm_ioctl_check_extension(): Return 1 if KVM_CAP_SET_GUEST_DEBUG is being checked.

kvm_arch_vcpu_ioctl_set_guest_debug(): Update the guest_debug flags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.

---
 arch/riscv/include/uapi/asm/kvm.h |  1 +
 arch/riscv/kvm/vcpu.c             | 15 +++++++++++++--
 arch/riscv/kvm/vm.c               |  1 +
 3 files changed, 15 insertions(+), 2 deletions(-)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index d6b7a5b95874..8890977836f0 100644
@@ -17,6 +17,7 @@
 
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_GUEST_DEBUG
 
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 
arch/riscv/kvm/vcpu.c +1 -2
index b5ca9f2e98ac..6cee974592ac 100644
@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
 int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
                                        struct kvm_guest_debug *dbg)
 {
-       /* TODO; To be implemented later. */
-       return -EINVAL;
+       if (dbg->control & KVM_GUESTDBG_ENABLE) {

arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488

Quoted 10 lines
arch/riscv/kvm/vcpu.c +10 -0
+               if (vcpu->guest_debug != dbg->control) {
+                       vcpu->guest_debug = dbg->control;
+                       csr_clear(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       } else {
+               if (vcpu->guest_debug != 0) {
+                       vcpu->guest_debug = 0;
+                       csr_set(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       }

This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.

To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "struct kvm_vcpu_config" which is pre-initialized in kvm_riscv_vcpu_setup_config() without setting EXC_BREAKPOINT bit.
2) The kvm_arch_vcpu_ioctl_set_guest_debug() should only set/clear EXC_BREAKPOINT bit in "hedeleg" of "struct kvm_vcpu_config".
3) The kvm_riscv_vcpu_swap_in_guest_state() must write the HEDELEG csr before entering the Guest/VM.

Quoted 3 lines

Regards,
Anup

Quoted 8 lines

Thanks for the review and detailed suggestion.
Maybe we could make it a bit easier:
1) The kvm_arch_vcpu_ioctl_set_guest_debug() only update vcpu->guest_debug accordingly.
2) The kvm_riscv_vcpu_swap_in_guest_state() check vcpu->guest_debug and set/clear the HEDELEG csr accordingly.

Could you confirm if this is OK?

Your suggestion will work but it adds an additional "if ()" check in kvm_riscv_vcpu_swap_in_guest_state() which is in the hot path.

I am still leaning towards what I suggested.

Regards,
Anup

Quoted 34 lines

If yes, I will post another revision.

Regards,
Chao

Quoted 21 lines
Quoted 21 lines
arch/riscv/kvm/vcpu.c +2 -0
+
+       return 0;
 }
 
 static void kvm_riscv_vcpu_setup_config(struct kvm_vcpu *vcpu)
arch/riscv/kvm/vm.c +1 -1
index ce58bc48e5b8..7396b8654f45 100644
@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
        case KVM_CAP_READONLY_MEM:
        case KVM_CAP_MP_STATE:
        case KVM_CAP_IMMEDIATE_EXIT:
+       case KVM_CAP_SET_GUEST_DEBUG:
                r = 1;
                break;
        case KVM_CAP_NR_VCPUS:
--

2.17.1

Quoted 4 lines
Quoted 4 lines

--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

author
#5 lore raw

Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()

Quoted 79 lines · Anup Patel
Quoted 77 lines · On Tue, Feb 20, 2024 at 8:31?AM Chao Du <[email protected]> wrote:
Quoted 64 lines · Anup Patel
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:

kvm_vm_ioctl_check_extension(): Return 1 if KVM_CAP_SET_GUEST_DEBUG is being checked.

kvm_arch_vcpu_ioctl_set_guest_debug(): Update the guest_debug flags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.

---
 arch/riscv/include/uapi/asm/kvm.h |  1 +
 arch/riscv/kvm/vcpu.c             | 15 +++++++++++++--
 arch/riscv/kvm/vm.c               |  1 +
 3 files changed, 15 insertions(+), 2 deletions(-)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index d6b7a5b95874..8890977836f0 100644
@@ -17,6 +17,7 @@
 
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_GUEST_DEBUG
 
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 
arch/riscv/kvm/vcpu.c +1 -2
index b5ca9f2e98ac..6cee974592ac 100644
@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
 int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
                                        struct kvm_guest_debug *dbg)
 {
-       /* TODO; To be implemented later. */
-       return -EINVAL;
+       if (dbg->control & KVM_GUESTDBG_ENABLE) {

arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488

Quoted 10 lines
arch/riscv/kvm/vcpu.c +10 -0
+               if (vcpu->guest_debug != dbg->control) {
+                       vcpu->guest_debug = dbg->control;
+                       csr_clear(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       } else {
+               if (vcpu->guest_debug != 0) {
+                       vcpu->guest_debug = 0;
+                       csr_set(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       }

This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.

To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "struct kvm_vcpu_config" which is pre-initialized in kvm_riscv_vcpu_setup_config() without setting EXC_BREAKPOINT bit.
2) The kvm_arch_vcpu_ioctl_set_guest_debug() should only set/clear EXC_BREAKPOINT bit in "hedeleg" of "struct kvm_vcpu_config".
3) The kvm_riscv_vcpu_swap_in_guest_state() must write the HEDELEG csr before entering the Guest/VM.

Quoted 3 lines

Regards,
Anup

Thanks for the review and detailed suggestion.
Maybe we could make it a bit easier:
1) The kvm_arch_vcpu_ioctl_set_guest_debug() only update vcpu->guest_debug accordingly.
2) The kvm_riscv_vcpu_swap_in_guest_state() check vcpu->guest_debug and set/clear the HEDELEG csr accordingly.

Quoted 5 lines
Quoted 1 line

Could you confirm if this is OK?

Your suggestion will work but it adds an additional "if ()" check in kvm_riscv_vcpu_swap_in_guest_state() which is in the hot path.

Yes, it makes sense.
I will prepare a V2 patch.

Thanks,
Chao

Quoted 39 lines

I am still leaning towards what I suggested.

Regards,
Anup

Quoted 34 lines

If yes, I will post another revision.

Regards,
Chao

Quoted 21 lines
Quoted 21 lines
arch/riscv/kvm/vcpu.c +2 -0
+
+       return 0;
 }
 
 static void kvm_riscv_vcpu_setup_config(struct kvm_vcpu *vcpu)
arch/riscv/kvm/vm.c +1 -1
index ce58bc48e5b8..7396b8654f45 100644
@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
        case KVM_CAP_READONLY_MEM:
        case KVM_CAP_MP_STATE:
        case KVM_CAP_IMMEDIATE_EXIT:
+       case KVM_CAP_SET_GUEST_DEBUG:
                r = 1;
                break;
        case KVM_CAP_NR_VCPUS:
--

2.17.1

Quoted 4 lines
Quoted 4 lines

--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

#6 lore raw

Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()

Quoted 81 lines · On Wed, Feb 21, 2024 at 7:28?AM Chao Du <[email protected]> wrote:
Quoted 79 lines · Anup Patel
Quoted 77 lines · On Tue, Feb 20, 2024 at 8:31?AM Chao Du <[email protected]> wrote:
Quoted 64 lines · Anup Patel
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:

kvm_vm_ioctl_check_extension(): Return 1 if KVM_CAP_SET_GUEST_DEBUG is being checked.

kvm_arch_vcpu_ioctl_set_guest_debug(): Update the guest_debug flags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.

---
 arch/riscv/include/uapi/asm/kvm.h |  1 +
 arch/riscv/kvm/vcpu.c             | 15 +++++++++++++--
 arch/riscv/kvm/vm.c               |  1 +
 3 files changed, 15 insertions(+), 2 deletions(-)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index d6b7a5b95874..8890977836f0 100644
@@ -17,6 +17,7 @@
 
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_GUEST_DEBUG
 
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 
arch/riscv/kvm/vcpu.c +1 -2
index b5ca9f2e98ac..6cee974592ac 100644
@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
 int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
                                        struct kvm_guest_debug *dbg)
 {
-       /* TODO; To be implemented later. */
-       return -EINVAL;
+       if (dbg->control & KVM_GUESTDBG_ENABLE) {

arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488

Quoted 10 lines
arch/riscv/kvm/vcpu.c +10 -0
+               if (vcpu->guest_debug != dbg->control) {
+                       vcpu->guest_debug = dbg->control;
+                       csr_clear(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       } else {
+               if (vcpu->guest_debug != 0) {
+                       vcpu->guest_debug = 0;
+                       csr_set(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       }

This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.

To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "struct kvm_vcpu_config" which is pre-initialized in kvm_riscv_vcpu_setup_config() without setting EXC_BREAKPOINT bit.
2) The kvm_arch_vcpu_ioctl_set_guest_debug() should only set/clear EXC_BREAKPOINT bit in "hedeleg" of "struct kvm_vcpu_config".
3) The kvm_riscv_vcpu_swap_in_guest_state() must write the HEDELEG csr before entering the Guest/VM.

Quoted 3 lines

Regards,
Anup

Thanks for the review and detailed suggestion.
Maybe we could make it a bit easier:
1) The kvm_arch_vcpu_ioctl_set_guest_debug() only update vcpu->guest_debug accordingly.
2) The kvm_riscv_vcpu_swap_in_guest_state() check vcpu->guest_debug and set/clear the HEDELEG csr accordingly.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Could you confirm if this is OK?

Your suggestion will work but it adds an additional "if ()" check in kvm_riscv_vcpu_swap_in_guest_state() which is in the hot path.

Yes, it makes sense.
I will prepare a V2 patch.

I just realized that kvm_arch_vcpu_load() will be a much better place to add hedeleg CSR write instead of kvm_riscv_vcpu_swap_in_guest_state(). This way it will be only at the start hot-path (aka run-loop).

Regards,
Anup

Quoted 43 lines

Thanks,
Chao

Quoted 39 lines

I am still leaning towards what I suggested.

Regards,
Anup

Quoted 34 lines

If yes, I will post another revision.

Regards,
Chao

Quoted 21 lines
Quoted 21 lines
arch/riscv/kvm/vcpu.c +2 -0
+
+       return 0;
 }
 
 static void kvm_riscv_vcpu_setup_config(struct kvm_vcpu *vcpu)
arch/riscv/kvm/vm.c +1 -1
index ce58bc48e5b8..7396b8654f45 100644
@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
        case KVM_CAP_READONLY_MEM:
        case KVM_CAP_MP_STATE:
        case KVM_CAP_IMMEDIATE_EXIT:
+       case KVM_CAP_SET_GUEST_DEBUG:
                r = 1;
                break;
        case KVM_CAP_NR_VCPUS:
--

2.17.1

Quoted 4 lines
Quoted 4 lines

--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

author
#7 lore raw

Re: [PATCH v1 1/3] RISC-V: KVM: Implement kvm_arch_vcpu_ioctl_set_guest_debug()

Quoted 89 lines · Anup Patel
Quoted 87 lines · On Wed, Feb 21, 2024 at 7:28?AM Chao Du <[email protected]> wrote:
Quoted 84 lines · Anup Patel
Quoted 70 lines · On Tue, Feb 20, 2024 at 8:31?AM Chao Du <[email protected]> wrote:
Quoted 64 lines · Anup Patel
Quoted 39 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:

kvm_vm_ioctl_check_extension(): Return 1 if KVM_CAP_SET_GUEST_DEBUG is being checked.

kvm_arch_vcpu_ioctl_set_guest_debug(): Update the guest_debug flags from userspace accordingly. Route the breakpoint exceptions to HS mode if the VM is being debugged by userspace, by clearing the corresponding bit in hedeleg CSR.

---
 arch/riscv/include/uapi/asm/kvm.h |  1 +
 arch/riscv/kvm/vcpu.c             | 15 +++++++++++++--
 arch/riscv/kvm/vm.c               |  1 +
 3 files changed, 15 insertions(+), 2 deletions(-)
arch/riscv/include/uapi/asm/kvm.h +1 -0
index d6b7a5b95874..8890977836f0 100644
@@ -17,6 +17,7 @@
 
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_GUEST_DEBUG
 
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 
arch/riscv/kvm/vcpu.c +1 -2
index b5ca9f2e98ac..6cee974592ac 100644
@@ -475,8 +475,19 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
 int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
                                        struct kvm_guest_debug *dbg)
 {
-       /* TODO; To be implemented later. */
-       return -EINVAL;
+       if (dbg->control & KVM_GUESTDBG_ENABLE) {

arch/riscv/kvm/vcpu.c int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, · line 488

Quoted 10 lines
arch/riscv/kvm/vcpu.c +10 -0
+               if (vcpu->guest_debug != dbg->control) {
+                       vcpu->guest_debug = dbg->control;
+                       csr_clear(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       } else {
+               if (vcpu->guest_debug != 0) {
+                       vcpu->guest_debug = 0;
+                       csr_set(CSR_HEDELEG, BIT(EXC_BREAKPOINT));
+               }
+       }

This is broken because directly setting breakpoint exception delegation in CSR also affects other VCPUs running on the same host CPU.

To address the above, we should do the following:
1) Add "unsigned long hedeleg" in "struct kvm_vcpu_config" which is pre-initialized in kvm_riscv_vcpu_setup_config() without setting EXC_BREAKPOINT bit.
2) The kvm_arch_vcpu_ioctl_set_guest_debug() should only set/clear EXC_BREAKPOINT bit in "hedeleg" of "struct kvm_vcpu_config".
3) The kvm_riscv_vcpu_swap_in_guest_state() must write the HEDELEG csr before entering the Guest/VM.

Quoted 3 lines

Regards,
Anup

Quoted 8 lines

Thanks for the review and detailed suggestion.
Maybe we could make it a bit easier:
1) The kvm_arch_vcpu_ioctl_set_guest_debug() only update vcpu->guest_debug accordingly.
2) The kvm_riscv_vcpu_swap_in_guest_state() check vcpu->guest_debug and set/clear the HEDELEG csr accordingly.

Could you confirm if this is OK?

Your suggestion will work but it adds an additional "if ()" check in kvm_riscv_vcpu_swap_in_guest_state() which is in the hot path.

Quoted 7 lines
Quoted 2 lines

Yes, it makes sense.
I will prepare a V2 patch.

I just realized that kvm_arch_vcpu_load() will be a much better place to add hedeleg CSR write instead of kvm_riscv_vcpu_swap_in_guest_state(). This way it will be only at the start hot-path (aka run-loop).

OK, I'm also considering doing so.

Quoted 47 lines

Regards,
Anup

Quoted 43 lines

Thanks,
Chao

Quoted 39 lines

I am still leaning towards what I suggested.

Regards,
Anup

Quoted 34 lines

If yes, I will post another revision.

Regards,
Chao

Quoted 21 lines
Quoted 21 lines
arch/riscv/kvm/vcpu.c +2 -0
+
+       return 0;
 }
 
 static void kvm_riscv_vcpu_setup_config(struct kvm_vcpu *vcpu)
arch/riscv/kvm/vm.c +1 -1
index ce58bc48e5b8..7396b8654f45 100644
@@ -186,6 +186,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
        case KVM_CAP_READONLY_MEM:
        case KVM_CAP_MP_STATE:
        case KVM_CAP_IMMEDIATE_EXIT:
+       case KVM_CAP_SET_GUEST_DEBUG:
                r = 1;
                break;
        case KVM_CAP_NR_VCPUS:
--

2.17.1

Quoted 4 lines
Quoted 4 lines

--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

--
kvm-riscv mailing list
kvm-riscv at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

author
#8 lore raw

PATCH 2/3 RISC-V: KVM: Handle breakpoint exits for VCPU

In reply to Chao Du

Exit to userspace for breakpoint traps. Set the exit_reason as KVM_EXIT_DEBUG before exit.

---
 arch/riscv/kvm/vcpu_exit.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/riscv/kvm/vcpu_exit.c +4 -0
index 2415722c01b8..5761f95abb60 100644
@@ -204,6 +204,10 @@ int kvm_riscv_vcpu_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,
 		if (vcpu->arch.guest_context.hstatus & HSTATUS_SPV)
 			ret = kvm_riscv_vcpu_sbi_ecall(vcpu, run);
 		break;
+	case EXC_BREAKPOINT:
+		run->exit_reason = KVM_EXIT_DEBUG;
+		ret = 0;
+		break;
 	default:
 		break;
 	}
Signature · 1 line
2.17.1
maintainer Reviewed-by on v1 2/3
#9 lore raw

Re: [PATCH v1 2/3] RISC-V: KVM: Handle breakpoint exits for VCPU

Quoted 5 lines · On Tue, Feb 6, 2024 at 1:22?PM Chao Du <[email protected]> wrote:

Exit to userspace for breakpoint traps. Set the exit_reason as KVM_EXIT_DEBUG before exit.

Looks good to me.

Regards,
Anup

Quoted 22 lines
---
 arch/riscv/kvm/vcpu_exit.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/riscv/kvm/vcpu_exit.c +4 -1
index 2415722c01b8..5761f95abb60 100644
@@ -204,6 +204,10 @@ int kvm_riscv_vcpu_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,
                if (vcpu->arch.guest_context.hstatus & HSTATUS_SPV)
                        ret = kvm_riscv_vcpu_sbi_ecall(vcpu, run);
                break;
+       case EXC_BREAKPOINT:
+               run->exit_reason = KVM_EXIT_DEBUG;
+               ret = 0;
+               break;
        default:
                break;
        }
--

2.17.1

author
#10 lore raw

PATCH 3/3 RISC-V: KVM: selftests: Add breakpoints test support

In reply to Chao Du

Initial support for RISC-V KVM breakpoint test. Check the exit reason and the PC when guest debug is enabled.

---
 tools/testing/selftests/kvm/Makefile          |  1 +
 .../testing/selftests/kvm/riscv/breakpoints.c | 49 +++++++++++++++++++
 2 files changed, 50 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/riscv/breakpoints.c
tools/testing/selftests/kvm/Makefile +1 -0
index 492e937fab00..5f9048a740b0 100644
@@ -184,6 +184,7 @@ TEST_GEN_PROGS_s390x += rseq_test
 TEST_GEN_PROGS_s390x += set_memory_region_test
 TEST_GEN_PROGS_s390x += kvm_binary_stats_test
 
+TEST_GEN_PROGS_riscv += riscv/breakpoints
 TEST_GEN_PROGS_riscv += demand_paging_test
 TEST_GEN_PROGS_riscv += dirty_log_test
 TEST_GEN_PROGS_riscv += get-reg-list
tools/testing/selftests/kvm/riscv/breakpoints.c new file +49 -0

49 lines not on this page · raw message

--
2.17.1