Skip to contentSkip to conversation

riscv: support for hardware breakpoints/watchpoints

RFC PATCH In review Sergey Matyukevich 3 participants 9 messages Dec 3, 2022 – Dec 5, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v2 cover riscv: support for hardware breakpoints/watchpoints

RISC-V Debug specification includes Sdtrig ISA extension. This extension describes Trigger Module. Triggers can cause a breakpoint exception, entry into Debug Mode, or a trace action without having to execute a special instruction. For native debugging triggers can be used to implement hardware breakpoints and watchpoints.

Software support for triggers consists of the following major components:
- U-mode: gdb support for hw breakpoints/watchpoints
- S-mode: hardware breakpoints framework in Linux kernel
- M-mode: SBI firmware code to handle triggers

SBI Debug Trigger extension proposal has been posted by Anup Patel to lists.riscv.org tech-debug mailing list, see:
https://lists.riscv.org/g/tech-debug/topic/92375492

This patch provides initial Linux support for RISC-V hardware breakpoints and watchpoints based on the proposed SBI Debug Trigger extension. The accompanying OpenSBI and GDB changes has also been posted for review:

- https://patchwork.ozlabs.org/project/opensbi/patch/[email protected]/
- https://patchwork.sourceware.org/project/gdb/patch/[email protected]/

Current revision has the following limitations:
- two trigger types are supported: mcontrol, mcontrol6
- no support for chained triggers
- no support for virtualization
- only build test for RV32

The functionality has been tested on QEMU together with the mentioned opensbi and gdb patches, including both target gdb and remote debug using gdbserver. Hardware breakpoints work just fine on upstream QEMU. However this is not the case for watchpoints since there is no way to figure out which watchpoint triggered. IIUC there are two possible options for doing this: using 'hit' bit in tdata1 or reading faulting virtual address from STVAL. QEMU implements neither of them. Current implementation opts for STVAL. The following experimental QEMU patch is required to make hw-watchpoints work:

: diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c : index 278d163803..8858be7411 100644
: --- a/target/riscv/cpu_helper.c
: +++ b/target/riscv/cpu_helper.c
: @@ -1639,6 +1639,10 @@ void riscv_cpu_do_interrupt(CPUState *cs)
: case RISCV_EXCP_VIRT_INSTRUCTION_FAULT:
: tval = env->bins;
: break;
: + case RISCV_EXCP_BREAKPOINT:

: +            tval = env->badaddr;
: +            env->badaddr = 0x0;
: +            break;

: default:
: break;
: }
: diff --git a/target/riscv/debug.c b/target/riscv/debug.c : index 26ea764407..b4d1d566ab 100644
: --- a/target/riscv/debug.c
: +++ b/target/riscv/debug.c
: @@ -560,6 +560,7 @@ void riscv_cpu_debug_excp_handler(CPUState *cs) :
: if (cs->watchpoint_hit) {
: if (cs->watchpoint_hit->flags & BP_CPU) {

: +            env->badaddr = cs->watchpoint_hit->hitaddr;
:              cs->watchpoint_hit = NULL;
:              do_trigger_action(env, DBG_ACTION_BP);

Changes v1 -> v2:
- switched to per-cpu buffers to exchange data with SBI firmware
- added support for type 2 (mcounter) triggers
- added ptrace interface to expose hw-breakpoints to debuggers

Sergey Matyukevich (3):
riscv: add support for hardware breakpoints/watchpoints
riscv: ptrace: expose hardware breakpoints to debuggers
riscv: hw-breakpoints: add more trigger controls

 arch/riscv/Kconfig                     |   2 +
 arch/riscv/include/asm/hw_breakpoint.h | 172 ++++++++
 arch/riscv/include/asm/kdebug.h        |   3 +-
 arch/riscv/include/asm/processor.h     |   5 +
 arch/riscv/include/asm/sbi.h           |  24 ++
 arch/riscv/include/uapi/asm/ptrace.h   |   9 +
 arch/riscv/kernel/Makefile             |   1 +
 arch/riscv/kernel/hw_breakpoint.c      | 540 +++++++++++++++++++++++++
 arch/riscv/kernel/process.c            |   3 +
 arch/riscv/kernel/ptrace.c             | 188 +++++++++
 arch/riscv/kernel/traps.c              |   5 +
 11 files changed, 951 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/include/asm/hw_breakpoint.h
 create mode 100644 arch/riscv/kernel/hw_breakpoint.c
Signature · 1 line
2.38.1
#1 lore raw

RFC PATCH v2 1/3 riscv: add support for hardware breakpoints/watchpoints

From: Sergey Matyukevich <[email protected]>

RISC-V backend for hw-breakpoint framework is built on top of SBI Debug Trigger extension. Architecture specific hooks are implemented as kernel wrappers around ecalls to SBI functions. This patch implements only a minimal set of hooks required to support user-space debug via ptrace.

---
 arch/riscv/Kconfig                     |   2 +
 arch/riscv/include/asm/hw_breakpoint.h | 157 +++++++++
 arch/riscv/include/asm/kdebug.h        |   3 +-
 arch/riscv/include/asm/processor.h     |   5 +
 arch/riscv/include/asm/sbi.h           |  24 ++
 arch/riscv/kernel/Makefile             |   1 +
 arch/riscv/kernel/hw_breakpoint.c      | 432 +++++++++++++++++++++++++
 arch/riscv/kernel/process.c            |   1 +
 arch/riscv/kernel/traps.c              |   5 +
 9 files changed, 629 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/include/asm/hw_breakpoint.h
 create mode 100644 arch/riscv/kernel/hw_breakpoint.c
arch/riscv/Kconfig +2 -0
index 593cf09264d8..fe7f63928235 100644
@@ -95,10 +95,12 @@ config RISCV
 	select HAVE_FUNCTION_ERROR_INJECTION
 	select HAVE_GCC_PLUGINS
 	select HAVE_GENERIC_VDSO if MMU && 64BIT
+	select HAVE_HW_BREAKPOINT if PERF_EVENTS
 	select HAVE_IRQ_TIME_ACCOUNTING
 	select HAVE_KPROBES if !XIP_KERNEL
 	select HAVE_KPROBES_ON_FTRACE if !XIP_KERNEL
 	select HAVE_KRETPROBES if !XIP_KERNEL
+	select HAVE_MIXED_BREAKPOINTS_REGS
 	select HAVE_MOVE_PMD
 	select HAVE_MOVE_PUD
 	select HAVE_PCI
arch/riscv/include/asm/hw_breakpoint.h new file +157 -0

157 lines not on this page · raw message

arch/riscv/include/asm/kdebug.h +2 -1
index 85ac00411f6e..53e989781aa1 100644
@@ -6,7 +6,8 @@
 enum die_val {
 	DIE_UNUSED,
 	DIE_TRAP,
-	DIE_OOPS
+	DIE_OOPS,
+	DIE_DEBUG
 };
 
 #endif
arch/riscv/include/asm/processor.h +5 -0
index 94a0590c6971..10c87fba2548 100644
@@ -11,6 +11,7 @@
 #include <vdso/processor.h>
 
 #include <asm/ptrace.h>
+#include <asm/hw_breakpoint.h>
 
 /*
  * This decides where the kernel will search for a free chunk of vm
@@ -29,6 +30,7 @@
 #ifndef __ASSEMBLY__
 
 struct task_struct;
+struct perf_event;
 struct pt_regs;
 
 /* CPU-specific state of a task */
@@ -39,6 +41,9 @@ struct thread_struct {
 	unsigned long s[12];	/* s[0]: frame pointer */
 	struct __riscv_d_ext_state fstate;
 	unsigned long bad_cause;
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+	struct perf_event *ptrace_bps[HBP_NUM_MAX];
+#endif
 };
 
 /* Whitelist the fstate from the task_struct for hardened usercopy */
arch/riscv/include/asm/sbi.h +24 -0
index 2a0ef738695e..ef41d60a5ed3 100644
@@ -31,6 +31,9 @@ enum sbi_ext_id {
 	SBI_EXT_SRST = 0x53525354,
 	SBI_EXT_PMU = 0x504D55,
 
+	/* Experimental: Debug Trigger Extension */
+	SBI_EXT_DBTR = 0x44425452,
+
 	/* Experimentals extensions must lie within this range */
 	SBI_EXT_EXPERIMENTAL_START = 0x08000000,
 	SBI_EXT_EXPERIMENTAL_END = 0x08FFFFFF,
@@ -113,6 +116,27 @@ enum sbi_srst_reset_reason {
 	SBI_SRST_RESET_REASON_SYS_FAILURE,
 };
 
+enum sbi_ext_dbtr_fid {
+	SBI_EXT_DBTR_NUM_TRIGGERS = 0,
+	SBI_EXT_DBTR_TRIGGER_READ,
+	SBI_EXT_DBTR_TRIGGER_INSTALL,
+	SBI_EXT_DBTR_TRIGGER_UNINSTALL,
+	SBI_EXT_DBTR_TRIGGER_ENABLE,
+	SBI_EXT_DBTR_TRIGGER_UPDATE,
+	SBI_EXT_DBTR_TRIGGER_DISABLE,
+};
+
+struct sbi_dbtr_data_msg {
+	unsigned long tstate;
+	unsigned long tdata1;
+	unsigned long tdata2;
+	unsigned long tdata3;
+};
+
+struct sbi_dbtr_id_msg {
+	unsigned long idx;
+};
+
 enum sbi_ext_pmu_fid {
 	SBI_EXT_PMU_NUM_COUNTERS = 0,
 	SBI_EXT_PMU_COUNTER_GET_INFO,
arch/riscv/kernel/Makefile +1 -0
index db6e4b1294ba..116697d0ca1d 100644
@@ -72,6 +72,7 @@ obj-$(CONFIG_TRACE_IRQFLAGS)	+= trace_irq.o
 
 obj-$(CONFIG_PERF_EVENTS)	+= perf_callchain.o
 obj-$(CONFIG_HAVE_PERF_REGS)	+= perf_regs.o
+obj-$(CONFIG_HAVE_HW_BREAKPOINT)	+= hw_breakpoint.o
 obj-$(CONFIG_RISCV_SBI)		+= sbi.o
 ifeq ($(CONFIG_RISCV_SBI), y)
 obj-$(CONFIG_SMP) += cpu_ops_sbi.o
arch/riscv/kernel/hw_breakpoint.c new file +432 -0

432 lines not on this page · raw message

arch/riscv/kernel/process.c +1 -0
index 8955f2432c2d..cd99bececed8 100644
@@ -187,5 +187,6 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 		p->thread.ra = (unsigned long)ret_from_fork;
 	}
 	p->thread.sp = (unsigned long)childregs; /* kernel sp */
+	clear_ptrace_hw_breakpoint(p);
 	return 0;
 }
arch/riscv/kernel/traps.c +5 -0
index 7abd8e4c4df6..34c93d2f159e 100644
@@ -174,6 +174,11 @@ asmlinkage __visible __trap_section void do_trap_break(struct pt_regs *regs)
 
 	if (uprobe_breakpoint_handler(regs))
 		return;
+#endif
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+	if (notify_die(DIE_DEBUG, "EBREAK", regs, 0, regs->cause, SIGTRAP)
+						       == NOTIFY_STOP)
+		return;
 #endif
 	current->thread.bad_cause = regs->cause;
 
Signature · 1 line
2.38.1
#2 lore raw

Re: [PATCH RFC v2 1/3] riscv: add support for hardware breakpoints/watchpoints

Quoted 13 lines · Sergey Matyukevich

From: Sergey Matyukevich <[email protected]>

RISC-V backend for hw-breakpoint framework is built on top of SBI Debug Trigger extension. Architecture specific hooks are implemented as kernel wrappers around ecalls to SBI functions. This patch implements only a minimal set of hooks required to support user-space debug via ptrace.

arch/riscv/include/asm/sbi.h +0 -0
index 2a0ef738695e..ef41d60a5ed3 100644

arch/riscv/include/asm/sbi.h enum sbi_ext_id { · line 39

Quoted 9 lines
arch/riscv/include/asm/sbi.h +3 -0
@@ -31,6 +31,9 @@ enum sbi_ext_id {
 	SBI_EXT_SRST = 0x53525354,
 	SBI_EXT_PMU = 0x504D55,
 
+	/* Experimental: Debug Trigger Extension */
+	SBI_EXT_DBTR = 0x44425452,
+
 	/* Experimentals extensions must lie within this range */
 	SBI_EXT_EXPERIMENTAL_START = 0x08000000,
 	SBI_EXT_EXPERIMENTAL_END = 0x08FFFFFF,

This is an RFC for something I know nothing about, so was just scrolling mindlessly... This caught my eye as odd - There's an explicit comment about the range for experimental stuff but you've not used it? I guess there must be some reason for that?

Confused,
Conor.

#3 lore raw

Re: [PATCH RFC v2 1/3] riscv: add support for hardware breakpoints/watchpoints

Quoted 14 lines
Quoted 14 lines

RISC-V backend for hw-breakpoint framework is built on top of SBI Debug Trigger extension. Architecture specific hooks are implemented as kernel wrappers around ecalls to SBI functions. This patch implements only a minimal set of hooks required to support user-space debug via ptrace.

arch/riscv/include/asm/sbi.h +0 -0
index 2a0ef738695e..ef41d60a5ed3 100644
@@ -31,6 +31,9 @@ enum sbi_ext_id {
 	SBI_EXT_SRST = 0x53525354,
 	SBI_EXT_PMU = 0x504D55,
 
Quoted 11 lines

arch/riscv/include/asm/sbi.h enum sbi_ext_id { · line 39

Quoted 6 lines
arch/riscv/include/asm/sbi.h +3 -0
+	/* Experimental: Debug Trigger Extension */
+	SBI_EXT_DBTR = 0x44425452,
+
 	/* Experimentals extensions must lie within this range */
 	SBI_EXT_EXPERIMENTAL_START = 0x08000000,
 	SBI_EXT_EXPERIMENTAL_END = 0x08FFFFFF,

This is an RFC for something I know nothing about, so was just scrolling mindlessly... This caught my eye as odd - There's an explicit comment about the range for experimental stuff but you've not used it? I guess there must be some reason for that?

IIUC it is not so experimental. This SBI extension accompanies the debug spec v1.0 (frozen but not yet ratified). So sooner or later is going to become a part of SBI spec.

I am using EID suggested in the draft v4 for this extension posted at lists.riscv.org tech-debug mailing list, see:
https://lists.riscv.org/g/tech-debug/topic/92375492

Regards,
Sergey

#4 lore raw

RFC PATCH v2 2/3 riscv: ptrace: expose hardware breakpoints to debuggers

In reply to Sergey Matyukevich

From: Sergey Matyukevich <[email protected]>

Implement regset-based ptrace interface that exposes hardware breakpoints to user-space debuggers.

---
 arch/riscv/include/uapi/asm/ptrace.h |   9 ++
 arch/riscv/kernel/process.c          |   2 +
 arch/riscv/kernel/ptrace.c           | 188 +++++++++++++++++++++++++++
 3 files changed, 199 insertions(+)
arch/riscv/include/uapi/asm/ptrace.h +9 -0
index 882547f6bd5c..e7a5e1b9ea58 100644
@@ -77,6 +77,15 @@ union __riscv_fp_state {
 	struct __riscv_q_ext_state q;
 };
 
+struct user_hwdebug_state {
+	__u64     dbg_info;
+	struct {
+		__u64 addr;
+		__u64 type;
+		__u64 len;
+	}       dbg_regs[16];
+};
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _UAPI_ASM_RISCV_PTRACE_H */
arch/riscv/kernel/process.c +2 -0
index cd99bececed8..b66936b02caf 100644
@@ -15,6 +15,7 @@
 #include <linux/tick.h>
 #include <linux/ptrace.h>
 #include <linux/uaccess.h>
+#include <linux/hw_breakpoint.h>
 
 #include <asm/unistd.h>
 #include <asm/processor.h>
@@ -148,6 +149,7 @@ void flush_thread(void)
 	fstate_off(current, task_pt_regs(current));
 	memset(&current->thread.fstate, 0, sizeof(current->thread.fstate));
 #endif
+	flush_ptrace_hw_breakpoint(current);
 }
 
 int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
arch/riscv/kernel/ptrace.c +188 -0

212 lines not on this page · raw message

Signature · 1 line
2.38.1
#5 lore raw

RFC PATCH v2 3/3 riscv: hw-breakpoints: add more trigger controls

In reply to Sergey Matyukevich

From: Sergey Matyukevich <[email protected]>

RISC-V SBI Debug Trigger extension proposal defines multiple functions to control debug triggers. The pair of install/uninstall functions was enough to implement ptrace interface for user-space debug. This patch implements kernel wrappers for start/update/stop SBI functions. The intended users of these control wrappers are kgdb and kernel modules that make use of kernel-wide hardware breakpoints and watchpoints.

---
 arch/riscv/include/asm/hw_breakpoint.h |  15 ++++
 arch/riscv/kernel/hw_breakpoint.c      | 116 ++++++++++++++++++++++++-
 2 files changed, 127 insertions(+), 4 deletions(-)
arch/riscv/include/asm/hw_breakpoint.h +15 -0
index 5bb3b55cd464..afc59f8e034e 100644
@@ -137,6 +137,9 @@ int hw_breakpoint_arch_parse(struct perf_event *bp,
 int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
 				    unsigned long val, void *data);
 
+void arch_enable_hw_breakpoint(struct perf_event *bp);
+void arch_update_hw_breakpoint(struct perf_event *bp);
+void arch_disable_hw_breakpoint(struct perf_event *bp);
 int arch_install_hw_breakpoint(struct perf_event *bp);
 void arch_uninstall_hw_breakpoint(struct perf_event *bp);
 void hw_breakpoint_pmu_read(struct perf_event *bp);
@@ -153,5 +156,17 @@ static inline void clear_ptrace_hw_breakpoint(struct task_struct *tsk)
 {
 }
 
+void arch_enable_hw_breakpoint(struct perf_event *bp)
+{
+}
+
+void arch_update_hw_breakpoint(struct perf_event *bp)
+{
+}
+
+void arch_disable_hw_breakpoint(struct perf_event *bp)
+{
+}
+
 #endif /* CONFIG_HAVE_HW_BREAKPOINT */
 #endif /* __RISCV_HW_BREAKPOINT_H */
arch/riscv/kernel/hw_breakpoint.c +112 -4

173 lines not on this page · raw message

Signature · 1 line
2.38.1
#6 lore raw

Re: [PATCH RFC v2 3/3] riscv: hw-breakpoints: add more trigger controls

Hi Sergey,

Quoted 11 lines · Sergey Matyukevich

From: Sergey Matyukevich <[email protected]>

RISC-V SBI Debug Trigger extension proposal defines multiple functions to control debug triggers. The pair of install/uninstall functions was enough to implement ptrace interface for user-space debug. This patch implements kernel wrappers for start/update/stop SBI functions. The intended users of these control wrappers are kgdb and kernel modules that make use of kernel-wide hardware breakpoints and watchpoints.

<snip>

Quoted 19 lines
arch/riscv/include/asm/hw_breakpoint.h +4 -0
index 5bb3b55cd464..afc59f8e034e 100644
@@ -137,6 +137,9 @@ int hw_breakpoint_arch_parse(struct perf_event *bp,
 int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
                                    unsigned long val, void *data);
 
+void arch_enable_hw_breakpoint(struct perf_event *bp);
+void arch_update_hw_breakpoint(struct perf_event *bp);
+void arch_disable_hw_breakpoint(struct perf_event *bp);
 int arch_install_hw_breakpoint(struct perf_event *bp);
 void arch_uninstall_hw_breakpoint(struct perf_event *bp);
 void hw_breakpoint_pmu_read(struct perf_event *bp);
@@ -153,5 +156,17 @@ static inline void clear_ptrace_hw_breakpoint(struct task_struct *tsk)
 {
 }
 
+void arch_enable_hw_breakpoint(struct perf_event *bp)

arch/riscv/include/asm/hw_breakpoint.h static inline void clear_ptrace_hw_breakpoint(struct task_struct *tsk) · line 169

Quoted 10 lines
arch/riscv/include/asm/hw_breakpoint.h +10 -0
+{
+}
+
+void arch_update_hw_breakpoint(struct perf_event *bp)
+{
+}
+
+void arch_disable_hw_breakpoint(struct perf_event *bp)
+{
+}

I don't see any references to {enable,update,disable}_hw_breakpoint in common kernel code, nor do any other architectures define these functions. Who are the intended users? Do we even need them for kgdb on RISC-V?

Quoted 13 lines
arch/riscv/kernel/hw_breakpoint.c +1 -0
index 8eddf512cd03..ca7df02830c2 100644
@@ -2,6 +2,7 @@
 
 #include <linux/hw_breakpoint.h>
 #include <linux/perf_event.h>
+#include <linux/spinlock.h>
 #include <linux/percpu.h>
 #include <linux/kdebug.h>
 

arch/riscv/kernel/hw_breakpoint.c line 14

Quoted 5 lines
arch/riscv/kernel/hw_breakpoint.c +2 -0
@@ -9,6 +10,8 @@
 
 /* bps/wps currently set on each debug trigger for each cpu */
 static DEFINE_PER_CPU(struct perf_event *, bp_per_reg[HBP_NUM_MAX]);
+static DEFINE_PER_CPU(unsigned long, msg_lock_flags);
+static DEFINE_PER_CPU(raw_spinlock_t, msg_lock);

I'm not sure I understand the point of this per-CPU spinlock. Just disable preemption (and interrupts, if necessary).

Also, arch_{install,uninstall}_hw_breakpoint are already expected to be called from atomic context; is the intention here that they be callable from outside atomic context?

-Andrew

Quoted 167 lines
arch/riscv/kernel/hw_breakpoint.c +109 -5
 static struct sbi_dbtr_data_msg __percpu *sbi_xmit;
 static struct sbi_dbtr_id_msg __percpu *sbi_recv;
@@ -318,6 +321,10 @@ int arch_install_hw_breakpoint(struct perf_event *bp)
        struct perf_event **slot;
        unsigned long idx;
        struct sbiret ret;
+       int err = 0;
+
+       raw_spin_lock_irqsave(this_cpu_ptr(&msg_lock),
+                         *this_cpu_ptr(&msg_lock_flags));
 
        xmit->tdata1 = cpu_to_lle(info->trig_data1.value);
        xmit->tdata2 = cpu_to_lle(info->trig_data2);
@@ -328,25 +335,31 @@ int arch_install_hw_breakpoint(struct perf_event *bp)
                        0, 0, 0);
        if (ret.error) {
                pr_warn("%s: failed to install trigger\n", __func__);
-               return -EIO;
+               err = -EIO;
+               goto done;
        }
 
        idx = lle_to_cpu(recv->idx);
 
        if (idx >= dbtr_total_num) {
                pr_warn("%s: invalid trigger index %lu\n", __func__, idx);
-               return -EINVAL;
+               err = -EINVAL;
+               goto done;
        }
 
        slot = this_cpu_ptr(&bp_per_reg[idx]);
        if (*slot) {
                pr_warn("%s: slot %lu is in use\n", __func__, idx);
-               return -EBUSY;
+               err = -EBUSY;
+               goto done;
        }
 
        *slot = bp;
 
-       return 0;
+done:
+       raw_spin_unlock_irqrestore(this_cpu_ptr(&msg_lock),
+                              *this_cpu_ptr(&msg_lock_flags));
+       return err;
 }
 
 /* atomic: counter->ctx->lock is held */
@@ -375,6 +388,96 @@ void arch_uninstall_hw_breakpoint(struct perf_event *bp)
                pr_warn("%s: failed to uninstall trigger %d\n", __func__, i);
 }
 
+void arch_enable_hw_breakpoint(struct perf_event *bp)
+{
+       struct sbiret ret;
+       int i;
+
+       for (i = 0; i < dbtr_total_num; i++) {
+               struct perf_event **slot = this_cpu_ptr(&bp_per_reg[i]);
+
+               if (*slot == bp)
+                       break;
+       }
+
+       if (i == dbtr_total_num) {
+               pr_warn("%s: unknown breakpoint\n", __func__);
+               return;
+       }
+
+       ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIGGER_ENABLE,
+                       i, 1, 0, 0, 0, 0);
+       if (ret.error) {
+               pr_warn("%s: failed to install trigger %d\n", __func__, i);
+               return;
+       }
+}
+EXPORT_SYMBOL_GPL(arch_enable_hw_breakpoint);
+
+void arch_update_hw_breakpoint(struct perf_event *bp)
+{
+       struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+       struct sbi_dbtr_data_msg *xmit = this_cpu_ptr(sbi_xmit);
+       struct perf_event **slot;
+       struct sbiret ret;
+       int i;
+
+       for (i = 0; i < dbtr_total_num; i++) {
+               slot = this_cpu_ptr(&bp_per_reg[i]);
+
+               if (*slot == bp)
+                       break;
+       }
+
+       if (i == dbtr_total_num) {
+               pr_warn("%s: unknown breakpoint\n", __func__);
+               return;
+       }
+
+       raw_spin_lock_irqsave(this_cpu_ptr(&msg_lock),
+                         *this_cpu_ptr(&msg_lock_flags));
+
+       xmit->tdata1 = cpu_to_lle(info->trig_data1.value);
+       xmit->tdata2 = cpu_to_lle(info->trig_data2);
+       xmit->tdata3 = cpu_to_lle(info->trig_data3);
+
+       ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIGGER_UPDATE,
+                       i, 1, __pa(xmit) >> 4,
+                       0, 0, 0);
+       if (ret.error)
+               pr_warn("%s: failed to update trigger %d\n", __func__, i);
+
+       raw_spin_unlock_irqrestore(this_cpu_ptr(&msg_lock),
+                              *this_cpu_ptr(&msg_lock_flags));
+}
+EXPORT_SYMBOL_GPL(arch_update_hw_breakpoint);
+
+void arch_disable_hw_breakpoint(struct perf_event *bp)
+{
+       struct sbiret ret;
+       int i;
+
+       for (i = 0; i < dbtr_total_num; i++) {
+               struct perf_event **slot = this_cpu_ptr(&bp_per_reg[i]);
+
+               if (*slot == bp)
+                       break;
+       }
+
+       if (i == dbtr_total_num) {
+               pr_warn("%s: unknown breakpoint\n", __func__);
+               return;
+       }
+
+       ret = sbi_ecall(SBI_EXT_DBTR, SBI_EXT_DBTR_TRIGGER_DISABLE,
+                       i, 1, 0, 0, 0, 0);
+       if (ret.error) {
+               pr_warn("%s: failed to uninstall trigger %d\n", __func__, i);
+               return;
+       }
+}
+EXPORT_SYMBOL_GPL(arch_disable_hw_breakpoint);
+
 void hw_breakpoint_pmu_read(struct perf_event *bp)
 {
 }
@@ -406,6 +509,8 @@ void flush_ptrace_hw_breakpoint(struct task_struct *tsk)
 
 static int __init arch_hw_breakpoint_init(void)
 {
+       unsigned int cpu;
+
        sbi_xmit = __alloc_percpu(sizeof(*sbi_xmit), SZ_16);
        if (!sbi_xmit) {
                pr_warn("failed to allocate SBI xmit message buffer\n");
@@ -418,6 +523,9 @@ static int __init arch_hw_breakpoint_init(void)
                return -ENOMEM;
        }
 
+       for_each_possible_cpu(cpu)
+               raw_spin_lock_init(&per_cpu(msg_lock, cpu));
+
        if (!dbtr_init)
                arch_hw_breakpoint_init_sbi();
 
--
Quoted 2 lines

2.38.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#7 lore raw

Re: [PATCH RFC v2 3/3] riscv: hw-breakpoints: add more trigger controls

Hi Andrew,

Quoted 35 lines

Hi Sergey,

Quoted 11 lines · Sergey Matyukevich

From: Sergey Matyukevich <[email protected]>

RISC-V SBI Debug Trigger extension proposal defines multiple functions to control debug triggers. The pair of install/uninstall functions was enough to implement ptrace interface for user-space debug. This patch implements kernel wrappers for start/update/stop SBI functions. The intended users of these control wrappers are kgdb and kernel modules that make use of kernel-wide hardware breakpoints and watchpoints.

<snip>

Quoted 18 lines
arch/riscv/include/asm/hw_breakpoint.h +3 -0
index 5bb3b55cd464..afc59f8e034e 100644
@@ -137,6 +137,9 @@ int hw_breakpoint_arch_parse(struct perf_event *bp,
 int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
                                    unsigned long val, void *data);
 
+void arch_enable_hw_breakpoint(struct perf_event *bp);
+void arch_update_hw_breakpoint(struct perf_event *bp);
+void arch_disable_hw_breakpoint(struct perf_event *bp);
 int arch_install_hw_breakpoint(struct perf_event *bp);
 void arch_uninstall_hw_breakpoint(struct perf_event *bp);
 void hw_breakpoint_pmu_read(struct perf_event *bp);
@@ -153,5 +156,17 @@ static inline void clear_ptrace_hw_breakpoint(struct task_struct *tsk)
 {
 }
 
Quoted 16 lines

arch/riscv/include/asm/hw_breakpoint.h static inline void clear_ptrace_hw_breakpoint(struct task_struct *tsk) · line 169

Quoted 11 lines
arch/riscv/include/asm/hw_breakpoint.h +11 -0
+void arch_enable_hw_breakpoint(struct perf_event *bp)
+{
+}
+
+void arch_update_hw_breakpoint(struct perf_event *bp)
+{
+}
+
+void arch_disable_hw_breakpoint(struct perf_event *bp)
+{
+}

I don't see any references to {enable,update,disable}_hw_breakpoint in common kernel code, nor do any other architectures define these functions. Who are the intended users? Do we even need them for kgdb on RISC-V?

SBI Debug Trigger extension proposal defines more functions than just install/uninstall required for ptrace hw-breakpoints API. So this patch is an attempt to expose some additional SBI features to the rest of the kernel.

For instance, I have been using these stop/update/start functions for managing kernel-wide hw-breakpoints when experimenting with a sample module from samples/hw_breakpoint. It can be convenient to modify a breakpoint or watchpoint when it fires, e.g. to perform single-step before proceeding execution. Full-fledged unregister/register cycle is not suitable for this task. And this is where disable/update/enable sequence can help.

I haven't yet tried to implement hw-breakpoint support in RISC-V kgdb, so I don't know for sure if these custom calls are needed there.

Quoted 22 lines
Quoted 13 lines
arch/riscv/kernel/hw_breakpoint.c +1 -0
index 8eddf512cd03..ca7df02830c2 100644
@@ -2,6 +2,7 @@
 
 #include <linux/hw_breakpoint.h>
 #include <linux/perf_event.h>
+#include <linux/spinlock.h>
 #include <linux/percpu.h>
 #include <linux/kdebug.h>
 

arch/riscv/kernel/hw_breakpoint.c line 14

Quoted 5 lines
arch/riscv/kernel/hw_breakpoint.c +2 -0
@@ -9,6 +10,8 @@
 
 /* bps/wps currently set on each debug trigger for each cpu */
 static DEFINE_PER_CPU(struct perf_event *, bp_per_reg[HBP_NUM_MAX]);
+static DEFINE_PER_CPU(unsigned long, msg_lock_flags);
+static DEFINE_PER_CPU(raw_spinlock_t, msg_lock);

I'm not sure I understand the point of this per-CPU spinlock. Just disable preemption (and interrupts, if necessary).

Quoted 3 lines

Also, arch_{install,uninstall}_hw_breakpoint are already expected to be called from atomic context; is the intention here that they be callable from outside atomic context?

These additional locsk are not needed for install/uninstall pair due to the reason that you mentioned: those calls are called by kernel event core in atomic context with ctx->lock held. These locks are needed only to make sure that new 'arch_update_hw_breakpoint' function can use the same message buffers as install/uninstall.

Regards,
Sergey

#8 lore raw

Re: [PATCH RFC v2 3/3] riscv: hw-breakpoints: add more trigger controls

Quoted 56 lines · Sergey Matyukevich

Hi Andrew,

Quoted 35 lines

Hi Sergey,

Quoted 11 lines · Sergey Matyukevich

From: Sergey Matyukevich <[email protected]>

RISC-V SBI Debug Trigger extension proposal defines multiple functions to control debug triggers. The pair of install/uninstall functions was enough to implement ptrace interface for user-space debug. This patch implements kernel wrappers for start/update/stop SBI functions. The intended users of these control wrappers are kgdb and kernel modules that make use of kernel-wide hardware breakpoints and watchpoints.

<snip>

Quoted 18 lines
arch/riscv/include/asm/hw_breakpoint.h +3 -0
index 5bb3b55cd464..afc59f8e034e 100644
@@ -137,6 +137,9 @@ int hw_breakpoint_arch_parse(struct perf_event *bp,
 int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
                                    unsigned long val, void *data);
 
+void arch_enable_hw_breakpoint(struct perf_event *bp);
+void arch_update_hw_breakpoint(struct perf_event *bp);
+void arch_disable_hw_breakpoint(struct perf_event *bp);
 int arch_install_hw_breakpoint(struct perf_event *bp);
 void arch_uninstall_hw_breakpoint(struct perf_event *bp);
 void hw_breakpoint_pmu_read(struct perf_event *bp);
@@ -153,5 +156,17 @@ static inline void clear_ptrace_hw_breakpoint(struct task_struct *tsk)
 {
 }
 
Quoted 16 lines

arch/riscv/include/asm/hw_breakpoint.h static inline void clear_ptrace_hw_breakpoint(struct task_struct *tsk) · line 169

Quoted 11 lines
arch/riscv/include/asm/hw_breakpoint.h +11 -0
+void arch_enable_hw_breakpoint(struct perf_event *bp)
+{
+}
+
+void arch_update_hw_breakpoint(struct perf_event *bp)
+{
+}
+
+void arch_disable_hw_breakpoint(struct perf_event *bp)
+{
+}

I don't see any references to {enable,update,disable}_hw_breakpoint in common kernel code, nor do any other architectures define these functions. Who are the intended users? Do we even need them for kgdb on RISC-V?

Quoted 4 lines

SBI Debug Trigger extension proposal defines more functions than just install/uninstall required for ptrace hw-breakpoints API. So this patch is an attempt to expose some additional SBI features to the rest of the kernel.

Got it. IMHO it's generally best not to add dead code unless a user for it is imminent. It's harder to review something if you don't know exactly how it'll be used.

Quoted 37 lines

For instance, I have been using these stop/update/start functions for managing kernel-wide hw-breakpoints when experimenting with a sample module from samples/hw_breakpoint. It can be convenient to modify a breakpoint or watchpoint when it fires, e.g. to perform single-step before proceeding execution. Full-fledged unregister/register cycle is not suitable for this task. And this is where disable/update/enable sequence can help.

I haven't yet tried to implement hw-breakpoint support in RISC-V kgdb, so I don't know for sure if these custom calls are needed there.

Quoted 22 lines
Quoted 13 lines
arch/riscv/kernel/hw_breakpoint.c +1 -0
index 8eddf512cd03..ca7df02830c2 100644
@@ -2,6 +2,7 @@
 
 #include <linux/hw_breakpoint.h>
 #include <linux/perf_event.h>
+#include <linux/spinlock.h>
 #include <linux/percpu.h>
 #include <linux/kdebug.h>
 

arch/riscv/kernel/hw_breakpoint.c line 14

Quoted 5 lines
arch/riscv/kernel/hw_breakpoint.c +2 -0
@@ -9,6 +10,8 @@
 
 /* bps/wps currently set on each debug trigger for each cpu */
 static DEFINE_PER_CPU(struct perf_event *, bp_per_reg[HBP_NUM_MAX]);
+static DEFINE_PER_CPU(unsigned long, msg_lock_flags);
+static DEFINE_PER_CPU(raw_spinlock_t, msg_lock);

I'm not sure I understand the point of this per-CPU spinlock. Just disable preemption (and interrupts, if necessary).

Quoted 3 lines

Also, arch_{install,uninstall}_hw_breakpoint are already expected to be called from atomic context; is the intention here that they be callable from outside atomic context?

Quoted 5 lines

These additional locsk are not needed for install/uninstall pair due to the reason that you mentioned: those calls are called by kernel event core in atomic context with ctx->lock held. These locks are needed only to make sure that new 'arch_update_hw_breakpoint' function can use the same message buffers as install/uninstall.

Sure, but you can achieve the same by disabling preemption. No need for an actual spinlock.

-Andrew

Quoted 3 lines

Regards,
Sergey