ARM: add support for hw-breakpoints [v2]
Subsystems (3)
cover ARM: add support for hw-breakpoints [v2]
This is version 2 of the RFC originally posted here:
http://lists.infradead.org/pipermail/linux-arm-kernel/2010-February/009084.html
Changes from the previous version include:
- Based on 2.6.33
- Removal of self-modifying code to access the debug registers
- Control and value registers now reset during init
- Removal of .name field from struct arch_hw_breakpoint and associated symbol resolution [following comments made about patches for PPC].
All comments welcome [the last submission didn't attract any!].
Will Deacon (4):
ARM: hw-breakpoint: add mechanism for hooking into prefetch aborts
ARM: hw-breakpoint: add ARM backend for the hw-breakpoint framework
ARM: hw-breakpoint: add new ptrace requests for hw-breakpoint
interaction
ARM: hw-breakpoint: add HAVE_HW_BREAKPOINT to Kconfig
arch/arm/Kconfig | 1 + arch/arm/include/asm/hw_breakpoint.h | 118 ++++++ arch/arm/include/asm/processor.h | 4 + arch/arm/include/asm/ptrace.h | 2 + arch/arm/include/asm/system.h | 3 + arch/arm/kernel/Makefile | 1 + arch/arm/kernel/hw_breakpoint.c | 711 ++++++++++++++++++++++++++++++++++ arch/arm/kernel/ptrace.c | 158 ++++++++ arch/arm/mm/fault.c | 11 + 9 files changed, 1009 insertions(+), 0 deletions(-) create mode 100644 arch/arm/include/asm/hw_breakpoint.h create mode 100644 arch/arm/kernel/hw_breakpoint.c
RFC PATCH 1/4 ARM: hw-breakpoint: add mechanism for hooking into prefetch aborts
On ARM processors with hardware breakpoint and watchpoint support, triggering these events results in a debug exception. These manifest as prefetch and data aborts respectively.
arch/arm/mm/fault.c already provides hook_fault_code for hooking into data aborts dependent on the DFSR. This patch adds a new function, hook_ifault_code for hooking into prefetch aborts in the same manner.
--- arch/arm/include/asm/system.h | 3 +++ arch/arm/mm/fault.c | 11 +++++++++++ 2 files changed, 14 insertions(+), 0 deletions(-)
arch/arm/include/asm/system.h +3 -0
@@ -83,6 +83,9 @@ void arm_notify_die(const char *str, struct pt_regs *regs, struct siginfo *info,void hook_fault_code (int nr,int (*fn)(unsigned long ,unsigned int ,struct pt_regs *),int sig,const char *name); +void hook_ifault_code (int nr,int (*fn)(unsigned long ,unsigned int , +struct pt_regs *), +int sig,const char *name);#define xchg (ptr,x) \ ((__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr))))
arch/arm/mm/fault.c +11 -0
@@ -567,6 +567,17 @@ static struct fsr_info ifsr_info[] = { { do_bad, SIGBUS, 0, "unknown 31" }, }; +void __init +hook_ifault_code (int nr,int (*fn)(unsigned long ,unsigned int ,struct pt_regs *), +int sig,const char *name) +{ +if (nr >=0 && nr <ARRAY_SIZE (ifsr_info)) { + ifsr_info[nr].fn = fn; + ifsr_info[nr].sig = sig; + ifsr_info[nr].name = name; + } +} + asmlinkage void __exceptiondo_PrefetchAbort (unsigned long addr,unsigned int ifsr,struct pt_regs *regs) {
Signature · 1 line
1.6.3.3
RFC PATCH 2/4 ARM: hw-breakpoint: add ARM backend for the hw-breakpoint framework
The hw-breakpoint framework in the kernel requires architecture-specific support in order to install, remove, validate and manage hardware breakpoints.
This patch adds preliminary support for this framework to the ARM architecture, but restricts the number of watchpoints to a single resource to get around the fact that the Data Fault Address Register is unpredictable when a watchpoint debug exception is taken.
Additionally, the memory-mapped extended debug interface is unsupported due to its unreliability in real implementations.
--- arch/arm/include/asm/hw_breakpoint.h | 118 ++++++ arch/arm/kernel/hw_breakpoint.c | 711 ++++++++++++++++++++++++++++++++++ 2 files changed, 829 insertions(+), 0 deletions(-) create mode 100644 arch/arm/include/asm/hw_breakpoint.h create mode 100644 arch/arm/kernel/hw_breakpoint.c
arch/arm/include/asm/hw_breakpoint.h new file +118 -0
@@ -0,0 +1,118 @@ +#ifndef _ARM_HW_BREAKPOINT_H +#define _ARM_HW_BREAKPOINT_H + +#ifdef __KERNEL__ +struct arch_hw_breakpoint { +u32 address; +u32 trigger; +u8 len; +u8 type; +u8 privilege; +}; + +/* Debug architecture numbers. */ +#define ARM_DEBUG_ARCH_V6 1 +#define ARM_DEBUG_ARCH_V6_1 2 +#define ARM_DEBUG_ARCH_V7_ECP14 3 +#define ARM_DEBUG_ARCH_V7_MM 4 + +/* Breakpoint */ +#define ARM_BREAKPOINT_EXECUTE 0 + +/* Watchpoints */ +#define ARM_BREAKPOINT_LOAD 1 +#define ARM_BREAKPOINT_STORE 2 + +/* Privilege Levels */ +#define ARM_BREAKPOINT_SUPER 1 +#define ARM_BREAKPOINT_USER 2 + +/* Lengths */ +#define ARM_BREAKPOINT_LEN_1 0x1 +#define ARM_BREAKPOINT_LEN_2 0x3 +#define ARM_BREAKPOINT_LEN_4 0xf +#define ARM_BREAKPOINT_LEN_8 0xff + +/* Limits */ +#define ARM_MAX_BRP 16 +#define ARM_MAX_WRP 16 +#define HBP_NUM (ARM_MAX_BRP + ARM_MAX_WRP) + +/* DSCR method of entry bits. */ +#define ARM_DSCR_MOE (x) ((x >> 2) & 0xf) +#define ARM_ENTRY_BREAKPOINT 0x1 +#define ARM_ENTRY_ASYNC_WATCHPOINT 0x2 +#define ARM_ENTRY_SYNC_WATCHPOINT 0xa + +/* DSCR monitor/halting bits. */ +#define ARM_DSCR_HDBGEN (1 << 14) +#define ARM_DSCR_MDBGEN (1 << 15) + +/* opcode2 numbers for the co-processor instructions. */ +#define ARM_OP2_BVR 4 +#define ARM_OP2_BCR 5 +#define ARM_OP2_WVR 6 +#define ARM_OP2_WCR 7 + +/* Base register numbers for the debug registers. */ +#define ARM_BASE_BVR 64 +#define ARM_BASE_BCR 80 +#define ARM_BASE_WVR 96 +#define ARM_BASE_WCR 112 + +/* Accessor macros for the debug registers. */ +#define ARM_DBG_READ (M ,OP2 ,VAL ) do {\ + asm volatile("mrc p14, 0, %0, c0," #M ", " #OP2 : "=r" (VAL));\ +} while (0) + +#define ARM_DBG_WRITE (M ,OP2 ,VAL ) do {\ + asm volatile("mcr p14, 0, %0, c0," #M ", " #OP2 : : "r" (VAL));\ +} while (0) + +/* ptrace interface macros and constants. +* +* command layout: +* +* 31 30 29 28 11 10 7 6 3 2 0 +* +-------+--+-------+-----+-----+----+ +* |Ver[00]|Op| SBZ | Num | Len |Type| +* +-------+--+-------+-----+-----+----+ +* +* Op: 1 => insert, 0 => remove. +* Num: index into hbp field for thread_struct debug. +* Len: HW_BREAKPOINT_LEN_{1,2,4,8}. +* Type: HW_BREAKPOINT_{R,W,X}. +*/ +#define PTRACE_HWBREAK_VER (x) ((x >> 30) & 3) +#define PTRACE_HWBREAK_OP (x) ((x >> 29) & 1) +#define PTRACE_HWBREAK_NUM (x) ((x >> 7) & 0xf) +#define PTRACE_HWBREAK_LEN (x) ((x >> 3) & 0xf) +#define PTRACE_HWBREAK_TYPE (x) (x & 7) + +#define PTRACE_HWBREAK_OP_REMOVE 0 +#define PTRACE_HWBREAK_OP_INSERT 1 + +#define PTRACE_HWBREAK_GET_NUM_BRPS 0 +#define PTRACE_HWBREAK_GET_NUM_WRPS 1 + +struct notifier_block ; +struct perf_event ; +struct pmu ; +struct task_struct ; + +extern struct pmu perf_ops_bp; +extern int arch_check_va_in_userspace (unsigned long va,u8 hbp_len); +extern int arch_validate_hwbkpt_settings (struct perf_event *bp, +struct task_struct *tsk); +extern int hw_breakpoint_exceptions_notify (struct notifier_block *unused, +unsigned long val,void *data); + +int arch_install_hw_breakpoint (struct perf_event *bp); +void arch_uninstall_hw_breakpoint (struct perf_event *bp); +void hw_breakpoint_pmu_read (struct perf_event *bp); +void hw_breakpoint_pmu_unthrottle (struct perf_event *bp); + +unsigned int ptrace_get_num_brps (void ); +unsigned int ptrace_get_num_wrps (void ); +#endif /* __KERNEL__ */ +#endif /* _ARM_HW_BREAKPOINT_H */
arch/arm/kernel/hw_breakpoint.c new file +711 -0
711 lines not on this page · raw message
Signature · 1 line
1.6.3.3
RFC PATCH 3/4 ARM: hw-breakpoint: add new ptrace requests for hw-breakpoint interaction
For debuggers to take advantage of the hw-breakpoint framework in the kernel, it is necessary to expose the API calls via a ptrace interface.
This patch adds support for a debugger to insert, modify and remove hardware breakpoints using PTRACE_SET_HWBKPT requests. The breakpoints are stored in the debug_info struct of the running thread. A further request, PTRACE_GET_HWBKPT can be used to query the number of hardware resources available.
The ptrace interaction is controlled using a 32-bit word, as described in arch/arm/include/asm/hw_breakpoint.h.
--- arch/arm/include/asm/processor.h | 4 + arch/arm/include/asm/ptrace.h | 2 + arch/arm/kernel/ptrace.c | 158 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 164 insertions(+), 0 deletions(-)
arch/arm/include/asm/processor.h +4 -0
@@ -19,6 +19,7 @@#ifdef __KERNEL__ +#include <asm/hw_breakpoint.h> #include <asm/ptrace.h> #include <asm/types.h>
@@ -41,6 +42,9 @@ struct debug_entry {struct debug_info {int nsaved;struct debug_entry bp[2 ]; +#ifdef CONFIG_HAVE_HW_BREAKPOINT +struct perf_event *hbp[HBP_NUM ]; +#endif };struct thread_struct {
arch/arm/include/asm/ptrace.h +2 -0
@@ -29,6 +29,8 @@#define PTRACE_SETCRUNCHREGS 26#define PTRACE_GETVFPREGS 27#define PTRACE_SETVFPREGS 28 +#define PTRACE_GET_HWBKPT 29 +#define PTRACE_SET_HWBKPT 30/* * PSR bits
arch/arm/kernel/ptrace.c +158 -0
176 lines not on this page · raw message
Signature · 1 line
1.6.3.3
RFC PATCH 4/4 ARM: hw-breakpoint: add HAVE_HW_BREAKPOINT to Kconfig
If we're targetting a v6 or v7 core and have at least software perf events available, then automatically add support for hardware breakpoints.
--- arch/arm/Kconfig | 1 + arch/arm/kernel/Makefile | 1 + 2 files changed, 2 insertions(+), 0 deletions(-)
arch/arm/Kconfig +1 -0
@@ -23,6 +23,7 @@ config ARM select HAVE_PERF_EVENTS select PERF_USE_VMALLOC select GENERIC_ATOMIC64 if (!CPU_32v6K) + select HAVE_HW_BREAKPOINT if (PERF_EVENTS && (CPU_V6 || CPU_V7)) help The ARM series is a line of low-power-consumption RISC chip designs licensed by ARM Ltd and targeted at embedded applications and
arch/arm/kernel/Makefile +1 -0
@@ -38,6 +38,7 @@ obj-$(CONFIG_ARM_THUMBEE) += thumbee.o obj-$(CONFIG_KGDB) += kgdb.o obj-$(CONFIG_ARM_UNWIND) += unwind.o obj-$(CONFIG_HAVE_TCM) += tcm.o +obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o obj-$(CONFIG_CRUNCH) += crunch.o crunch-bits.o AFLAGS_crunch-bits.o := -Wa,-mcpu=ep9312
Signature · 1 line
1.6.3.3
Re: [RFC PATCH 2/4] ARM: hw-breakpoint: add ARM backend for the hw-breakpoint framework
In reply to Will Deacon(WARNING: I've browsed the ARMv7 breakpoints implementation but I may have an erratic/incomplete understanding, then parts of this review might make little sense)
2010/3/10 Will Deacon <[email protected]>:
Quoted 8 lines
The hw-breakpoint framework in the kernel requires architecture-specific support in order to install, remove, validate and manage hardware breakpoints.
This patch adds preliminary support for this framework to the ARM architecture, but restricts the number of watchpoints to a single resource to get around the fact that the Data Fault Address Register is unpredictable when a watchpoint debug exception is taken.
What do you mean here by unpredictable? It would be a pity to limit the resources to one register.
arch/arm/kernel/hw_breakpoint.c line 68
Quoted 9 lines
arch/arm/kernel/hw_breakpoint.c new file +9
-0+} + +/* Determine number of WRP registers available. */ +static int get_num_wrps (void ) +{ +/* +* FIXME: When a watchpoint fires, the only way to work out which +* watchpoint it was is by disassembling the faulting instruction +* and working out the address of the memory access.
Doh! That must explain the problem with DFAR...
That's really not convenient :-(
arch/arm/kernel/hw_breakpoint.c line 79
Quoted 11 lines
arch/arm/kernel/hw_breakpoint.c new file +11
-0+* +* Furthermore, we can only do this if the watchpoint was precise +* since imprecise watchpoints prevent us from calculating register +* based addresses. +* +* For the time being, we only report 1 watchpoint register so we +* always know which watchpoint fired. In the future we can either +* add a disassembler and address generation emulator, or we can +* insert a check to see if the DFAR is set on watchpoint exception +* entry [the ARM ARM states that the DFAR is UNPREDICTABLE, but +* experience shows that it is set on some implementations].
Ok...
Quoted 38 lines
arch/arm/kernel/hw_breakpoint.c new file +38
-0+/* +* Install a perf counter breakpoint. +*/ +int arch_install_hw_breakpoint (struct perf_event *bp) +{ +struct arch_hw_breakpoint *info =counter_arch_bp (bp); +struct perf_event **slot, **slots; +int i, max_slots, ctrl_base, val_base, ret =0 ; + +/* Ensure that we are in monitor mode and halting mode is disabled. */ + ret =enable_monitor_mode (); +if (ret) + goto out; + +if (info->type ==ARM_BREAKPOINT_EXECUTE ) { +/* Breakpoint */ + ctrl_base =ARM_BASE_BCR ; + val_base =ARM_BASE_BVR ; + slots =__get_cpu_var (bp_on_reg); + max_slots = core_num_brps; + }else { +/* Watchpoint */ + ctrl_base =ARM_BASE_WCR ; + val_base =ARM_BASE_WVR ; + slots =__get_cpu_var (wp_on_reg); + max_slots = core_num_wrps; + } + +for (i =0 ; i < max_slots; ++i) { + slot = &slots[i]; + +if (!*slot) { + *slot = bp; +break ; + } + } + +if (WARN_ONCE (i == max_slots,"Can't find any breakpoint slot" )) {
arch/arm/kernel/hw_breakpoint.c line 311
Quoted 10 lines
arch/arm/kernel/hw_breakpoint.c new file +10
-0+ ret = -EBUSY ; + goto out; + } + +/* Setup the address register. */ +write_wb_reg (val_base + i, info->address); + +/* Setup the control register. */ +write_wb_reg (ctrl_base + i, (info->type <<3 ) | + (info->privilege <<1 ) | (info->len <<5 ) |1 );
Ok, alternatively, why not having the control register in the arch breakpoint structure:
u32 __reserved : 19,
len : 7,
type : 2,
privilege : 2,
enabled : 1;
It will avoid you to play with bitwise operations that may scale into dirty and error prone as you may support more features from the control register (link, mask, etc...).
So in the future if you want to support more things, you can just split out the __reserved field into the features it has, depending on the ARM versions.
Ah and it will make ptrace support easier: the user writes into the val/ctrl fields directly as if they were the true registers, then you can just validate the fields you want without bitwise ops.
Quoted 25 lines
arch/arm/kernel/hw_breakpoint.c new file +25
-0+/* +* Validate the arch-specific HW Breakpoint register settings. +*/ +int arch_validate_hwbkpt_settings (struct perf_event *bp, +struct task_struct *tsk) +{ +struct arch_hw_breakpoint *info =counter_arch_bp (bp); +int ret =0 ; +u32 alignment_mask =0x3 ; + +/* Build the arch_hw_breakpoint. */ + ret =arch_build_bp_info (bp); +if (ret) + goto out; + +/* Check address alignment. */ +if (info->len ==ARM_BREAKPOINT_LEN_8 ) + alignment_mask =0x7 ; +if (info->address & alignment_mask) { + ret = -EINVAL ; + goto out; + } + +/* Check that the virtual address is in the proper range. */ +if (tsk) {
arch/arm/kernel/hw_breakpoint.c line 503
Quoted 10 lines
arch/arm/kernel/hw_breakpoint.c new file +10
-0+if (!(info->privilege &ARM_BREAKPOINT_USER )) { + ret = -EFAULT ; + goto out; + } + }else { +if (!(info->privilege &ARM_BREAKPOINT_SUPER )) { + ret = -EFAULT ; + goto out; + } + }
You seem to make a wrong assumption here.
This is not because the breakpoint is task-bound that we don't want it to trigger on the kernel. We may want to trigger breakpoints on tasklist_lock accesses from a given task.
The only case for which we don't want it to trigger on the kernel is for ptrace breakpoints.
I guess I should remove this tsk parameter as it makes the things only confusing. We should simply create ptrace breakpoints with bp->attr.exclude_kernel set to 1.
Because when bp->attr.exclude_kernel is set to 1, then it truly makes sense to think about user and supervisor privileges, just to avoid to filter the event in the software level. Note we do this filtering on software level, but it going to be less costly if done from hardware.
Even if you don't support ptrace right now, we can define a kernel exluded breakpoint from perf syscall.
(Note for myself: set exclude_kernel to ptrace breakpoints in x86)
arch/arm/kernel/hw_breakpoint.c line 521
Quoted 13 lines
arch/arm/kernel/hw_breakpoint.c new file +13
-0+/* +* Release the user breakpoints used by ptrace. +*/ +void flush_ptrace_hw_breakpoint (struct task_struct *tsk) +{ +int i; +struct thread_struct *t = &tsk->thread; + +for (i =0 ; i <HBP_NUM ; i++) { +unregister_hw_breakpoint (t->debug.hbp[i]); + t->debug.hbp[i] =NULL ; + } +}
Do you actually support ptrace breakpoints?
arch/arm/kernel/hw_breakpoint.c line 702
Quoted 5 lines
arch/arm/kernel/hw_breakpoint.c new file +5
-0+ +void hw_breakpoint_pmu_unthrottle (struct perf_event *bp) +{ +/* TODO */ +}
We don't need this callback anymore, it has been removed because of ptrace corner cases...
Thanks!
Re: [RFC PATCH 2/4] ARM: hw-breakpoint: add ARM backend for the hw-breakpoint framework
Hi Frederic,
Many thanks for taking a look at the these patches, I appreciate the feedback.
Quoted 4 lines · Frederic Weisbecker
(WARNING: I've browsed the ARMv7 breakpoints implementation but I may have an erratic/incomplete understanding, then parts of this review might make little sense)
It looks like you've understood it correctly. Actually, I have a disclaimer of my own; I'm using a new mail client in the hope that LKML won't drop my messages. Anyway...
Quoted 6 lines
2010/3/10 Will Deacon <[email protected]>:
Quoted 4 lines
The hw-breakpoint framework in the kernel requires architecture-specific support in order to install, remove, validate and manage hardware breakpoints.
Quoted 8 lines
Quoted 4 lines
This patch adds preliminary support for this framework to the ARM architecture, but restricts the number of watchpoints to a single resource to get around the fact that the Data Fault Address Register is unpredictable when a watchpoint debug exception is taken.
What do you mean here by unpredictable? It would be a pity to limit the resources to one register.
By unpredictable I mean that the value in the DFAR is not defined to correspond to the causative address in any way. This isn't the case for a standard data abort, but it is in the case of a watchpoint exception.
Quoted 14 lines
arch/arm/kernel/hw_breakpoint.c line 68
Quoted 10 lines
arch/arm/kernel/hw_breakpoint.c new file +9
-0+} + +/* Determine number of WRP registers available. */ +static int get_num_wrps (void ) +{ +/* +* FIXME: When a watchpoint fires, the only way to work out which +* watchpoint it was is by disassembling the faulting instruction +* and working out the address of the memory access. Doh! That must explain the problem with DFAR...
That's really not convenient :-(
I know, it makes life a lot harder for us. The most annoying thing is that I'm yet to find a real implementation that *doesn't* set the DFAR to what we want - we just can't rely on that!
Quoted 1 line
<snip>
Quoted 16 lines
arch/arm/kernel/hw_breakpoint.c line 311
Quoted 6 lines
arch/arm/kernel/hw_breakpoint.c new file +6
-0+/* Setup the address register. */ +write_wb_reg (val_base + i, info->address); + +/* Setup the control register. */ +write_wb_reg (ctrl_base + i, (info->type <<3 ) | + (info->privilege <<1 ) | (info->len <<5 ) |1 );Ok, alternatively, why not having the control register in the arch breakpoint structure:
u32
__reserved: 19,
len : 7,
type : 2,
privilege : 2,
enabled : 1;
Quoted 4 lines
It will avoid you to play with bitwise operations that may scale into dirty and error prone as you may support more features from the control register (link, mask, etc...).
Nice, I like that a lot!
Quoted 4 lines
Ah and it will make ptrace support easier: the user writes into the val/ctrl fields directly as if they were the true registers, then you can just validate the fields you want without bitwise ops.
I'm unsure about this. As mainline stands, ARM has no ptrace support for the hardware breakpoint registers. This means that we could expose the hardware resources in an idealised fashion via ptrace so that if the interface varies between CPUs, userspace doesn't need to care. I had a crack at this with another patch in the series here:
http://lists.infradead.org/pipermail/linux-arm-kernel/2010-March/011169.html
Quoted 14 lines
arch/arm/kernel/hw_breakpoint.c line 503
Quoted 12 lines
arch/arm/kernel/hw_breakpoint.c new file +12
-0+/* Check that the virtual address is in the proper range. */ +if (tsk) { +if (!(info->privilege &ARM_BREAKPOINT_USER )) { + ret = -EFAULT ; + goto out; + } + }else { +if (!(info->privilege &ARM_BREAKPOINT_SUPER )) { + ret = -EFAULT ; + goto out; + } + }
Quoted 8 lines
You seem to make a wrong assumption here.
This is not because the breakpoint is task-bound that we don't want it to trigger on the kernel. We may want to trigger breakpoints ontasklist_lockaccesses from a given task.The only case for which we don't want it to trigger on the kernel is for ptrace breakpoints.
Surely we can't have breakpoints triggering on *any* part of the breakpoint handling code path? Otherwise we'll just get stuck. That's why I disallow breakpoints in the exception section.
Quoted 3 lines
I guess I should remove this tsk parameter as it makes the things only confusing. We should simply create ptrace breakpoints with
bp->attr.exclude_kernelset to 1.
Sounds good to me. That also means that it's one less thing for the arch to worry about!
Quoted 18 lines
Quoted 4 lines
arch/arm/kernel/hw_breakpoint.c new file +3
-0+/* +* Release the user breakpoints used by ptrace. +*/ arch/arm/kernel/hw_breakpoint.c line 521
Quoted 10 lines
arch/arm/kernel/hw_breakpoint.c new file +10
-0+void flush_ptrace_hw_breakpoint (struct task_struct *tsk) +{ +int i; +struct thread_struct *t = &tsk->thread; + +for (i =0 ; i <HBP_NUM ; i++) { +unregister_hw_breakpoint (t->debug.hbp[i]); + t->debug.hbp[i] =NULL ; + } +}Do you actually support ptrace breakpoints?
Yep. See the link mentioned above.
Quoted 11 lines
arch/arm/kernel/hw_breakpoint.c line 702
Quoted 6 lines
arch/arm/kernel/hw_breakpoint.c new file +5
-0+ +void hw_breakpoint_pmu_unthrottle (struct perf_event *bp) +{ +/* TODO */ +}We don't need this callback anymore, it has been removed because of ptrace corner cases...
Ok, I'll remove it.
I'll take a look at the new constraints and allocation patches you posted this morning and then adjust these patches accordingly.
Thanks,
Will
Re: [RFC PATCH 2/4] ARM: hw-breakpoint: add ARM backend for the hw-breakpoint framework
Quoted 8 lines · Will Deacon
Quoted 4 lines · Frederic Weisbecker
(WARNING: I've browsed the ARMv7 breakpoints implementation but I may have an erratic/incomplete understanding, then parts of this review might make little sense)
It looks like you've understood it correctly. Actually, I have a disclaimer of my own; I'm using a new mail client in the hope that LKML won't drop my messages. Anyway...
Looks like that worked as well :)
Quoted 10 lines
Quoted 10 lines
2010/3/10 Will Deacon <[email protected]>:
Quoted 8 lines
The hw-breakpoint framework in the kernel requires architecture-specific support in order to install, remove, validate and manage hardware breakpoints.
This patch adds preliminary support for this framework to the ARM architecture, but restricts the number of watchpoints to a single resource to get around the fact that the Data Fault Address Register is unpredictable when a watchpoint debug exception is taken.
Quoted 7 lines
Quoted 3 lines
What do you mean here by unpredictable? It would be a pity to limit the resources to one register.
By unpredictable I mean that the value in the DFAR is not defined to correspond to the causative address in any way. This isn't the case for a standard data abort, but it is in the case of a watchpoint exception.
Ok.
Quoted 14 lines
Quoted 14 lines
arch/arm/kernel/hw_breakpoint.c line 68
Quoted 10 lines
arch/arm/kernel/hw_breakpoint.c new file +9
-0+} + +/* Determine number of WRP registers available. */ +static int get_num_wrps (void ) +{ +/* +* FIXME: When a watchpoint fires, the only way to work out which +* watchpoint it was is by disassembling the faulting instruction +* and working out the address of the memory access. Doh! That must explain the problem with DFAR...
That's really not convenient :-(
Quoted 3 lines
I know, it makes life a lot harder for us. The most annoying thing is that I'm yet to find a real implementation that *doesn't* set the DFAR to what we want - we just can't rely on that!
You mean that sometimes DFAR doesn't have the true ip origin of the exception?
May be you can check the trapped regs to find the instruction pointer that caused the exception?
Quoted 11 lines
Quoted 4 lines
Ah and it will make ptrace support easier: the user writes into the val/ctrl fields directly as if they were the true registers, then you can just validate the fields you want without bitwise ops.
I'm unsure about this. As mainline stands, ARM has no ptrace support for the hardware breakpoint registers. This means that we could expose the hardware resources in an idealised fashion via ptrace so that if the interface varies between CPUs, userspace doesn't need to care. I had a crack at this with another patch in the series here:
http://lists.infradead.org/pipermail/linux-arm-kernel/2010-March/011169.html
Ah, indeed if you need to abstract out various ARM versions, that's quite sensitive. I'm going to look at this thread too.
Quoted 11 lines
Quoted 8 lines
You seem to make a wrong assumption here.
This is not because the breakpoint is task-bound that we don't want it to trigger on the kernel. We may want to trigger breakpoints ontasklist_lockaccesses from a given task.The only case for which we don't want it to trigger on the kernel is for ptrace breakpoints.
Surely we can't have breakpoints triggering on *any* part of the breakpoint handling code path? Otherwise we'll just get stuck. That's why I disallow breakpoints in the exception section.
Ah indeed you really need to protect against recursion.
But I'm unclear about the difference between Supervisor and System. May be System means the common kernel ring? And you enter into Supervisor when an exception triggers?
Do these exceptions also concern page faults and not only debug exceptions?
Quoted 6 lines
Quoted 3 lines
I guess I should remove this tsk parameter as it makes the things only confusing. We should simply create ptrace breakpoints with
bp->attr.exclude_kernelset to 1.Sounds good to me. That also means that it's one less thing for the arch to worry about!
Yeah!
Quoted 2 lines
I'll take a look at the new constraints and allocation patches you posted this morning and then adjust these patches accordingly.
Thanks.
Re: [RFC PATCH 2/4] ARM: hw-breakpoint: add ARM backend for the hw-breakpoint framework
Hello Frederic,
Quoted 11 lines · Frederic Weisbecker
Quoted 11 lines
Quoted 11 lines
arch/arm/kernel/hw_breakpoint.c line 68
Quoted 7 lines
arch/arm/kernel/hw_breakpoint.c new file +7
-0+/* Determine number of WRP registers available. */ +static int get_num_wrps (void ) +{ +/* +* FIXME: When a watchpoint fires, the only way to work out which +* watchpoint it was is by disassembling the faulting instruction +* and working out the address of the memory access. Doh! That must explain the problem with DFAR...
That's really not convenient :-(
Quoted 8 lines
Quoted 3 lines
I know, it makes life a lot harder for us. The most annoying thing is that I'm yet to find a real implementation that *doesn't* set the DFAR to what we want - we just can't rely on that!
You mean that sometimes DFAR doesn't have the true ip origin of the exception?
May be you can check the trapped regs to find the instruction pointer that caused the exception?
There are two related issues here:
1.) A watchpoint can be synchronous or asynchronous, you can determine which when
you take the exception. If the watchpoint is synchronous, then yes, we can look at the trapped regs to find the causative instruction. Then we would need to disassemble the instruction and emulate the address generation part [which may also need the trapped regs] in order to calculate the faulting data address. If the watchpoint exception is asynchronous, we can read the WFAR register to find the address of the causative instruction. Unfortunately, we could still be screwed in this case because we won't be able to emulate the address calculation for a register-relative load/store. Thankfully, v7 cores only generate synchronous watchpoints [apart from some very early revisions which you won't see in the wild] but v6 cores are the opposite; they only generate asynchronous watchpoint exceptions.
2.) For a standard data abort, the DFAR is updated to contain the data address which
caused the fault. Ideally this would also be the case for watchpoint exceptions because then we wouldn't have to disassemble anything [and as I mentioned previously, I'm yet to find a real implementation that doesn't set the DFAR on taking a watchpoint, but it's not something that the architecture guarantees].
Quoted 10 lines
Quoted 10 lines
Quoted 4 lines
Ah and it will make ptrace support easier: the user writes into the val/ctrl fields directly as if they were the true registers, then you can just validate the fields you want without bitwise ops.
I'm unsure about this. As mainline stands, ARM has no ptrace support for the hardware breakpoint registers. This means that we could expose the hardware resources in an idealised fashion via ptrace so that if the interface varies between CPUs, userspace doesn't need to care. I had a crack at this with another patch in the series here:
Quoted 6 lines
Ah, indeed if you need to abstract out various ARM versions, that's quite sensitive. I'm going to look at this thread too.
It's not so much about necessity, but I think it would be nice if userspace didn't have to care so much about the breakpoint/watchpoint debug hardware on the target. Since we don't have any backwards compatibility issues [because ptrace in mainline has never supported hardware breakpoints on ARM] we can define a stable API regardless of the hardware.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
You seem to make a wrong assumption here.
This is not because the breakpoint is task-bound that we don't want it to trigger on the kernel. We may want to trigger breakpoints ontasklist_lockaccesses from a given task.The only case for which we don't want it to trigger on the kernel is for ptrace breakpoints.
Quoted 7 lines
Quoted 3 lines
Surely we can't have breakpoints triggering on *any* part of the breakpoint handling code path? Otherwise we'll just get stuck. That's why I disallow breakpoints in the exception section.
Ah indeed you really need to protect against recursion.
I'm unsure about how to provide full protection though. You could still set a breakpoint on the breakpoint handling code that exists outside of the exception text. Do other archs suffer from this problem? I'm tempted just to disallow Kernel breakpoints for now, but that would be a shame.
Quoted 3 lines
But I'm unclear about the difference between Supervisor and System. May be System means the common kernel ring? And you enter into Supervisor when an exception triggers?
Sorry, I think my ARM_BREAKPOINT_SUPER #define is confusing. I'll rename it to ARM_BREAKPOINT_PRIV because the hardware only distinguishes between privileged and unprivileged accesses, regardless of the mode.
Quoted 2 lines
Do these exceptions also concern page faults and not only debug exceptions?
I think so.
One thing that has hit me is the behaviour of the framework when a breakpoint is hit. If the breakpoint is synchronous, after the exception has been handled [and perf_bp_event has been called], execution will restart at the faulting instruction. It is down to the client program [traditionally a debugger] to remove the breakpoint, single-step and re-insert the breakpoint. Obviously the perf tool doesn't do this, so if you try to perf stat a breakpoint event you will livelock [potentially another reason to disable Kernel breakpoints?]
Cheers,
Will
Re: [RFC PATCH 2/4] ARM: hw-breakpoint: add ARM backend for the hw-breakpoint framework
Quoted 38 lines · Will Deacon
Hello Frederic,
Quoted 11 lines · Frederic Weisbecker
Quoted 11 lines
Quoted 11 lines
arch/arm/kernel/hw_breakpoint.c line 68
Quoted 7 lines
arch/arm/kernel/hw_breakpoint.c new file +7
-0+/* Determine number of WRP registers available. */ +static int get_num_wrps (void ) +{ +/* +* FIXME: When a watchpoint fires, the only way to work out which +* watchpoint it was is by disassembling the faulting instruction +* and working out the address of the memory access. Doh! That must explain the problem with DFAR...
That's really not convenient :-(Quoted 8 lines
Quoted 3 lines
I know, it makes life a lot harder for us. The most annoying thing is that I'm yet to find a real implementation that *doesn't* set the DFAR to what we want - we just can't rely on that!
You mean that sometimes DFAR doesn't have the true ip origin of the exception?
May be you can check the trapped regs to find the instruction pointer that caused the exception?There are two related issues here:
1.) A watchpoint can be synchronous or asynchronous, you can determine which when
you take the exception. If the watchpoint is synchronous, then yes, we can look at the trapped regs to find the causative instruction. Then we would need to disassemble the instruction and emulate the address generation part [which may also need the trapped regs] in order to calculate the faulting data address. If the watchpoint exception is asynchronous, we can read the WFAR register to find the address of the causative instruction. Unfortunately, we could still be screwed in this case because we won't be able to emulate the address calculation for a register-relative load/store. Thankfully, v7 cores only generate synchronous watchpoints [apart from some very early revisions which you won't see in the wild] but v6 cores are the opposite; they only generate asynchronous watchpoint exceptions.
Quoted 5 lines
2.) For a standard data abort, the DFAR is updated to contain the data address which
caused the fault. Ideally this would also be the case for watchpoint exceptions because then we wouldn't have to disassemble anything [and as I mentioned previously, I'm yet to find a real implementation that doesn't set the DFAR on taking a watchpoint, but it's not something that the architecture guarantees].
It's a cruel architecture...
Quoted 14 lines
Quoted 14 lines
Quoted 11 lines
Quoted 4 lines
Ah and it will make ptrace support easier: the user writes into the val/ctrl fields directly as if they were the true registers, then you can just validate the fields you want without bitwise ops.
I'm unsure about this. As mainline stands, ARM has no ptrace support for the hardware breakpoint registers. This means that we could expose the hardware resources in an idealised fashion via ptrace so that if the interface varies between CPUs, userspace doesn't need to care. I had a crack at this with another patch in the series here:
http://lists.infradead.org/pipermail/linux-arm-kernel/2010-March/011169.html
Quoted 8 lines
Quoted 2 lines
Ah, indeed if you need to abstract out various ARM versions, that's quite sensitive. I'm going to look at this thread too.
It's not so much about necessity, but I think it would be nice if userspace didn't have to care so much about the breakpoint/watchpoint debug hardware on the target. Since we don't have any backwards compatibility issues [because ptrace in mainline has never supported hardware breakpoints on ARM] we can define a stable API regardless of the hardware.
Yeah, looking at how this is implemented across versions, it seems that they all share the same registers, but early versions have unfeatured reserved bytes in control registers that are filled with new features over time in subsequent versions.
It looks like the things are then forward compatible but not backward.
Isn't that enough to provide a ptrace interface? I mean you can't do much magic here. Having something that works everywhere would consist in implementing only the minimal set of features only.
Quoted 14 lines
Quoted 14 lines
Quoted 11 lines
Quoted 8 lines
You seem to make a wrong assumption here.
This is not because the breakpoint is task-bound that we don't want it to trigger on the kernel. We may want to trigger breakpoints ontasklist_lockaccesses from a given task.The only case for which we don't want it to trigger on the kernel is for ptrace breakpoints.
Surely we can't have breakpoints triggering on *any* part of the breakpoint handling code path? Otherwise we'll just get stuck. That's why I disallow breakpoints in the exception section.
Quoted 6 lines
Quoted 1 line
Ah indeed you really need to protect against recursion.
I'm unsure about how to provide full protection though. You could still set a breakpoint on the breakpoint handling code that exists outside of the exception text. Do other archs suffer from this problem? I'm tempted just to disallow Kernel breakpoints for now, but that would be a shame.
In x86 we just write the control register to disable the breakpoints in the exception handler. (I suspect we don't do it early enough though).
You can do the same in ARM.
Currently, there is no true security flow because if a breakpoint is task bound, x86 refuse breakpoints in kernel addresses. But as I noticed before, this is a wrong behaviour as we also want to be able to trace kernel var accesses while in a given task context, except for ptrace.
I should just add a CAP_SYS_ADMIN() && !ptrace check for kernel addresses.
Quoted 7 lines
Quoted 3 lines
But I'm unclear about the difference between Supervisor and System. May be System means the common kernel ring? And you enter into Supervisor when an exception triggers?
Sorry, I think my
ARM_BREAKPOINT_SUPER#define is confusing. I'll rename it toARM_BREAKPOINT_PRIVbecause the hardware only distinguishes between privileged and unprivileged accesses, regardless of the mode.
Ok.
Quoted 5 lines
Quoted 2 lines
Do these exceptions also concern page faults and not only debug exceptions?
I think so.
Quoted 7 lines
One thing that has hit me is the behaviour of the framework when a breakpoint is hit. If the breakpoint is synchronous, after the exception has been handled [and
perf_bp_eventhas been called], execution will restart at the faulting instruction. It is down to the client program [traditionally a debugger] to remove the breakpoint, single-step and re-insert the breakpoint. Obviously the perf tool doesn't do this, so if you try to perf stat a breakpoint event you will livelock [potentially another reason to disable Kernel breakpoints?]
If I understand you correctly,
sync = breakpoint triggers before the instruction is executed,
trap return to the faulting instruction
async = breakpoint triggers after the instruction is executed
Then yeah, the current breakpoint framework only supports async ones.
If an arch has no choice but using sync breakpoints, it needs to handle that. It seems PowerPc has a similar problem, may be have a look at Prasad's patches.
If there is something that can be handled from the generic layer to help solving this problem, I'll try something.
Re: [RFC PATCH 2/4] ARM: hw-breakpoint: add ARM backend for the hw-breakpoint framework
Hi Frederic,
Thanks for addressing some of these problems.
Quoted 23 lines
Quoted 15 lines
There are two related issues here:
1.) A watchpoint can be synchronous or asynchronous, you can determine which when
you take the exception. If the watchpoint is synchronous, then yes, we can look at the trapped regs to find the causative instruction. Then we would need to disassemble the instruction and emulate the address generation part [which may also need the trapped regs] in order to calculate the faulting data address. If the watchpoint exception is asynchronous, we can read the WFAR register to find the address of the causative instruction. Unfortunately, we could still be screwed in this case because we won't be able to emulate the address calculation for a register-relative load/store. Thankfully, v7 cores only generate synchronous watchpoints [apart from some very early revisions which you won't see in the wild] but v6 cores are the opposite; they only generate asynchronous watchpoint exceptions.Quoted 5 lines
2.) For a standard data abort, the DFAR is updated to contain the data address which
caused the fault. Ideally this would also be the case for watchpoint exceptions because then we wouldn't have to disassemble anything [and as I mentioned previously, I'm yet to find a real implementation that doesn't set the DFAR on taking a watchpoint, but it's not something that the architecture guarantees].
Quoted 2 lines
It's a cruel architecture...
It grows on you :)
Quoted 29 lines
Quoted 14 lines
Quoted 14 lines
Quoted 11 lines
Quoted 4 lines
Ah and it will make ptrace support easier: the user writes into the val/ctrl fields directly as if they were the true registers, then you can just validate the fields you want without bitwise ops.
I'm unsure about this. As mainline stands, ARM has no ptrace support for the hardware breakpoint registers. This means that we could expose the hardware resources in an idealised fashion via ptrace so that if the interface varies between CPUs, userspace doesn't need to care. I had a crack at this with another patch in the series here:
http://lists.infradead.org/pipermail/linux-arm-kernel/2010-March/011169.html
Quoted 8 lines
Quoted 2 lines
Ah, indeed if you need to abstract out various ARM versions, that's quite sensitive. I'm going to look at this thread too.
It's not so much about necessity, but I think it would be nice if userspace didn't have to care so much about the breakpoint/watchpoint debug hardware on the target. Since we don't have any backwards compatibility issues [because ptrace in mainline has never supported hardware breakpoints on ARM] we can define a stable API regardless of the hardware.
Yeah, looking at how this is implemented across versions, it seems that they all share the same registers, but early versions have unfeatured reserved bytes in control registers that are filled with new features over time in subsequent versions.
Quoted 6 lines
It looks like the things are then forward compatible but not backward.
Isn't that enough to provide a ptrace interface? I mean you can't do much magic here. Having something that works everywhere would consist in implementing only the minimal set of features only.
I suppose the ptrace interface will involve bit-packing at some point anyway, so mirroring the hardware configuration might not be as bad as I initially thought. The hardware has some slightly more esoteric features [breakpoint linking, context matching etc] that won't be supported in software for now so I'll ignore those bits.
Quoted 10 lines
Quoted 4 lines
I'm unsure about how to provide full protection though. You could still set a breakpoint on the breakpoint handling code that exists outside of the exception text. Do other archs suffer from this problem? I'm tempted just to disallow Kernel breakpoints for now, but that would be a shame.
In x86 we just write the control register to disable the breakpoints in the exception handler. (I suspect we don't do it early enough though).
You can do the same in ARM.
Ok.
Quoted 9 lines
Quoted 7 lines
One thing that has hit me is the behaviour of the framework when a breakpoint is hit. If the breakpoint is synchronous, after the exception has been handled [and
perf_bp_eventhas been called], execution will restart at the faulting instruction. It is down to the client program [traditionally a debugger] to remove the breakpoint, single-step and re-insert the breakpoint. Obviously the perf tool doesn't do this, so if you try to perf stat a breakpoint event you will livelock [potentially another reason to disable Kernel breakpoints?]
Quoted 5 lines
If I understand you correctly,
sync = breakpoint triggers before the instruction is executed,
trap return to the faulting instruction
async = breakpoint triggers after the instruction is executed
That's right.
Quoted 2 lines
Then yeah, the current breakpoint framework only supports async ones.
Oh no!
Quoted 3 lines
If an arch has no choice but using sync breakpoints, it needs to handle that. It seems PowerPc has a similar problem, may be have a look at Prasad's patches.
Could you give me a pointer to these patches please [I can't see them on LKML]? I *think* PowerPC is lucky enough to have hardware single step, so handling synchronous breakpoints isn't *too* difficult. In ARM we have to work out the address of the following instruction, which is quite tricky because it could be a branch instruction [from a selection of instruction sets] and also be predicated. GDB can handle this, but the Kernel doesn't have the capability to do so [there's some code in arch/arm/ptrace.c but it's not feature complete].
Quoted 2 lines
If there is something that can be handled from the generic layer to help solving this problem, I'll try something.
Unfortunately this is very much an arch problem. As a half-way house, I might implement only the ptrace part of hw-breakpoint for ARM. That way we can force the client to handle stepping over the breakpoint and we also won't have to worry about recursive breakpoints in the Kernel.
Cheers,
Will
Re: [RFC PATCH 2/4] ARM: hw-breakpoint: add ARM backend for the hw-breakpoint framework
Quoted 5 lines · Will Deacon
Hi Frederic,
Quoted 2 lines
It's a cruel architecture...
It grows on you :)
:-)))
Quoted 8 lines
Quoted 8 lines
Yeah, looking at how this is implemented across versions, it seems that they all share the same registers, but early versions have unfeatured reserved bytes in control registers that are filled with new features over time in subsequent versions.
It looks like the things are then forward compatible but not backward.
Quoted 8 lines
Quoted 3 lines
Isn't that enough to provide a ptrace interface? I mean you can't do much magic here. Having something that works everywhere would consist in implementing only the minimal set of features only.
I suppose the ptrace interface will involve bit-packing at some point anyway, so mirroring the hardware configuration might not be as bad as I initially thought. The hardware has some slightly more esoteric features [breakpoint linking, context matching etc] that won't be supported in software for now so I'll ignore those bits.
In fact, I guess watchpoint to breakpoint linking is something that userspace can use. But the context ID linking....well I haven't understood this part in ARM documentation :)
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
I'm unsure about how to provide full protection though. You could still set a breakpoint on the breakpoint handling code that exists outside of the exception text. Do other archs suffer from this problem? I'm tempted just to disallow Kernel breakpoints for now, but that would be a shame.
Quoted 6 lines
Quoted 4 lines
In x86 we just write the control register to disable the breakpoints in the exception handler. (I suspect we don't do it early enough though).
You can do the same in ARM.
Ok.
Speaking about this made me wonder about the current state in x86. For now it's safe because task bound breakpoint only allow userspace addresses.
But this is not right because we want task bound to be able to trigger on kernel. Once we'll allow this, we need a bit of policy.
Because yeah we can protect against recursion, but always partially. The only solution to really protect would be disabling the breakpoint as the very first operation in the trap handler.
But that's ugly and may be costly as well.
So what I'm going to do is to simply check attr.exclude_kernel on breakpoint creation and look if the address is in the kernel space.
So that if you want to crash your machine why lurking at the very tight window of trap handler possible recursion, you need to be root :)
Because when a perf event registers and doesn't have exclude_kernel, you need to be admin.
Fortunately we don't have this security hole with the current rules, but I'll try to make this check more generic (will require archs that support breakpoints to implement arch_check_va_in_kernelspace()).
Quoted 11 lines
Quoted 3 lines
If an arch has no choice but using sync breakpoints, it needs to handle that. It seems PowerPc has a similar problem, may be have a look at Prasad's patches.
Could you give me a pointer to these patches please [I can't see them on LKML]? I *think* PowerPC is lucky enough to have hardware single step, so handling synchronous breakpoints isn't *too* difficult. In ARM we have to work out the address of the following instruction, which is quite tricky because it could be a branch instruction [from a selection of instruction sets] and also be predicated. GDB can handle this, but the Kernel doesn't have the capability to do so [there's some code in
arch/arm/ptrace.cbut it's not feature complete].
I don't remember exactly how this is handled. I remember a story about a one-shot breakpoint, ie: that triggers only once.
Anyway, I have yet to review the latest version:
http://lists.ozlabs.org/pipermail/linuxppc-dev/2010-April/081714.html
http://lists.ozlabs.org/pipermail/linuxppc-dev/2010-April/081715.html
http://lists.ozlabs.org/pipermail/linuxppc-dev/2010-April/081716.html
Quoted 8 lines
Quoted 3 lines
If there is something that can be handled from the generic layer to help solving this problem, I'll try something.
Unfortunately this is very much an arch problem. As a half-way house, I might implement only the ptrace part of hw-breakpoint for ARM. That way we can force the client to handle stepping over the breakpoint and we also won't have to worry about recursive breakpoints in the Kernel.
:-s
Yeah, we'll see what we can do.