hw-breakpoints: Rewrite on top of perf events v4
Subsystems (7)
Hi all,
This is the v4 of the hw-breakpoints API rewrite on top of perf events.
This integrates the following fixes:
Changes in v4:
- Drop the hw_breakpoint_restore() stub as it is only used by KVM
- EXPORT_SYMBOL_GPL hw_breakpoint_restore() as KVM can be built as a module
- Restore the breakpoints unconditionally on kvm guest exit:
TIF_DEBUG_THREAD doesn't anymore cover every cases of running breakpoints and vcpu->arch.switch_db_regs might not always be set when the guest used debug registers.
(Waiting for a reliable optimization)
- Simplify a bit the callback attribution as suggested by Paul Mackerras
- Remove the wrong comment about the fact perf_event_create_kernel_counter must be called from a kernel thread.
Changes in v3:
- Fix broken CONFIG_KVM, propagate the breakpoint api changes to kvm when we exit the guest and restore the bp registers to the host. The only change is in the 4th patch.
Changes in v2:
- Follow the perf "event " rename
- The ptrace regression have been fixed (ptrace breakpoint perf events weren't released when a task ended)
- Drop the struct hw_breakpoint and store generic fields in perf_event_attr.
- Separate core and arch specific headers, drop asm-generic/hw_breakpoint.h and create linux/hw_breakpoint.h
- Use new generic len/type for breakpoint
- Handle off case: when breakpoints api is not supported by an arch
- Use proper in-kernel perf api provided by Arjan.
This tree can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
perfevents/hw-breakpoint-v4
Arjan van de Ven (1):
perf/core: Provide a kernel-internal interface to get to performance counters
Frederic Weisbecker (3):
perf/core: Add a callback to perf events
hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
hw-breakpoints: Arbitrate access to pmu following registers constraints
Li Zefan (1):
ksym_tracer: Remove KSYM_SELFTEST_ENTRY
Paul Mundt (1):
x86/hw-breakpoints: Actually flush thread breakpoints in flush_thread().
arch/Kconfig | 3 + arch/x86/include/asm/debugreg.h | 11 +- arch/x86/include/asm/hw_breakpoint.h | 58 +++-- arch/x86/include/asm/processor.h | 12 +- arch/x86/kernel/hw_breakpoint.c | 391 +++++++++++++++-------- arch/x86/kernel/process.c | 9 +- arch/x86/kernel/process_32.c | 26 +-- arch/x86/kernel/process_64.c | 26 +-- arch/x86/kernel/ptrace.c | 182 +++++++---- arch/x86/kernel/smpboot.c | 3 - arch/x86/kvm/x86.c | 15 +- arch/x86/power/cpu.c | 6 - include/asm-generic/hw_breakpoint.h | 139 -------- include/linux/hw_breakpoint.h | 131 ++++++++ include/linux/perf_event.h | 37 ++- kernel/exit.c | 5 + kernel/hw_breakpoint.c | 595 +++++++++++++++++++++------------- kernel/perf_event.c | 136 ++++++++- kernel/trace/trace.h | 1 - kernel/trace/trace_entries.h | 6 +- kernel/trace/trace_ksym.c | 126 ++++---- kernel/trace/trace_selftest.c | 2 +- 22 files changed, 1180 insertions(+), 740 deletions(-) delete mode 100644 include/asm-generic/hw_breakpoint.h create mode 100644 include/linux/hw_breakpoint.h
PATCH 1/6 perf/core: Provide a kernel-internal interface to get to performance counters
From: Arjan van de Ven <[email protected]>
There are reasons for kernel code to ask for, and use, performance counters.
For example, in CPU freq governors this tends to be a good idea, but there are other examples possible as well of course.
This patch adds the needed bits to do enable this functionality; they have been tested in an experimental cpufreq driver that I'm working on, and the changes are all that I needed to access counters properly.
[[email protected]: added pid to perf_event_create_kernel_counter so that we can profile a particular task too
TODO: Have a better error reporting, don't just return NULL in fail case.]
v2: Remove the wrong comment about the fact
perf_event_create_kernel_counter must be called from a kernel thread.
LKML-Reference: <[email protected]>
--- include/linux/perf_event.h | 6 +++ kernel/perf_event.c | 75 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 80 insertions(+), 1 deletions(-)
include/linux/perf_event.h +6 -0
@@ -744,6 +744,12 @@ extern int hw_perf_group_sched_in(struct perf_event *group_leader,struct perf_cpu_context *cpuctx, struct perf_event_context *ctx, int cpu);extern void perf_event_update_userpage (struct perf_event *event); +extern int perf_event_release_kernel (struct perf_event *event); +extern struct perf_event * +perf_event_create_kernel_counter (struct perf_event_attr *attr, +int cpu, +pid_t pid); +extern u64 perf_event_read_value (struct perf_event *event);struct perf_sample_data {u64 type;
kernel/perf_event.c +74 -1
99 lines not on this page · raw message
Signature · 1 line
1.6.2.3
PATCH 2/6 x86/hw-breakpoints: Actually flush thread breakpoints in flush_thread().
In reply to Frederic WeisbeckerFrom: Paul Mundt <[email protected]>
flush_thread() tries to do a TIF_DEBUG check before calling in to flush_thread_hw_breakpoint() (which subsequently clears the thread flag), but for some reason, the x86 code is manually clearing TIF_DEBUG immediately before the test, so this path will never be taken.
This kills off the erroneous clear_tsk_thread_flag() and lets flush_thread_hw_breakpoint() actually get invoked.
Presumably folks were getting lucky with testing and the free_thread_info() -> free_thread_xstate() path was taking care of the flush there.
LKML-Reference: <[email protected]>
--- arch/x86/kernel/process.c | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-)
arch/x86/kernel/process.c +0 -2
@@ -107,8 +107,6 @@ void flush_thread(void) }#endif -clear_tsk_thread_flag (tsk,TIF_DEBUG ); -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG )))flush_thread_hw_breakpoint (tsk);memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array));
Signature · 1 line
1.6.2.3
PATCH 3/6 perf/core: Add a callback to perf events
In reply to Frederic WeisbeckerA simple callback in a perf event can be used for multiple purposes. For example it is useful for triggered based events like hardware breakpoints that need a callback to dispatch a triggered breakpoint event.
v2: Simplify a bit the callback attribution as suggested by Paul
Mackerras
--- include/linux/perf_event.h | 7 ++++++- kernel/perf_event.c | 14 ++++++++++---- 2 files changed, 16 insertions(+), 5 deletions(-)
include/linux/perf_event.h +6 -1
@@ -544,6 +544,8 @@ struct perf_pending_entry {void (*func)(struct perf_pending_entry *); }; +typedef void (*perf_callback_t )(struct perf_event *,void *); +/** * struct perf_event - performance event kernel representation: */
@@ -639,6 +641,8 @@ struct perf_event {struct event_filter *filter;#endif +perf_callback_t callback; +#endif /* CONFIG_PERF_EVENTS */ };
@@ -748,7 +752,8 @@ extern int perf_event_release_kernel(struct perf_event *event);extern struct perf_event *perf_event_create_kernel_counter (struct perf_event_attr *attr,int cpu, -pid_t pid); +pid_t pid, +perf_callback_t callback);extern u64 perf_event_read_value (struct perf_event *event);struct perf_sample_data {
kernel/perf_event.c +10 -4
@@ -4293,6 +4293,7 @@ perf_event_alloc(struct perf_event_attr *attr,struct perf_event_context *ctx, struct perf_event *group_leader, struct perf_event *parent_event, + perf_callback_t callback, gfp_t gfpflags) { const struct pmu *pmu;
@@ -4335,6 +4336,11 @@ perf_event_alloc(struct perf_event_attr *attr, event->state =PERF_EVENT_STATE_INACTIVE ; +if (!callback && parent_event) + callback = parent_event->callback; + + event->callback = callback; +if (attr->disabled) event->state =PERF_EVENT_STATE_OFF ;
@@ -4611,7 +4617,7 @@ SYSCALL_DEFINE5(perf_event_open, } event =perf_event_alloc (&attr, cpu, ctx, group_leader, -NULL ,GFP_KERNEL ); +NULL ,NULL ,GFP_KERNEL ); err =PTR_ERR (event);if (IS_ERR (event)) goto err_put_context;
@@ -4668,7 +4674,7 @@ err_put_context:*/ struct perf_event *perf_event_create_kernel_counter (struct perf_event_attr *attr,int cpu, -pid_t pid) +pid_t pid,perf_callback_t callback) {struct perf_event *event;struct perf_event_context *ctx;
@@ -4683,7 +4689,7 @@ perf_event_create_kernel_counter(struct perf_event_attr *attr, int cpu,return NULL ; event =perf_event_alloc (attr, cpu, ctx,NULL , -NULL ,GFP_KERNEL ); +NULL , callback,GFP_KERNEL ); err =PTR_ERR (event);if (IS_ERR (event)) goto err_put_context;
@@ -4736,7 +4742,7 @@ inherit_event(struct perf_event *parent_event, child_event =perf_event_alloc (&parent_event->attr, parent_event->cpu, child_ctx, group_leader, parent_event, -GFP_KERNEL ); +NULL ,GFP_KERNEL );if (IS_ERR (child_event))return child_event;get_ctx (child_ctx);
Signature · 1 line
1.6.2.3
PATCH 4/6 hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
In reply to Frederic WeisbeckerThis patch rebase the implementation of the breakpoints API on top of perf events instances.
Each breakpoints are now perf events that handle the register scheduling, thread/cpu attachment, etc..
The new layering is now made as follows:
ptrace kgdb ftrace perf syscall
\ | / /
\ | / /
/
Core breakpoint API /
/
| /
| /
Breakpoints perf events
|
|
Breakpoints PMU ---- Debug Register constraints handling
(Part of core breakpoint API)
|
|
Hardware debug registers
Reasons of this rewrite:
- Use the centralized/optimized pmu registers scheduling, implying an easier arch integration
- More powerful register handling: perf attributes (pinned/flexible events, exclusive/non-exclusive, tunable period, etc...)
Impact:
- New perf ABI: the hardware breakpoints counters
- Ptrace breakpoints setting remains tricky and still needs some per thread breakpoints references.
Todo (in the order):
- Support breakpoints perf counter events for perf tools (ie: implement perf_bpcounter_event())
- Support from perf tools
Changes in v2:
- Follow the perf "event " rename
- The ptrace regression have been fixed (ptrace breakpoint perf events weren't released when a task ended)
- Drop the struct hw_breakpoint and store generic fields in perf_event_attr.
- Separate core and arch specific headers, drop asm-generic/hw_breakpoint.h and create linux/hw_breakpoint.h
- Use new generic len/type for breakpoint
- Handle off case: when breakpoints api is not supported by an arch
Changes in v3:
- Fix broken CONFIG_KVM, we need to propagate the breakpoint api changes to kvm when we exit the guest and restore the bp registers to the host.
Changes in v4:
- Drop the hw_breakpoint_restore() stub as it is only used by KVM
- EXPORT_SYMBOL_GPL hw_breakpoint_restore() as KVM can be built as a module
- Restore the breakpoints unconditionally on kvm guest exit:
TIF_DEBUG_THREAD doesn't anymore cover every cases of running breakpoints and vcpu->arch.switch_db_regs might not always be set when the guest used debug registers.
(Waiting for a reliable optimization)
--- arch/Kconfig | 3 + arch/x86/include/asm/debugreg.h | 11 +- arch/x86/include/asm/hw_breakpoint.h | 58 +++-- arch/x86/include/asm/processor.h | 12 +- arch/x86/kernel/hw_breakpoint.c | 391 ++++++++++++++++++++----------- arch/x86/kernel/process.c | 7 +- arch/x86/kernel/process_32.c | 26 +-- arch/x86/kernel/process_64.c | 26 +-- arch/x86/kernel/ptrace.c | 182 ++++++++++----- arch/x86/kernel/smpboot.c | 3 - arch/x86/kvm/x86.c | 15 +- arch/x86/power/cpu.c | 6 - include/asm-generic/hw_breakpoint.h | 139 ----------- include/linux/hw_breakpoint.h | 131 +++++++++++ include/linux/perf_event.h | 26 ++- kernel/exit.c | 5 + kernel/hw_breakpoint.c | 424 ++++++++++++++-------------------- kernel/perf_event.c | 53 ++++- kernel/trace/trace_entries.h | 6 +- kernel/trace/trace_ksym.c | 126 +++++------ 20 files changed, 889 insertions(+), 761 deletions(-) delete mode 100644 include/asm-generic/hw_breakpoint.h create mode 100644 include/linux/hw_breakpoint.h
arch/Kconfig +3 -0
@@ -128,6 +128,9 @@ config HAVE_DEFAULT_NO_SPIN_MUTEXES config HAVE_HW_BREAKPOINT bool + depends on HAVE_PERF_EVENTS + select ANON_INODES + select PERF_EVENTS source "kernel/gcov/Kconfig"
arch/x86/include/asm/debugreg.h +4 -7
@@ -75,13 +75,6 @@*/ #ifdef __KERNEL__ -/* For process management */ -extern void flush_thread_hw_breakpoint (struct task_struct *tsk); -extern int copy_thread_hw_breakpoint (struct task_struct *tsk, -struct task_struct *child,unsigned long clone_flags); - -/* For CPU management */ -extern void load_debug_registers (void );static inline void hw_breakpoint_disable (void ) {/* Zero the control register for HW Breakpoint */
@@ -94,6 +87,10 @@ static inline void hw_breakpoint_disable(void)set_debugreg (0UL ,3 ); } +#ifdef CONFIG_KVM +extern void hw_breakpoint_restore (void ); +#endif +#endif /* __KERNEL__ */ #endif /* _ASM_X86_DEBUGREG_H */
arch/x86/include/asm/hw_breakpoint.h +38 -20
88 lines not on this page · raw message
arch/x86/include/asm/processor.h +6 -6
@@ -423,6 +423,8 @@ extern unsigned int xstate_size;extern void free_thread_xstate (struct task_struct *);extern struct kmem_cache *task_xstate_cachep; +struct perf_event ; +struct thread_struct {/* Cached TLS descriptors: */ struct desc_struct tls_array[GDT_ENTRY_TLS_ENTRIES ];
@@ -444,12 +446,10 @@ struct thread_struct {unsigned long fs;#endif unsigned long gs; -/* Hardware debugging registers: */ -unsigned long debugreg[HBP_NUM ]; -unsigned long debugreg6; -unsigned long debugreg7; -/* Hardware breakpoint info */ -struct hw_breakpoint *hbp[HBP_NUM ]; +/* Save middle states of ptrace breakpoints */ +struct perf_event *ptrace_bps[HBP_NUM ]; +/* Debug status used for traps, single steps, etc... */ +unsigned long debugreg6;/* Fault info: */ unsigned long cr2;unsigned long trap_no;
arch/x86/kernel/hw_breakpoint.c +255 -136
549 lines not on this page · raw message
arch/x86/kernel/process.c +2 -5
@@ -10,6 +10,7 @@#include <linux/clockchips.h> #include <linux/random.h> #include <trace/events/power.h> +#include <linux/hw_breakpoint.h> #include <asm/system.h> #include <asm/apic.h> #include <asm/syscalls.h>
@@ -18,7 +19,6 @@#include <asm/i387.h> #include <asm/ds.h> #include <asm/debugreg.h> -#include <asm/hw_breakpoint.h> unsigned long idle_halt;EXPORT_SYMBOL (idle_halt);
@@ -47,8 +47,6 @@ void free_thread_xstate(struct task_struct *tsk)kmem_cache_free (task_xstate_cachep, tsk->thread.xstate); tsk->thread.xstate =NULL ; } -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG ))) -flush_thread_hw_breakpoint (tsk);WARN (tsk->thread.ds_ctx,"leaking DS context\n" ); }
@@ -107,8 +105,7 @@ void flush_thread(void) }#endif -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG ))) -flush_thread_hw_breakpoint (tsk); +flush_ptrace_hw_breakpoint (tsk);memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array));/* * Forget coprocessor state..
arch/x86/kernel/process_32.c +2 -24
@@ -59,7 +59,6 @@#include <asm/syscalls.h> #include <asm/ds.h> #include <asm/debugreg.h> -#include <asm/hw_breakpoint.h> asmlinkage void ret_from_fork(void) __asm__("ret_from_fork");
@@ -264,9 +263,8 @@ int copy_thread(unsigned long clone_flags, unsigned long sp, p->thread.io_bitmap_ptr =NULL ; tsk = current; err = -ENOMEM ; -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG ))) -if (copy_thread_hw_breakpoint (tsk, p, clone_flags)) - goto out; + +memset (p->thread.ptrace_bps,0 ,sizeof (p->thread.ptrace_bps));if (unlikely (test_tsk_thread_flag (tsk,TIF_IO_BITMAP ))) { p->thread.io_bitmap_ptr = kmemdup(tsk->thread.io_bitmap_ptr,
@@ -287,13 +285,10 @@ int copy_thread(unsigned long clone_flags, unsigned long sp, err =do_set_thread_area (p,-1 , (struct user_desc __user *)childregs->si,0 ); -out:if (err && p->thread.io_bitmap_ptr) {kfree (p->thread.io_bitmap_ptr); p->thread.io_bitmap_max =0 ; } -if (err) -flush_thread_hw_breakpoint (p);clear_tsk_thread_flag (p,TIF_DS_AREA_MSR ); p->thread.ds_ctx =NULL ;
@@ -437,23 +432,6 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)lazy_load_gs (next->gs);percpu_write (current_task, next_p); -/* -* There's a problem with moving the arch_install_thread_hw_breakpoint() -* call before current is updated. Suppose a kernel breakpoint is -* triggered in between the two, the hw-breakpoint handler will see that -* the 'current' task does not have TIF_DEBUG flag set and will think it -* is leftover from an old task (lazy switching) and will erase it. Then -* until the next context switch, no user-breakpoints will be installed. -* -* The real problem is that it's impossible to update both current and -* physical debug registers at the same instant, so there will always be -* a window in which they disagree and a breakpoint might get triggered. -* Since we use lazy switching, we are forced to assume that a -* disagreement means that current is correct and the exception is due -* to lazy debug register switching. -*/ -if (unlikely (test_tsk_thread_flag (next_p,TIF_DEBUG ))) -arch_install_thread_hw_breakpoint (next_p);return prev_p; }
arch/x86/kernel/process_64.c +1 -25
@@ -53,7 +53,6 @@#include <asm/syscalls.h> #include <asm/ds.h> #include <asm/debugreg.h> -#include <asm/hw_breakpoint.h> asmlinkage extern void ret_from_fork(void);
@@ -244,8 +243,6 @@ void release_thread(struct task_struct *dead_task)BUG (); } } -if (unlikely (dead_task->thread.debugreg7)) -flush_thread_hw_breakpoint (dead_task); }static inline void set_32bit_tls (struct task_struct *t,int tls,u32 addr)
@@ -309,9 +306,7 @@ int copy_thread(unsigned long clone_flags, unsigned long sp,savesegment (ds, p->thread.ds); err = -ENOMEM ; -if (unlikely (test_tsk_thread_flag (me,TIF_DEBUG ))) -if (copy_thread_hw_breakpoint (me, p, clone_flags)) - goto out; +memset (p->thread.ptrace_bps,0 ,sizeof (p->thread.ptrace_bps));if (unlikely (test_tsk_thread_flag (me,TIF_IO_BITMAP ))) { p->thread.io_bitmap_ptr =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL );
@@ -351,8 +346,6 @@ out:kfree (p->thread.io_bitmap_ptr); p->thread.io_bitmap_max =0 ; } -if (err) -flush_thread_hw_breakpoint (p);return err; }
@@ -508,23 +501,6 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)*/ if (preload_fpu)__math_state_restore (); -/* -* There's a problem with moving the arch_install_thread_hw_breakpoint() -* call before current is updated. Suppose a kernel breakpoint is -* triggered in between the two, the hw-breakpoint handler will see that -* the 'current' task does not have TIF_DEBUG flag set and will think it -* is leftover from an old task (lazy switching) and will erase it. Then -* until the next context switch, no user-breakpoints will be installed. -* -* The real problem is that it's impossible to update both current and -* physical debug registers at the same instant, so there will always be -* a window in which they disagree and a breakpoint might get triggered. -* Since we use lazy switching, we are forced to assume that a -* disagreement means that current is correct and the exception is due -* to lazy debug register switching. -*/ -if (unlikely (test_tsk_thread_flag (next_p,TIF_DEBUG ))) -arch_install_thread_hw_breakpoint (next_p);return prev_p; }
arch/x86/kernel/ptrace.c +125 -57
257 lines not on this page · raw message
arch/x86/kernel/smpboot.c +0 -3
@@ -64,7 +64,6 @@#include <asm/apic.h> #include <asm/setup.h> #include <asm/uv/uv.h> -#include <asm/debugreg.h> #include <linux/mc146818rtc.h> #include <asm/smpboot_hooks.h>
@@ -328,7 +327,6 @@ notrace static void __cpuinit start_secondary(void *unused) x86_cpuinit.setup_percpu_clockev ();wmb (); -load_debug_registers ();cpu_idle (); }
@@ -1269,7 +1267,6 @@ void cpu_disable_common(void)remove_cpu_from_maps (cpu);unlock_vector_lock ();fixup_irqs (); -hw_breakpoint_disable (); }int native_cpu_disable (void )
arch/x86/kvm/x86.c +7 -8
@@ -42,6 +42,7 @@#define CREATE_TRACE_POINTS #include "trace.h" +#include <asm/debugreg.h> #include <asm/uaccess.h> #include <asm/msr.h> #include <asm/desc.h>
@@ -3643,14 +3644,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)trace_kvm_entry (vcpu->vcpu_id); kvm_x86_ops->run (vcpu, kvm_run); -if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) { -set_debugreg (current->thread.debugreg[0 ],0 ); -set_debugreg (current->thread.debugreg[1 ],1 ); -set_debugreg (current->thread.debugreg[2 ],2 ); -set_debugreg (current->thread.debugreg[3 ],3 ); -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } +/* +* CHECKME: is vcpu->arch.switch_db_regs sufficient to check +* if the guest is using breakpoints? If so we may want to do +* this check before. +*/ +hw_breakpoint_restore ();set_bit (KVM_REQ_KICK , &vcpu->requests);local_irq_enable ();
arch/x86/power/cpu.c +0 -6
@@ -105,7 +105,6 @@ static void __save_processor_state(struct saved_context *ctxt) ctxt->cr4 =read_cr4 (); ctxt->cr8 =read_cr8 ();#endif -hw_breakpoint_disable (); }/* Needed by apm.c */
@@ -144,11 +143,6 @@ static void fix_processor_context(void)#endif load_TR_desc ();/* This does ltr */ load_LDT (¤t->active_mm->context);/* This does lldt */ - -/* -* Now maybe reload the debug registers -*/ -load_debug_registers (); }/**
include/asm-generic/hw_breakpoint.h deleted +0 -139
139 lines not on this page · raw message
include/linux/hw_breakpoint.h new file +131 -0
131 lines not on this page · raw message
include/linux/perf_event.h +25 -1
@@ -18,6 +18,10 @@#include <linux/ioctl.h> #include <asm/byteorder.h> +#ifdef CONFIG_HAVE_HW_BREAKPOINT +#include <asm/hw_breakpoint.h> +#endif +/* * User-space ABI bits: */
@@ -31,6 +35,7 @@ enum perf_type_id {PERF_TYPE_TRACEPOINT =2 ,PERF_TYPE_HW_CACHE =3 ,PERF_TYPE_RAW =4 , +PERF_TYPE_BREAKPOINT =5 , PERF_TYPE_MAX, /* non-ABI */ };
@@ -207,6 +212,15 @@ struct perf_event_attr {__u32 wakeup_events;/* wakeup every n events */ __u32 wakeup_watermark;/* bytes before wakeup */ }; + +union { +struct {/* Hardware breakpoint info */ +__u64 bp_addr; +__u32 bp_type; +__u32 bp_len; + }; + }; +__u32 __reserved_2;__u64 __reserved_3;
@@ -476,6 +490,11 @@ struct hw_perf_event {atomic64_t count;struct hrtimer hrtimer; }; +#ifdef CONFIG_HAVE_HW_BREAKPOINT +union {/* breakpoint */ +struct arch_hw_breakpoint info; + }; +#endif };atomic64_t prev_count;u64 sample_period;
@@ -588,7 +607,7 @@ struct perf_event {u64 tstamp_running;u64 tstamp_stopped; -struct perf_event_attr attr; +struct perf_event_attr attr;struct hw_perf_event hw;struct perf_event_context *ctx;
@@ -643,6 +662,8 @@ struct perf_event {perf_callback_t callback; +perf_callback_t event_callback; +#endif /* CONFIG_PERF_EVENTS */ };
@@ -831,6 +852,7 @@ extern int sysctl_perf_event_sample_rate;extern void perf_event_init (void );extern void perf_tp_event (int event_id,u64 addr,u64 count,void *record,int entry_size); +extern void perf_bp_event (struct perf_event *event,void *data);#ifndef perf_misc_flags#define perf_misc_flags (regs) (user_mode(regs) ? PERF_RECORD_MISC_USER : \
@@ -865,6 +887,8 @@ static inline int perf_event_task_enable(void) { return -EINVAL; }static inline void perf_sw_event (u32 event_id,u64 nr,int nmi,struct pt_regs *regs,u64 addr) { } +static inline void +perf_bp_event (struct perf_event *event,void *data) { }static inline void perf_event_mmap (struct vm_area_struct *vma) { }static inline void perf_event_comm (struct task_struct *tsk) { }
kernel/exit.c +5 -0
@@ -49,6 +49,7 @@#include <linux/init_task.h> #include <linux/perf_event.h> #include <trace/events/sched.h> +#include <linux/hw_breakpoint.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -980,6 +981,10 @@ NORET_TYPE void do_exit(long code)proc_exit_connector (tsk);/* +* FIXME: do that only when needed, using sched_exit tracepoint +*/ +flush_ptrace_hw_breakpoint (tsk); +/* * Flush inherited counters to the parent - before the parent * gets woken up by child-exit notifications. */
kernel/hw_breakpoint.c +170 -254
515 lines not on this page · raw message
kernel/perf_event.c +52 -1
77 lines not on this page · raw message
kernel/trace/trace_entries.h +3 -3
@@ -372,11 +372,11 @@ FTRACE_ENTRY(ksym_trace, ksym_trace_entry, F_STRUCT( __field( unsigned long, ip ) __field( unsigned char, type ) - __array( char , ksym_name, KSYM_NAME_LEN ) __array( char , cmd, TASK_COMM_LEN ) + __field( unsigned long, addr ) ), - F_printk("ip: %pF type: %d ksym_name: %s cmd: %s", + F_printk("ip: %pF type: %d ksym_name: %pS cmd: %s", (void *)__entry->ip, (unsigned int )__entry->type, - __entry->ksym_name, __entry->cmd) + (void *)__entry->addr, __entry->cmd) );
kernel/trace/trace_ksym.c +60 -66
241 lines not on this page · raw message
Signature · 1 line
1.6.2.3
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 20 lines · Frederic Weisbecker
arch/x86/kvm/x86.c +1
-4@@ -42,6 +42,7 @@#define CREATE_TRACE_POINTS #include "trace.h" +#include <asm/debugreg.h> #include <asm/uaccess.h> #include <asm/msr.h> #include <asm/desc.h> @@ -3643,14 +3644,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)trace_kvm_entry (vcpu->vcpu_id); kvm_x86_ops->run (vcpu, kvm_run); -if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) { -set_debugreg (current->thread.debugreg[0 ],0 ); -set_debugreg (current->thread.debugreg[1 ],1 ); -set_debugreg (current->thread.debugreg[2 ],2 );
arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) · line 3652
Quoted 10 lines
arch/x86/kvm/x86.c +6
-4-set_debugreg (current->thread.debugreg[3 ],3 ); -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } +/* +* CHECKME: is vcpu->arch.switch_db_regs sufficient to check +* if the guest is using breakpoints? If so we may want to do +* this check before. +*/ +hw_breakpoint_restore ();
Obviously, this variant will make KVM users very unhappy. But trying to reduce this performance regression via vcpu->arch.switch_db_regs will make hw-breakpoint users unhappy: KVM leaves at least dr7 clobbered behind, even if the guest does not use breakpoints.
We really need a replacement for TIF_DEBUG (but we only need this [1]).
Jan
[1]http://thread.gmane.org/gmane.comp.emulators.kvm.devel/39784/focus=39827
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 28 lines · Jan Kiszka
Quoted 27 lines · Frederic Weisbecker
arch/x86/kvm/x86.c +4
-8@@ -42,6 +42,7 @@#define CREATE_TRACE_POINTS #include "trace.h" +#include <asm/debugreg.h> #include <asm/uaccess.h> #include <asm/msr.h> #include <asm/desc.h> @@ -3643,14 +3644,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)trace_kvm_entry (vcpu->vcpu_id); kvm_x86_ops->run (vcpu, kvm_run); -if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) { -set_debugreg (current->thread.debugreg[0 ],0 ); -set_debugreg (current->thread.debugreg[1 ],1 ); -set_debugreg (current->thread.debugreg[2 ],2 ); -set_debugreg (current->thread.debugreg[3 ],3 ); -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } +/* +* CHECKME: is vcpu->arch.switch_db_regs sufficient to check +* if the guest is using breakpoints? If so we may want to do
Quoted 8 lines
arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) · line 3652
Quoted 3 lines
arch/x86/kvm/x86.c +3
-0+* this check before. +*/ +hw_breakpoint_restore ();Obviously, this variant will make KVM users very unhappy. But trying to reduce this performance regression via
vcpu->arch.switch_db_regswill make hw-breakpoint users unhappy: KVM leaves at least dr7 clobbered behind, even if the guest does not use breakpoints.
Yeah, that's why I've made unconditionally. At least it works in every cases, but this is temporary.
Quoted 6 lines
We really need a replacement for
TIF_DEBUG(but we only need this [1]).Jan
[1]http://thread.gmane.org/gmane.comp.emulators.kvm.devel/39784/focus=39827
Thinking about it, this check should cover every cases:
if (vcpu->arch.switch_db_regs || __get_cpu_var(dr7) & DR_GLOBAL_ENABLE_MASK)
If we have __get_cpu_var(dr7) & DR_GLOBAL_ENABLE_MASK, it means there is an active breakpoint and then we should restore the current state.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 50 lines · Frederic Weisbecker
Quoted 27 lines · Jan Kiszka
Quoted 26 lines · Frederic Weisbecker
arch/x86/kvm/x86.c +3
-8@@ -42,6 +42,7 @@#define CREATE_TRACE_POINTS #include "trace.h" +#include <asm/debugreg.h> #include <asm/uaccess.h> #include <asm/msr.h> #include <asm/desc.h> @@ -3643,14 +3644,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)trace_kvm_entry (vcpu->vcpu_id); kvm_x86_ops->run (vcpu, kvm_run); -if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) { -set_debugreg (current->thread.debugreg[0 ],0 ); -set_debugreg (current->thread.debugreg[1 ],1 ); -set_debugreg (current->thread.debugreg[2 ],2 ); -set_debugreg (current->thread.debugreg[3 ],3 ); -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } +/* +* CHECKME: is vcpu->arch.switch_db_regs sufficient to check Quoted 8 lines
arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) · line 3652
Quoted 4 lines
arch/x86/kvm/x86.c +4
-0+* if the guest is using breakpoints? If so we may want to do +* this check before. +*/ +hw_breakpoint_restore ();Obviously, this variant will make KVM users very unhappy. But trying to reduce this performance regression via
vcpu->arch.switch_db_regswill make hw-breakpoint users unhappy: KVM leaves at least dr7 clobbered behind, even if the guest does not use breakpoints.Yeah, that's why I've made unconditionally. At least it works in every cases, but this is temporary.
Quoted 6 lines
We really need a replacement for
TIF_DEBUG(but we only need this [1]).Jan
[1]http://thread.gmane.org/gmane.comp.emulators.kvm.devel/39784/focus=39827
Quoted 7 lines
Thinking about it, this check should cover every cases:
if (
vcpu->arch.switch_db_regs||__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK)If we have
__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK, it means there is an active breakpoint and then we should restore the current state.
And what about (__get_cpu_var(dr7) & DR_GLOBAL_ENABLE_MASK) only? Would you be able to live with unsync'ed hardware and software states?
Jan
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 4 lines · Jan Kiszka
Quoted 4 lines
Thinking about it, this check should cover every cases:
if (
vcpu->arch.switch_db_regs||__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK)
Quoted 9 lines
Quoted 3 lines
If we have
__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK, it means there is an active breakpoint and then we should restore the current state.And what about (
__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK) only? Would you be able to live with unsync'ed hardware and software states?Jan
But if the guest has breakpoints activated, the host will inherit them, which is really not something we want, assuming vcpu->arch.switch_db_regs already protects us about that.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 11 lines · Frederic Weisbecker
Quoted 10 lines · Jan Kiszka
Quoted 7 lines
Thinking about it, this check should cover every cases:
if (
vcpu->arch.switch_db_regs||__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK)If we have
__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK, it means there is an active breakpoint and then we should restore the current state.And what about (
__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK) only? Would you be able to live with unsync'ed hardware and software states?
Quoted 8 lines
Quoted 2 lines
Jan
But if the guest has breakpoints activated, the host will inherit them, which is really not something we want, assuming
vcpu->arch.switch_db_regsalready protects us about that.
Nope, vmx&svm will clear dr7 on vmexit for us. Really, switch_db_regs should not be needed if we can leave the debug registers clobbered but disabled behind.
Jan
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 21 lines · Jan Kiszka
Quoted 11 lines · Frederic Weisbecker
Quoted 10 lines · Jan Kiszka
Quoted 7 lines
Thinking about it, this check should cover every cases:
if (
vcpu->arch.switch_db_regs||__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK)If we have
__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK, it means there is an active breakpoint and then we should restore the current state.And what about (
__get_cpu_var(dr7)&DR_GLOBAL_ENABLE_MASK) only? Would you be able to live with unsync'ed hardware and software states?Quoted 8 lines
Quoted 2 lines
Jan
But if the guest has breakpoints activated, the host will inherit them, which is really not something we want, assuming
vcpu->arch.switch_db_regsalready protects us about that.
Quoted 6 lines
Nope, vmx&svm will clear dr7 on vmexit for us. Really,
switch_db_regsshould not be needed if we can leave the debug registers clobbered but disabled behind.Jan
That looks fine then.
We never manipulate the breakpoints as beeing disabled from perf (address reg set but dr7 disabled) so we can have such unstable state in the address registers if dr7 is not enabled for the given address registers.
Fine, I'll send another patch for that.
Thanks.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
In reply to Frederic WeisbeckerQuoted 5 lines · Frederic Weisbecker
This patch rebase the implementation of the breakpoints API on top of perf events instances.
Each breakpoints are now perf events that handle the register scheduling, thread/cpu attachment, etc..
What I haven't managed to understand yet is how you provide reliable breakpoints for debugging purposes. If I'm debugging a program and I have set a breakpoint, I'll be very unhappy if the breakpoint should trigger but doesn't because the perf_event infrastructure has decided it can't schedule that breakpoint in. If the breakpoint isn't going to work then I want to know that at the time that I set it.
We can go some distance towards that with the pinned attribute, but not far enough. The pinned attribute doesn't guarantee that the event will always be scheduled in, it just says that we'll do our best to schedule it in, and if we can't, we'll put the event into error state so that the user knows we didn't manage to schedule it in. But there's no way to get back to gdb and tell it that a breakpoint that it had previously successfully created is no longer working.
Also, we don't currently fail the creation of a pinned event if it would conflict with another pinned event already created in the same context. We would need to do something like that if we want to use pinned events for debugging breakpoints (as distinct from breakpoints for performance monitoring purposes, for which it matters less if they are sometimes not scheduled in).
And then there's the question of whether a per-cpu pinned breakpoint event conflicts with a per-task pinned breakpoint event if you only have one breakpoint register (as is the case on Power server CPUs). Plus the fact that we don't currently give per-task pinned events priority over per-cpu non-pinned events (perhaps that would be a good idea anyway).
Paul.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 8 lines · Paul Mackerras
Quoted 5 lines · Frederic Weisbecker
This patch rebase the implementation of the breakpoints API on top of perf events instances.
Each breakpoints are now perf events that handle the register scheduling, thread/cpu attachment, etc..
Quoted 7 lines
What I haven't managed to understand yet is how you provide reliable breakpoints for debugging purposes. If I'm debugging a program and I have set a breakpoint, I'll be very unhappy if the breakpoint should trigger but doesn't because the
perf_eventinfrastructure has decided it can't schedule that breakpoint in. If the breakpoint isn't going to work then I want to know that at the time that I set it.
The hw-breakpoint layer avoids such issues by not over-committing debug registers i.e. 'n' number of debug registers are reserved for user-space where n = max(no. of breakpoints requested by any given process/thread). And book-keeping certainly helps here too.
Thanks,
K.Prasad
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 9 lines · K.Prasad
Quoted 7 lines · Paul Mackerras
What I haven't managed to understand yet is how you provide reliable breakpoints for debugging purposes. If I'm debugging a program and I have set a breakpoint, I'll be very unhappy if the breakpoint should trigger but doesn't because the
perf_eventinfrastructure has decided it can't schedule that breakpoint in. If the breakpoint isn't going to work then I want to know that at the time that I set it.
Quoted 4 lines
The hw-breakpoint layer avoids such issues by not over-committing debug registers i.e. 'n' number of debug registers are reserved for user-space where n = max(no. of breakpoints requested by any given process/thread). And book-keeping certainly helps here too.
Yes, I missed the fact that there was still a little bit of the hw-breakpoints layer underneath the perf_event layer. I'm not sure that what Frederic has now will suit those Power CPUs that have both an IABR and DABR, but we can fix that later.
Paul.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
In reply to Paul MackerrasQuoted 8 lines · Paul Mackerras
Quoted 5 lines · Frederic Weisbecker
This patch rebase the implementation of the breakpoints API on top of perf events instances.
Each breakpoints are now perf events that handle the register scheduling, thread/cpu attachment, etc..
Quoted 6 lines
What I haven't managed to understand yet is how you provide reliable breakpoints for debugging purposes. If I'm debugging a program and I have set a breakpoint, I'll be very unhappy if the breakpoint should trigger but doesn't because the
perf_eventinfrastructure has decided it can't schedule that breakpoint in. If the breakpoint isn't going to work then I want to know that at the time that I set it.
That won't happen because of the set of constraints we have. We never overcommit the debug register resources, except in the case of non-pinned counter, but that's in their nature :)
Quoted 15 lines
We can go some distance towards that with the pinned attribute, but not far enough. The pinned attribute doesn't guarantee that the event will always be scheduled in, it just says that we'll do our best to schedule it in, and if we can't, we'll put the event into error state so that the user knows we didn't manage to schedule it in. But there's no way to get back to gdb and tell it that a breakpoint that it had previously successfully created is no longer working.
Also, we don't currently fail the creation of a pinned event if it would conflict with another pinned event already created in the same context. We would need to do something like that if we want to use pinned events for debugging breakpoints (as distinct from breakpoints for performance monitoring purposes, for which it matters less if they are sometimes not scheduled in).
Quoted 8 lines
And then there's the question of whether a per-cpu pinned breakpoint event conflicts with a per-task pinned breakpoint event if you only have one breakpoint register (as is the case on Power server CPUs). Plus the fact that we don't currently give per-task pinned events priority over per-cpu non-pinned events (perhaps that would be a good idea anyway).
Paul.
All that is already handled by the constraints.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 10 lines · Frederic Weisbecker
Quoted 6 lines · Paul Mackerras
What I haven't managed to understand yet is how you provide reliable breakpoints for debugging purposes. If I'm debugging a program and I have set a breakpoint, I'll be very unhappy if the breakpoint should trigger but doesn't because the
perf_eventinfrastructure has decided it can't schedule that breakpoint in. If the breakpoint isn't going to work then I want to know that at the time that I set it.
Quoted 3 lines
That won't happen because of the set of constraints we have. We never overcommit the debug register resources, except in the case of non-pinned counter, but that's in their nature :)
Suppose you have 4 breakpoint registers per cpu and there are two pinned per-cpu breakpoint events, three non-pinned per-cpu breakpoint events, and one pinned per-task breakpoint event. I believe your constraints will allow that situation.
What will happen is that the two pinned per-cpu breakpoint events will use two of the hardware registers, and the three non-pinned per-cpu breakpoint events will get round-robined onto the other two hardware registers. The per-task breakpoint will never get to use a hardware register, because the code in perf_event.c schedules per-cpu events before it schedules per-task events (see for example perf_event_task_tick()).
We will have to make the event scheduling in kernel/perf_event.c a bit more sophisticated before we can guarantee that a pinned breakpoint event will always get to use a hardware register.
Paul.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 21 lines · Paul Mackerras
Quoted 10 lines · Frederic Weisbecker
Quoted 6 lines · Paul Mackerras
What I haven't managed to understand yet is how you provide reliable breakpoints for debugging purposes. If I'm debugging a program and I have set a breakpoint, I'll be very unhappy if the breakpoint should trigger but doesn't because the
perf_eventinfrastructure has decided it can't schedule that breakpoint in. If the breakpoint isn't going to work then I want to know that at the time that I set it.Quoted 3 lines
That won't happen because of the set of constraints we have. We never overcommit the debug register resources, except in the case of non-pinned counter, but that's in their nature :)
Suppose you have 4 breakpoint registers per cpu and there are two pinned per-cpu breakpoint events, three non-pinned per-cpu breakpoint events, and one pinned per-task breakpoint event. I believe your constraints will allow that situation.
Quoted 7 lines
What will happen is that the two pinned per-cpu breakpoint events will use two of the hardware registers, and the three non-pinned per-cpu breakpoint events will get round-robined onto the other two hardware registers. The per-task breakpoint will never get to use a hardware register, because the code in
perf_event.cschedules per-cpu events before it schedules per-task events (see for exampleperf_event_task_tick()).
Oh! :-(
Quoted 5 lines
We will have to make the event scheduling in
kernel/perf_event.ca bit more sophisticated before we can guarantee that a pinned breakpoint event will always get to use a hardware register.Paul.
Ok, so the only solution for now (a part from fixing that into perf) is to consider the non-pinned events as being pinned in the constraints.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
In reply to Frederic Weisbeckerinclude/linux/perf_event.h struct perf_event_attr { · line 222
Quoted 12 lines · Frederic Weisbecker
include/linux/perf_event.h +8
-0@@ -207,6 +212,15 @@ struct perf_event_attr {__u32 wakeup_events;/* wakeup every n events */ __u32 wakeup_watermark;/* bytes before wakeup */ }; + +union { +struct {/* Hardware breakpoint info */ +__u64 bp_addr; +__u32 bp_type; +__u32 bp_len; + }; + };
I'm wondering why you don't use attr.config for anything with breakpoint events?
Paul.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 18 lines · Paul Mackerras
include/linux/perf_event.h struct perf_event_attr { · line 222
Quoted 12 lines · Frederic Weisbecker
include/linux/perf_event.h +8
-0@@ -207,6 +212,15 @@ struct perf_event_attr {__u32 wakeup_events;/* wakeup every n events */ __u32 wakeup_watermark;/* bytes before wakeup */ }; + +union { +struct {/* Hardware breakpoint info */ +__u64 bp_addr; +__u32 bp_type; +__u32 bp_len; + }; + };I'm wondering why you don't use attr.config for anything with breakpoint events?
Quoted 1 line
Paul.
Because this is not sufficient to host the breakpoint parameters given from userspace. May be I'm missing something?
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
In reply to Frederic WeisbeckerOn Tue, Nov 03, 2009 at 08:11:12PM +0100, Frederic Weisbecker wrote:
[snipped]
arch/x86/include/asm/hw_breakpoint.h struct arch_hw_breakpoint { · line 27
Quoted 11 lines
/* Available HW breakpoint length encodings */
arch/x86/include/asm/hw_breakpoint.h +4
-4-#define HW_BREAKPOINT_LEN_1 0x40 -#define HW_BREAKPOINT_LEN_2 0x44 -#define HW_BREAKPOINT_LEN_4 0x4c -#define HW_BREAKPOINT_LEN_EXECUTE 0x40 +#define X86_BREAKPOINT_LEN_1 0x40 +#define X86_BREAKPOINT_LEN_2 0x44 +#define X86_BREAKPOINT_LEN_4 0x4c +#define X86_BREAKPOINT_LEN_EXECUTE 0x40
It had previously been suggested http://lkml.org/lkml/2009/5/28/554
that users be allowed to specify the lengths in numerals. Despite having some divergent views initially, I see that it would help minimise the amount of code required to request a breakpoint if numerals (such as 1, 2, 4 and 8 for x86_64) are allowed.
The conversion to encoded values can happen later inside the bkpt-specific code.
include/asm-generic/hw_breakpoint.h
Quoted 2 lines
include/asm-generic/hw_breakpoint.h deleted +0
-0
Can you split this patch into fine granular ones? It is very difficult to review the changes this way.
include/linux/hw_breakpoint.h
Quoted 5 lines
include/linux/hw_breakpoint.h new file +0
-0
Have you clubbed file renaming along with changes inside the file? Again, it'd be good to have them in separate patches for easy review.
include/linux/hw_breakpoint.h line 47
Quoted 9 lines
include/linux/hw_breakpoint.h new file +9
-0+#ifdef CONFIG_HAVE_HW_BREAKPOINT +extern struct perf_event * +register_user_hw_breakpoint (unsigned long addr, +int len, +int type, +perf_callback_t triggered, +struct task_struct *tsk, +bool active); +
I don't understand the benefit behind bringing these parameters into the interfaces' prototype. Besides they will make addition of new attributes (if needed later) quite cumbersome. Given that these values are eventually copied into members of perf_event_attr, I'd suggest that they accept a pointer to an instance of the structure.
Quoted 9 lines
include/linux/hw_breakpoint.h new file +9
-0+/* FIXME: only change from the attr, and don't unregister */ +extern struct perf_event * +modify_user_hw_breakpoint (struct perf_event *bp, +unsigned long addr, +int len, +int type, +perf_callback_t triggered, +struct task_struct *tsk, +bool active);
include/linux/hw_breakpoint.h line 66
Quoted 10 lines
include/linux/hw_breakpoint.h new file +10
-0+ +/* +* Kernel breakpoints are not associated with any particular thread. +*/ +extern struct perf_event * +register_wide_hw_breakpoint_cpu(unsigned long addr, + int len, + int type, + perf_callback_t triggered, + int cpu,
Can't it be cpumask_t instead of int cpu? Given that per-cpu breakpoints will be implemented, it should be very different to implement them for a subset of cpus too.
Quoted 9 lines
include/linux/hw_breakpoint.h new file +9
-0+ bool active); + +extern struct perf_event ** +register_wide_hw_breakpoint (unsigned long addr, +int len, +int type, +perf_callback_t triggered, +bool active); +
<snipped>
Quoted 19 lines
kernel/hw_breakpoint.c +3
-1@@ -15,6 +15,7 @@* * Copyright (C) 2007 Alan Stern * Copyright (C) IBM Corporation, 2009 +* Copyright (C) 2009, Frederic Weisbecker <[email protected]> */ /* @@ -35,334 +36,242 @@#include <linux/init.h> #include <linux/smp.h> -#include <asm/hw_breakpoint.h> +#include <linux/hw_breakpoint.h> +
kernel/hw_breakpoint.c line 48
Quoted 10 lines
kernel/hw_breakpoint.c +0
-5#include <asm/processor.h> #ifdef CONFIG_X86 #include <asm/debugreg.h> #endif -/* -* Spinlock that protects all (un)register operations over kernel/user-space -* breakpoint requests -*/ -static DEFINE_SPINLOCK (hw_breakpoint_lock);
Wouldn't you want to hold this lock while checking for system-wide availability of debug registers (during rollbacks) to avoid contenders from checking for the same simultaneously?
<snipped>
Quoted 15 lines
kernel/hw_breakpoint.c +12
-2-int register_kernel_hw_breakpoint (struct hw_breakpoint *bp) +struct perf_event ** +register_wide_hw_breakpoint (unsigned long addr, +int len, +int type, +perf_callback_t triggered, +bool active) { -int rc; +struct perf_event **cpu_events, **pevent, *bp; +long err; +int cpu; + + cpu_events =alloc_percpu (typeof (*cpu_events)); +if (!cpu_events)
kernel/hw_breakpoint.c line 236
Quoted 10 lines
kernel/hw_breakpoint.c +5
-3+return ERR_PTR (-ENOMEM ); - rc =arch_validate_hwbkpt_settings (bp,NULL ); -if (rc) -return rc; +for_each_possible_cpu (cpu) { + pevent =per_cpu_ptr (cpu_events, cpu); + bp =register_kernel_hw_breakpoint_cpu (addr, len, type, + triggered, cpu, active);
I'm assuming that there'd be an implementation for system-wide perf-events (and hence breakpoints) in the forthcoming version(s) of this patchset.
Quoted 33 lines
kernel/hw_breakpoint.c +17
-11-spin_lock_bh (&hw_breakpoint_lock); + *pevent = bp; - rc = -ENOSPC ; -/* Check if we are over-committing */ -if ((hbp_kernel_pos >0 ) && (!hbp_user_refcount[hbp_kernel_pos-1 ])) { - hbp_kernel_pos--; - hbp_kernel[hbp_kernel_pos] = bp; -on_each_cpu (arch_update_kernel_hw_breakpoint,NULL ,1 ); - rc =0 ; +if (IS_ERR (bp) || !bp) { + err =PTR_ERR (bp); + goto fail; + } } -spin_unlock_bh (&hw_breakpoint_lock); -return rc; +return cpu_events; + +fail: +for_each_possible_cpu (cpu) { + pevent =per_cpu_ptr (cpu_events, cpu); +if (IS_ERR (*pevent) || !*pevent) +break ; +unregister_hw_breakpoint (*pevent); + } +free_percpu (cpu_events); +/* return the error if any */ +return ERR_PTR (err); } -EXPORT_SYMBOL_GPL (register_kernel_hw_breakpoint);
<snipped>
Quoted 77 lines
kernel/perf_event.c +51
-0@@ -29,6 +29,7 @@#include <linux/kernel_stat.h> #include <linux/perf_event.h> #include <linux/ftrace_event.h> +#include <linux/hw_breakpoint.h> #include <asm/irq_regs.h> @@ -4229,6 +4230,51 @@ static void perf_event_free_filter(struct perf_event *event)#endif /* CONFIG_EVENT_PROFILE */ +#ifdef CONFIG_HAVE_HW_BREAKPOINT +static void bp_perf_event_destroy (struct perf_event *event) +{ +release_bp_slot (event); +} + +static const struct pmu *bp_perf_event_init (struct perf_event *bp) +{ +int err; +/* +* The breakpoint is already filled if we haven't created the counter +* through perf syscall +* FIXME: manage to get trigerred to NULL if it comes from syscalls +*/ +if (!bp->callback) + err =register_perf_hw_breakpoint (bp); +else + err =__register_perf_hw_breakpoint (bp); +if (err) +return ERR_PTR (err); + + bp->destroy = bp_perf_event_destroy; + +return &perf_ops_bp; +} + +void perf_bp_event (struct perf_event *bp,void *regs) +{ +/* TODO */ +} +#else +static void bp_perf_event_destroy (struct perf_event *event) +{ +} + +static const struct pmu *bp_perf_event_init (struct perf_event *bp) +{ +return NULL ; +} + +void perf_bp_event (struct perf_event *bp,void *regs) +{ +} +#endif +atomic_t perf_swevent_enabled[PERF_COUNT_SW_MAX ];static void sw_perf_event_destroy (struct perf_event *event)@@ -4375,6 +4421,11 @@ perf_event_alloc(struct perf_event_attr *attr, pmu =tp_perf_event_init (event);break ; +case PERF_TYPE_BREAKPOINT : + pmu =bp_perf_event_init (event); +break ; + +default :break ; }
kernel/perf_event.c perf_event_create_kernel_counter(struct perf_event_attr *attr, int cpu, · line 4743
Quoted 8 lines
kernel/perf_event.c +1
-1@@ -4686,7 +4737,7 @@ perf_event_create_kernel_counter(struct perf_event_attr *attr, int cpu, ctx =find_get_context (pid, cpu);if (IS_ERR (ctx)) -return NULL ; +return NULL ; event =perf_event_alloc (attr, cpu, ctx,NULL ,NULL , callback,GFP_KERNEL );
Have you tested these changes from perf-events' user-space command? Would you like to re-use the patches from here:
http://lkml.org/lkml/2009/10/29/304 to test them?
Thanks,
K.Prasad
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 20 lines · K.Prasad
On Tue, Nov 03, 2009 at 08:11:12PM +0100, Frederic Weisbecker wrote:
[snipped]arch/x86/include/asm/hw_breakpoint.h struct arch_hw_breakpoint { · line 27
Quoted 11 lines
/* Available HW breakpoint length encodings */
arch/x86/include/asm/hw_breakpoint.h +4
-4-#define HW_BREAKPOINT_LEN_1 0x40 -#define HW_BREAKPOINT_LEN_2 0x44 -#define HW_BREAKPOINT_LEN_4 0x4c -#define HW_BREAKPOINT_LEN_EXECUTE 0x40 +#define X86_BREAKPOINT_LEN_1 0x40 +#define X86_BREAKPOINT_LEN_2 0x44 +#define X86_BREAKPOINT_LEN_4 0x4c +#define X86_BREAKPOINT_LEN_EXECUTE 0x40It had previously been suggested http://lkml.org/lkml/2009/5/28/554
that users be allowed to specify the lengths in numerals. Despite having some divergent views initially, I see that it would help minimise the amount of code required to request a breakpoint if numerals (such as 1, 2, 4 and 8 forx86_64) are allowed.
Quoted 2 lines
The conversion to encoded values can happen later inside the bkpt-specific code.
That's what I did, I've redefined them in linux/hw_breakpoint.h:
#define HW_BREAKPOINT_LEN_1 1 #define HW_BREAKPOINT_LEN_2 2 #define HW_BREAKPOINT_LEN_4 4
And the arch interpret that using its own corresponding values.
Quoted 5 lines
include/asm-generic/hw_breakpoint.h
Quoted 2 lines
include/asm-generic/hw_breakpoint.h deleted +0
-0Can you split this patch into fine granular ones? It is very difficult to review the changes this way.
Sure, I personally don't like either this big monolithic patch, but it is hard/impossible to split it as we change the whole base of a subsystem inside.
But this header moving has been done in the v2 and I thought git-format-patch would detect the rename but the file has probably too much changed.
I'll do another iteration that split up this part.
Quoted 8 lines
include/linux/hw_breakpoint.h
Quoted 5 lines
include/linux/hw_breakpoint.h new file +0
-0Have you clubbed file renaming along with changes inside the file? Again, it'd be good to have them in separate patches for easy review.
There have been this rename only. But I'll split up this part.
Quoted 7 lines
Quoted 7 lines
include/linux/hw_breakpoint.h new file +7
-0+#ifdef CONFIG_HAVE_HW_BREAKPOINT +extern struct perf_event * +register_user_hw_breakpoint(unsigned long addr, + int len, + int type, + perf_callback_t triggered, + struct task_struct *tsk,
Quoted 8 lines
include/linux/hw_breakpoint.h
Quoted 2 lines
include/linux/hw_breakpoint.h new file +2
-0+ bool active); +I don't understand the benefit behind bringing these parameters into the interfaces' prototype. Besides they will make addition of new attributes (if needed later) quite cumbersome. Given that these values are eventually copied into members of
perf_event_attr, I'd suggest that they accept a pointer to an instance of the structure.
Yeah, that's a bit intended as a temporary thing. The preffered way for that would be to pass a pointer to a perf_event_attr structure.
I plan to do this change incrementally, once we have defined breakpoints attributes generic enough to support most archs possibilities.
Quoted 15 lines
Quoted 15 lines
include/linux/hw_breakpoint.h new file +15
-0+/* FIXME: only change from the attr, and don't unregister */ +extern struct perf_event * +modify_user_hw_breakpoint (struct perf_event *bp, +unsigned long addr, +int len, +int type, +perf_callback_t triggered, +struct task_struct *tsk, +bool active); + +/* +* Kernel breakpoints are not associated with any particular thread. +*/ +extern struct perf_event * +register_wide_hw_breakpoint_cpu(unsigned long addr,
Quoted 8 lines
include/linux/hw_breakpoint.h line 66
Quoted 4 lines
include/linux/hw_breakpoint.h new file +4
-0+int len, + int type, + perf_callback_t triggered, + int cpu,Can't it be
cpumask_tinstead of int cpu? Given that per-cpu breakpoints will be implemented, it should be very different to implement them for a subset of cpus too.
I can't figure out any usecase where we want to only bind to, say, cpu 1 and 3 or any kind of such strange combination.
Either we want a wide breakpoint, or we want to profile a single cpu, but I don't imagine we need a middle case.
Quoted 5 lines
Quoted 1 line
-static
DEFINE_SPINLOCK(hw_breakpoint_lock);Wouldn't you want to hold this lock while checking for system-wide availability of debug registers (during rollbacks) to avoid contenders from checking for the same simultaneously?
If we want to lock such path, we probably more likely want a mutex. Registering a breakpoint is not a fastpath and also perf does some sleepable things while creating a counter.
The check to register constraints, which is part of this path, is itself a mutex.
But we'll probably need something NMI safe in the future so that it can be used without any problem by kgdb.
Quoted 19 lines
<snipped>
Quoted 17 lines
include/linux/hw_breakpoint.h new file +13
-2-int register_kernel_hw_breakpoint (struct hw_breakpoint *bp) +struct perf_event ** +register_wide_hw_breakpoint (unsigned long addr, +int len, +int type, +perf_callback_t triggered, +bool active) { -int rc; +struct perf_event **cpu_events, **pevent, *bp; +long err; +int cpu; + + cpu_events =alloc_percpu (typeof (*cpu_events)); +if (!cpu_events) +return ERR_PTR (-ENOMEM );
Quoted 12 lines
include/linux/hw_breakpoint.h
Quoted 8 lines
include/linux/hw_breakpoint.h new file +4
-3- rc =arch_validate_hwbkpt_settings (bp,NULL ); -if (rc) -return rc; +for_each_possible_cpu (cpu) { + pevent =per_cpu_ptr (cpu_events, cpu); + bp =register_kernel_hw_breakpoint_cpu (addr, len, type, + triggered, cpu, active);I'm assuming that there'd be an implementation for system-wide perf-events (and hence breakpoints) in the forthcoming version(s) of this patchset.
If that becomes a necessary feature, then yeah.
Quoted 3 lines
Have you tested these changes from perf-events' user-space command? Would you like to re-use the patches from here:
http://lkml.org/lkml/2009/10/29/304 to test them?
Yeah, I have planned to reuse your patches for the perf subcommand support :)
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 16 lines · Frederic Weisbecker
Quoted 14 lines · K.Prasad
include/linux/hw_breakpoint.h line 66
Quoted 9 lines · Frederic Weisbecker
include/linux/hw_breakpoint.h new file +9
-0+/* +* Kernel breakpoints are not associated with any particular thread. +*/ +extern struct perf_event * +register_wide_hw_breakpoint_cpu(unsigned long addr, + int len, + int type, + perf_callback_t triggered, + int cpu,Can't it be
cpumask_tinstead of int cpu? Given that per-cpu breakpoints will be implemented, it should be very different to implement them for a subset of cpus too.
Quoted 5 lines
I can't figure out any usecase where we want to only bind to, say, cpu 1 and 3 or any kind of such strange combination.
Either we want a wide breakpoint, or we want to profile a single cpu, but I don't imagine we need a middle case.
When we originally had this discussion on LKML, one of the use-cases cited was http://lkml.org/lkml/2009/7/29/243. I can't see why such need should be restricted to a given CPU only, rather than a subset of CPUs (say 'x' is a variable normally read/written-to in the interrupt path, and if the said interrupt is has a cpu affinity to a subset of cpus only).
Although in the normal case, this feature could be implemented later, in case of breakpoints we accept that as input from the user (and hence part of the well-defined interface), so it is better to design it for a subset of CPUs from start. The logic isn't very different and given that there are plenty of helper routines in cpumask.h the implementation is easy too.
Quoted 12 lines
Quoted 6 lines
Quoted 2 lines
-static
DEFINE_SPINLOCK(hw_breakpoint_lock);Wouldn't you want to hold this lock while checking for system-wide availability of debug registers (during rollbacks) to avoid contenders from checking for the same simultaneously?
If we want to lock such path, we probably more likely want a mutex. Registering a breakpoint is not a fastpath and also perf does some sleepable things while creating a counter.
Quoted 6 lines
The check to register constraints, which is part of this path, is itself a mutex.
But we'll probably need something NMI safe in the future so that it can be used without any problem by kgdb.
I suspect that it will be required for cpu-hotplug handler, where previously load_debug_registers() was called from a softirq context.
Quoted 34 lines
Quoted 20 lines
<snipped>
Quoted 17 lines
kernel/hw_breakpoint.c +13
-2-int register_kernel_hw_breakpoint (struct hw_breakpoint *bp) +struct perf_event ** +register_wide_hw_breakpoint (unsigned long addr, +int len, +int type, +perf_callback_t triggered, +bool active) { -int rc; +struct perf_event **cpu_events, **pevent, *bp; +long err; +int cpu; + + cpu_events =alloc_percpu (typeof (*cpu_events)); +if (!cpu_events) +return ERR_PTR (-ENOMEM );Quoted 12 lines
kernel/hw_breakpoint.c line 235
Quoted 8 lines
kernel/hw_breakpoint.c +4
-3- rc =arch_validate_hwbkpt_settings (bp,NULL ); -if (rc) -return rc; +for_each_possible_cpu (cpu) { + pevent =per_cpu_ptr (cpu_events, cpu); + bp =register_kernel_hw_breakpoint_cpu (addr, len, type, + triggered, cpu, active);I'm assuming that there'd be an implementation for system-wide perf-events (and hence breakpoints) in the forthcoming version(s) of this patchset.
Quoted 3 lines
If that becomes a necessary feature, then yeah.
Apart from the several benefits of having system-wide perf-events, implementing them in the first iteration itself will help us fully realise the cost of perf-events + hw-breakpoint integration! When implemented, perf-events will also be ready to accomodate future users (apart from bp and perf-top) having a need for system-wide counter.
Thanks,
K.Prasad
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 18 lines · K.Prasad
Quoted 9 lines · Frederic Weisbecker
Quoted 3 lines
Can't it be
cpumask_tinstead of int cpu? Given that per-cpu breakpoints will be implemented, it should be very different to implement them for a subset of cpus too.I can't figure out any usecase where we want to only bind to, say, cpu 1 and 3 or any kind of such strange combination.
Either we want a wide breakpoint, or we want to profile a single cpu, but I don't imagine we need a middle case.
When we originally had this discussion on LKML, one of the use-cases cited was http://lkml.org/lkml/2009/7/29/243. I can't see why such need should be restricted to a given CPU only, rather than a subset of CPUs (say 'x' is a variable normally read/written-to in the interrupt path, and if the said interrupt is has a cpu affinity to a subset of cpus only).
Quoted 5 lines
Although in the normal case, this feature could be implemented later, in case of breakpoints we accept that as input from the user (and hence part of the well-defined interface), so it is better to design it for a subset of CPUs from start. The logic isn't very different and given that there are plenty of helper routines in
cpumask.hthe implementation is easy too.
Well, if one day someone wants to profile a subset of cpus and then need this feature, I'll implement it. But I don't think we should anticipate every possible corner usecases for now.
It's not possible to request that from any user interface anyway (either ptrace, perf or ftrace). And if it becomes needed for in-kernel use, then it's trivial to change.
Quoted 7 lines
Quoted 7 lines
If we want to lock such path, we probably more likely want a mutex. Registering a breakpoint is not a fastpath and also perf does some sleepable things while creating a counter.
The check to register constraints, which is part of this path, is itself a mutex.
Quoted 6 lines
Quoted 3 lines
But we'll probably need something NMI safe in the future so that it can be used without any problem by kgdb.
I suspect that it will be required for cpu-hotplug handler, where previously
load_debug_registers()was called from a softirq context.
Nop. There is no register/unregister on cpu hotplug time. Perf will just reschedule the events on that cpu (through pmu::enable/disable calls).
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines
I'm assuming that there'd be an implementation for system-wide perf-events (and hence breakpoints) in the forthcoming version(s) of this patchset.
If that becomes a necessary feature, then yeah.
Quoted 6 lines
Apart from the several benefits of having system-wide perf-events, implementing them in the first iteration itself will help us fully realise the cost of perf-events + hw-breakpoint integration! When implemented, perf-events will also be ready to accomodate future users (apart from bp and perf-top) having a need for system-wide counter.
For now this is meant to be costly (wrt cross cpu contention) as I explained you before.
But if the ftrace ring buffer becomes integrated by perf (which seem to be in the plans), then yeah this may become a very useful feature because we could use a single counter for wide profiling without the cost of the cpu contention (ftrace ring buffer is per cpu and fully lockless).
Thanks.
PATCH 5/6 hw-breakpoints: Arbitrate access to pmu following registers constraints
In reply to Frederic WeisbeckerAllow or refuse to build a counter using the breakpoints pmu following given constraints.
We keep track of the pmu users by using three per cpu variables:
- nr_cpu_bp_pinned stores the number of pinned cpu breakpoints counters in the given cpu
- nr_bp_flexible stores the number of non-pinned breakpoints counters in the given cpu.
- task_bp_pinned stores the number of pinned task breakpoints in a cpu
The latter is not a simple counter but gathers the number of tasks that have n pinned breakpoints.
Considering HBP_NUM the number of available breakpoint address registers:
task_bp_pinned[0] is the number of tasks having 1 breakpoint task_bp_pinned[1] is the number of tasks having 2 breakpoints [...]
task_bp_pinned[HBP_NUM - 1] is the number of tasks having the maximum number of registers (HBP_NUM).
When a breakpoint counter is created and wants an access to the pmu, we evaluate the following constraints:
== Non-pinned counter ==
- If attached to a single cpu, check:
(per_cpu(nr_bp_flexible, cpu) || (per_cpu(nr_cpu_bp_pinned, cpu)
+ max(per_cpu(task_bp_pinned, cpu)))) < HBP_NUM
-> If there are already non-pinned counters in this cpu, it
means there is already a free slot for them. Otherwise, we check that the maximum number of per task breakpoints (for this cpu) plus the number of per cpu breakpoint (for this cpu) doesn't cover every registers.
- If attached to every cpus, check:
(per_cpu(nr_bp_flexible, *) || (max(per_cpu(nr_cpu_bp_pinned, *))
+ max(per_cpu(task_bp_pinned, *)))) < HBP_NUM
-> This is roughly the same, except we check the number of per
cpu bp for every cpu and we keep the max one. Same for the per tasks breakpoints.
== Pinned counter ==
- If attached to a single cpu, check:
((per_cpu(nr_bp_flexible, cpu) > 1)
+ per_cpu(nr_cpu_bp_pinned, cpu)
+ max(per_cpu(task_bp_pinned, cpu))) < HBP_NUM
-> Same checks as before. But now the nr_bp_flexible, if any,
must keep one register at least (or flexible breakpoints will never be be fed).
- If attached to every cpus, check:
((per_cpu(nr_bp_flexible, *) > 1)
+ max(per_cpu(nr_cpu_bp_pinned, *))
+ max(per_cpu(task_bp_pinned, *))) < HBP_NUM
Changes in v2:
- Counter -> event rename
--- kernel/hw_breakpoint.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 231 insertions(+), 6 deletions(-)
kernel/hw_breakpoint.c +231 -6
@@ -16,6 +16,8 @@* Copyright (C) 2007 Alan Stern * Copyright (C) IBM Corporation, 2009 * Copyright (C) 2009, Frederic Weisbecker <[email protected]> +* +* Thanks to Ingo Molnar for his many suggestions. */ /*
@@ -44,24 +46,247 @@#include <asm/debugreg.h> #endif -static atomic_t bp_slot; +/* +* Constraints data +*/ + +/* Number of pinned cpu breakpoints in a cpu */ +static DEFINE_PER_CPU(unsigned int, nr_cpu_bp_pinned); -int reserve_bp_slot (struct perf_event *bp) +/* Number of pinned task breakpoints in a cpu */ +static DEFINE_PER_CPU(unsigned int, task_bp_pinned[HBP_NUM]); + +/* Number of non-pinned cpu/task breakpoints in a cpu */ +static DEFINE_PER_CPU(unsigned int, nr_bp_flexible); + +/* Gather the number of total pinned and un-pinned bp in a cpuset */ +struct bp_busy_slots { +unsigned int pinned; +unsigned int flexible; +}; + +/* Serialize accesses to the above constraints */ +static DEFINE_MUTEX (nr_bp_mutex); + +/* +* Report the maximum number of pinned breakpoints a task +* have in this cpu +*/ +static unsigned int max_task_bp_pinned (int cpu) { -if (atomic_inc_return (&bp_slot) ==HBP_NUM ) { -atomic_dec (&bp_slot); +int i; +unsigned int *tsk_pinned =per_cpu (task_bp_pinned, cpu); -return -ENOSPC ; +for (i =HBP_NUM -1 ; i >=0 ; i--) { +if (tsk_pinned[i] >0 ) +return i +1 ; }return 0 ; } +/* +* Report the number of pinned/un-pinned breakpoints we have in +* a given cpu (cpu > -1) or in all of them (cpu = -1). +*/ +static void fetch_bp_busy_slots (struct bp_busy_slots *slots,int cpu) +{ +if (cpu >=0 ) { + slots->pinned =per_cpu (nr_cpu_bp_pinned, cpu); + slots->pinned +=max_task_bp_pinned (cpu); + slots->flexible =per_cpu (nr_bp_flexible, cpu); + +return ; + } + +for_each_online_cpu (cpu) { +unsigned int nr; + + nr =per_cpu (nr_cpu_bp_pinned, cpu); + nr +=max_task_bp_pinned (cpu); + +if (nr > slots->pinned) + slots->pinned = nr; + + nr =per_cpu (nr_bp_flexible, cpu); + +if (nr > slots->flexible) + slots->flexible = nr; + } +} + +/* +* Add a pinned breakpoint for the given task in our constraint table +*/ +static void toggle_bp_task_slot (struct task_struct *tsk,int cpu,bool enable) +{ +int count =0 ; +struct perf_event *bp; +struct perf_event_context *ctx = tsk->perf_event_ctxp; +unsigned int *task_bp_pinned; +struct list_head *list; +unsigned long flags; + +if (WARN_ONCE (!ctx,"No perf context for this task" )) +return ; + + list = &ctx->event_list; + +spin_lock_irqsave (&ctx->lock, flags); + +/* +* The current breakpoint counter is not included in the list +* at the open() callback time +*/ +list_for_each_entry (bp, list, event_entry) { +if (bp->attr.type ==PERF_TYPE_BREAKPOINT ) + count++; + } + +spin_unlock_irqrestore (&ctx->lock, flags); + +if (WARN_ONCE (count <0 ,"No breakpoint counter found in the counter list" )) +return ; + + task_bp_pinned =per_cpu (task_bp_pinned, cpu); +if (enable) { + task_bp_pinned[count]++; +if (count >0 ) + task_bp_pinned[count-1 ]--; + }else { + task_bp_pinned[count]--; +if (count >0 ) + task_bp_pinned[count-1 ]++; + } +} + +/* +* Add/remove the given breakpoint in our constraint table +*/ +static void toggle_bp_slot (struct perf_event *bp,bool enable) +{ +int cpu = bp->cpu; +unsigned int *nr; +struct task_struct *tsk = bp->ctx->task; + +/* Flexible */ +if (!bp->attr.pinned) { +if (cpu >=0 ) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle; + } + +for_each_online_cpu (cpu) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle; + } + } +/* Pinned counter task profiling */ +if (tsk) { +if (cpu >=0 ) { +toggle_bp_task_slot (tsk, cpu, enable); +return ; + } + +for_each_online_cpu (cpu) +toggle_bp_task_slot (tsk, cpu, enable); +return ; + } + +/* Pinned counter cpu profiling */ + nr = &per_cpu (nr_cpu_bp_pinned, bp->cpu); + +toggle: + *nr = enable ? *nr +1 : *nr -1 ; +} + +/* +* Contraints to check before allowing this new breakpoint counter: +* +* == Non-pinned counter == +* +* - If attached to a single cpu, check: +* +* (per_cpu(nr_bp_flexible, cpu) || (per_cpu(nr_cpu_bp_pinned, cpu) +* + max(per_cpu(task_bp_pinned, cpu)))) < HBP_NUM +* +* -> If there are already non-pinned counters in this cpu, it means +* there is already a free slot for them. +* Otherwise, we check that the maximum number of per task +* breakpoints (for this cpu) plus the number of per cpu breakpoint +* (for this cpu) doesn't cover every registers. +* +* - If attached to every cpus, check: +* +* (per_cpu(nr_bp_flexible, *) || (max(per_cpu(nr_cpu_bp_pinned, *)) +* + max(per_cpu(task_bp_pinned, *)))) < HBP_NUM +* +* -> This is roughly the same, except we check the number of per cpu +* bp for every cpu and we keep the max one. Same for the per tasks +* breakpoints. +* +* +* == Pinned counter == +* +* - If attached to a single cpu, check: +* +* ((per_cpu(nr_bp_flexible, cpu) > 1) + per_cpu(nr_cpu_bp_pinned, cpu) +* + max(per_cpu(task_bp_pinned, cpu))) < HBP_NUM +* +* -> Same checks as before. But now the nr_bp_flexible, if any, must keep +* one register at least (or they will never be fed). +* +* - If attached to every cpus, check: +* +* ((per_cpu(nr_bp_flexible, *) > 1) + max(per_cpu(nr_cpu_bp_pinned, *)) +* + max(per_cpu(task_bp_pinned, *))) < HBP_NUM +*/ +int reserve_bp_slot (struct perf_event *bp) +{ +struct bp_busy_slots slots = {0 }; +int ret =0 ; + +mutex_lock (&nr_bp_mutex); + +fetch_bp_busy_slots (&slots, bp->cpu); + +if (!bp->attr.pinned) { +/* +* If there are already flexible counters here, +* there is at least one slot reserved for all +* of them. Just join the party. +* +* Otherwise, check there is at least one free slot +*/ +if (!slots.flexible && slots.pinned ==HBP_NUM ) { + ret = -ENOSPC ; + goto end; + } + +/* Flexible counters need to keep at least one slot */ + }else if (slots.pinned + (!!slots.flexible) ==HBP_NUM ) { + ret = -ENOSPC ; + goto end; + } + +toggle_bp_slot (bp, true); + +end: +mutex_unlock (&nr_bp_mutex); + +return ret; +} +void release_bp_slot (struct perf_event *bp) { -atomic_dec (&bp_slot); +mutex_lock (&nr_bp_mutex); + +toggle_bp_slot (bp, false); + +mutex_unlock (&nr_bp_mutex); } +int __register_perf_hw_breakpoint (struct perf_event *bp) {int ret;
Signature · 1 line
1.6.2.3
Re: [PATCH 5/6] hw-breakpoints: Arbitrate access to pmu following registers constraints
Quoted 2 lines · Frederic Weisbecker
Allow or refuse to build a counter using the breakpoints pmu following given constraints.
As far as I can see, you assume each CPU has HBP_NUM breakpoint registers which are all interchangeable and can all be used either for data breakpoints or instruction breakpoints. Is that accurate?
If so, we'll need to extend it a bit for Power since we have some CPUs that have one data breakpoint register and one instruction breakpoint register. In general on powerpc the instruction and data breakpoint facilities are separate, i.e. we have no registers that can be used for either.
Quoted 16 lines
kernel/hw_breakpoint.c +16
-0+static void toggle_bp_slot (struct perf_event *bp,bool enable) +{ +int cpu = bp->cpu; +unsigned int *nr; +struct task_struct *tsk = bp->ctx->task; + +/* Flexible */ +if (!bp->attr.pinned) { +if (cpu >=0 ) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle; + } + +for_each_online_cpu (cpu) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle;
...
kernel/hw_breakpoint.c line 201
Quoted 3 lines
kernel/hw_breakpoint.c +3
-0+toggle: + *nr = enable ? *nr +1 : *nr -1 ; +}
This won't do what I think you want. In the case where !bp->attr.pinned and cpu == -1, it will only update the count for the first online cpu, not all of them.
Paul.
Re: [PATCH 5/6] hw-breakpoints: Arbitrate access to pmu following registers constraints
Quoted 8 lines · Paul Mackerras
Quoted 2 lines · Frederic Weisbecker
Allow or refuse to build a counter using the breakpoints pmu following given constraints.
As far as I can see, you assume each CPU has
HBP_NUMbreakpoint registers which are all interchangeable and can all be used either for data breakpoints or instruction breakpoints. Is that accurate?
Yes, they are interchangeable at runtime while calling enable/disable callbacks of the pmu.
I'm not sure instruction breakpoints are supported though.
Quoted 5 lines
If so, we'll need to extend it a bit for Power since we have some CPUs that have one data breakpoint register and one instruction breakpoint register. In general on powerpc the instruction and data breakpoint facilities are separate, i.e. we have no registers that can be used for either.
Sure. I would be glad to help in that area. That said I won't be able to test anything as I don't have a PowerPc box.
Quoted 27 lines
Quoted 16 lines
kernel/hw_breakpoint.c +16
-0+static void toggle_bp_slot (struct perf_event *bp,bool enable) +{ +int cpu = bp->cpu; +unsigned int *nr; +struct task_struct *tsk = bp->ctx->task; + +/* Flexible */ +if (!bp->attr.pinned) { +if (cpu >=0 ) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle; + } + +for_each_online_cpu (cpu) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle;...
kernel/hw_breakpoint.c line 201
Quoted 3 lines
kernel/hw_breakpoint.c +3
-0+toggle: + *nr = enable ? *nr +1 : *nr -1 ; +}This won't do what I think you want. In the case where !bp->attr.pinned and cpu == -1, it will only update the count for the first online cpu, not all of them.
Quoted 1 line
Paul.
Oh right! That's really idiotic. Will fix.
Thanks!
Re: [PATCH 5/6] hw-breakpoints: Arbitrate access to pmu following registers constraints
In reply to Paul MackerrasQuoted 9 lines · Paul Mackerras
Quoted 2 lines · Frederic Weisbecker
Allow or refuse to build a counter using the breakpoints pmu following given constraints.
As far as I can see, you assume each CPU has
HBP_NUMbreakpoint registers which are all interchangeable and can all be used either for data breakpoints or instruction breakpoints. Is that accurate?
Quoted 5 lines
If so, we'll need to extend it a bit for Power since we have some CPUs that have one data breakpoint register and one instruction breakpoint register. In general on powerpc the instruction and data breakpoint facilities are separate, i.e. we have no registers that can be used for either.
Additionally, we have more fancy facilities that I don't see exposed at all through this interface (we are building an ad-hoc ptrace based interface today so that gdb can make use of them) and we have one guy with crazy constraints that we don't know yet how to deal with:
Among others features:
- Pairing of two data or instruction breakpoints to create a ranges
breakpoint
- Data value compare option
- Instruction value compare option
And now the crazy constraints:
- On one embedded core at least we have a case where the core has 4
threads, but the data (4) and instruction (2) breakpoint registers are shared. The 'enable' bits are split so a given data breakpoint can be enabled only on some HW threads but that's about it.
I'm not sure if there's a realistic way to handle the later constraint though other than just not allowing use of the HW breakpoint function on those cores at all.
Ben.
Quoted 33 lines
Quoted 16 lines
kernel/hw_breakpoint.c +16
-0+static void toggle_bp_slot (struct perf_event *bp,bool enable) +{ +int cpu = bp->cpu; +unsigned int *nr; +struct task_struct *tsk = bp->ctx->task; + +/* Flexible */ +if (!bp->attr.pinned) { +if (cpu >=0 ) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle; + } + +for_each_online_cpu (cpu) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle;...
kernel/hw_breakpoint.c line 201
Quoted 3 lines
kernel/hw_breakpoint.c +3
-0+toggle: + *nr = enable ? *nr +1 : *nr -1 ; +}This won't do what I think you want. In the case where !bp->attr.pinned and cpu == -1, it will only update the count for the first online cpu, not all of them.
Paul.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 5/6] hw-breakpoints: Arbitrate access to pmu following registers constraints
Quoted 21 lines · Benjamin Herrenschmidt
Quoted 9 lines · Paul Mackerras
Quoted 2 lines · Frederic Weisbecker
Allow or refuse to build a counter using the breakpoints pmu following given constraints.
As far as I can see, you assume each CPU has
HBP_NUMbreakpoint registers which are all interchangeable and can all be used either for data breakpoints or instruction breakpoints. Is that accurate?Quoted 5 lines
If so, we'll need to extend it a bit for Power since we have some CPUs that have one data breakpoint register and one instruction breakpoint register. In general on powerpc the instruction and data breakpoint facilities are separate, i.e. we have no registers that can be used for either.
Additionally, we have more fancy facilities that I don't see exposed at all through this interface (we are building an ad-hoc ptrace based interface today so that gdb can make use of them) and we have one guy with crazy constraints that we don't know yet how to deal with:
Quoted 6 lines
Among others features:
- Pairing of two data or instruction breakpoints to create a ranges
breakpoint
- Data value compare option
- Instruction value compare option
Yeah. The current generic interface is a draft. I'll try to write something generic enough to fit in every archs needs. This is needed before we expose its perf interface to userpace anyway.
Quoted 7 lines
And now the crazy constraints:
- On one embedded core at least we have a case where the core has 4
threads, but the data (4) and instruction (2) breakpoint registers are shared. The 'enable' bits are split so a given data breakpoint can be enabled only on some HW threads but that's about it.
Quoted 5 lines
I'm not sure if there's a realistic way to handle the later constraint though other than just not allowing use of the HW breakpoint function on those cores at all.
Ben.
Yeah this latter one is tricky. Not sure how to handle it either. How are these hw-threads considered by the kernel core? As different cpu?
Re: [PATCH 5/6] hw-breakpoints: Arbitrate access to pmu following registers constraints
Quoted 9 lines · Frederic Weisbecker
Quoted 9 lines
- On one embedded core at least we have a case where the core has 4
threads, but the data (4) and instruction (2) breakpoint registers are shared. The 'enable' bits are split so a given data breakpoint can be enabled only on some HW threads but that's about it.I'm not sure if there's a realistic way to handle the later constraint though other than just not allowing use of the HW breakpoint function on those cores at all.
Quoted 6 lines
Quoted 1 line
Ben.
Yeah this latter one is tricky. Not sure how to handle it either. How are these hw-threads considered by the kernel core? As different cpu?
Yes.
So it basically looks like you have 4 data and 2 HW instruction breakpoint registers shared by 4 CPUs in a group :-)
Cheers,
Ben.
Re: [PATCH 5/6] hw-breakpoints: Arbitrate access to pmu following registers constraints
Quoted 17 lines · Benjamin Herrenschmidt
Quoted 9 lines · Frederic Weisbecker
Quoted 9 lines
- On one embedded core at least we have a case where the core has 4
threads, but the data (4) and instruction (2) breakpoint registers are shared. The 'enable' bits are split so a given data breakpoint can be enabled only on some HW threads but that's about it.I'm not sure if there's a realistic way to handle the later constraint though other than just not allowing use of the HW breakpoint function on those cores at all.
Quoted 6 lines
Quoted 1 line
Ben.
Yeah this latter one is tricky. Not sure how to handle it either. How are these hw-threads considered by the kernel core? As different cpu?
Quoted 9 lines
Yes.
So it basically looks like you have 4 data and 2 HW instruction breakpoint registers shared by 4 CPUs in a group :-)
Cheers,
Ben.
That's not a simple situation :)
I guess we'll need to let powerpc handle the constraints from the arch.
PATCH 6/6 ksym_tracer: Remove KSYM_SELFTEST_ENTRY
In reply to Frederic WeisbeckerFrom: Li Zefan <[email protected]>
The macro used to be used in both trace_selftest.c and trace_ksym.c, but no longer, so remove it from header file.
--- kernel/trace/trace.h | 1 - kernel/trace/trace_selftest.c | 2 +- 2 files changed, 1 insertions(+), 2 deletions(-)
kernel/trace/trace.h +0 -1
@@ -370,7 +370,6 @@ int register_tracer(struct tracer *type);void unregister_tracer (struct tracer *type);int is_tracing_stopped (void ); -#define KSYM_SELFTEST_ENTRY "ksym_selftest_dummy"extern int process_new_ksym_entry (char *ksymname,int op,unsigned long addr);extern unsigned long nsecs_to_usecs (unsigned long nsecs);
kernel/trace/trace_selftest.c +1 -1
@@ -828,7 +828,7 @@ trace_selftest_startup_ksym(struct tracer *trace, struct trace_array *tr) ksym_selftest_dummy =0 ;/* Register the read-write tracing request */ - ret =process_new_ksym_entry (KSYM_SELFTEST_ENTRY ,HW_BREAKPOINT_RW , + ret =process_new_ksym_entry ("ksym_selftest_dummy" ,HW_BREAKPOINT_RW , (unsigned long )(&ksym_selftest_dummy));if (ret <0 ) {
Signature · 1 line
1.6.2.3
Quoted 6 lines · Frederic Weisbecker
Hi all,
This is the v4 of the hw-breakpoints API rewrite on top of perf events.
This integrates the following fixes:
Is there a missing patch 2/6 or are they numbered incorrectly? I see a patch 1/6 followed by 3/6.
Thanks,
K.Prasad
Quoted 5 lines · K.Prasad
Quoted 4 lines · Frederic Weisbecker
Hi all,
This is the v4 of the hw-breakpoints API rewrite on top of perf events.
Quoted 9 lines
Quoted 2 lines
This integrates the following fixes:
Is there a missing patch 2/6 or are they numbered incorrectly? I see a patch 1/6 followed by 3/6.
Thanks,
K.Prasad
Yeah, that's the problem with git-send-email, it appends the emails in the Cc list using the Cc: tags but it doesn't take the other tags (may be just the Signed-off-by).
And you've acked this one, but I haven't put you in the Cc tag. I should have, sorry. You can find it there: