hw-breakpoints: Rewrite on top of perf events
Subsystems (7)
Hi all,
This is the v2 of the hw-breakpoints API rewrite on top of perf events. You can find the previous version here:
http://lwn.net/Articles/351922/
Changes in v2:
- Follow the perf "event " rename
- The ptrace regression have been fixed (ptrace breakpoint perf events weren't released when a task ended)
- Drop the struct hw_breakpoint and store generic fields in perf_event_attr.
- Separate core and arch specific headers, drop asm-generic/hw_breakpoint.h and create linux/hw_breakpoint.h
- Use new generic len/type for breakpoint
- Handle off case: when breakpoints api is not supported by an arch
- Use proper in-kernel perf api provided by Arjan.
There are still a lot of things that need to be cleaned, simplified, improved (ptrace side, the bp api, etc....) I guess these things can be done incrementally if you agree.
I've also tried to get an arch-independent api. Generic fields for breakpoints are stored in perf_event_attr structure (type, len, addr). This needs to be discussed and improved before it becomes a perf userspace ABI. We need to find a generic enough structure to host the breakpoints parameters, something that can better fit to most arch (handling breakpoint ranges in powerpc, etc...).
Thanks.
---
The following patchset are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
perfevents/hw-breakpoint
Arjan van de Ven (1):
perf/core: Provide a kernel-internal interface to get to performance counters
Frederic Weisbecker (3):
perf/core: Add a callback to perf events
hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
hw-breakpoints: Arbitrate access to pmu following registers constraints
Li Zefan (1):
ksym_tracer: Remove KSYM_SELFTEST_ENTRY
Paul Mundt (1):
x86/hw-breakpoints: Actually flush thread breakpoints in flush_thread().
arch/Kconfig | 3 + arch/x86/include/asm/debugreg.h | 7 - arch/x86/include/asm/hw_breakpoint.h | 58 +++-- arch/x86/include/asm/processor.h | 12 +- arch/x86/kernel/hw_breakpoint.c | 376 ++++++++++++++-------- arch/x86/kernel/process.c | 9 +- arch/x86/kernel/process_32.c | 26 +-- arch/x86/kernel/process_64.c | 26 +-- arch/x86/kernel/ptrace.c | 182 +++++++---- arch/x86/kernel/smpboot.c | 3 - arch/x86/power/cpu.c | 6 - include/asm-generic/hw_breakpoint.h | 139 -------- include/linux/hw_breakpoint.h | 131 ++++++++ include/linux/perf_event.h | 37 ++- kernel/exit.c | 5 + kernel/hw_breakpoint.c | 595 +++++++++++++++++++++------------- kernel/perf_event.c | 137 ++++++++- kernel/trace/trace.h | 1 - kernel/trace/trace_entries.h | 6 +- kernel/trace/trace_ksym.c | 126 ++++---- kernel/trace/trace_selftest.c | 2 +- 21 files changed, 1154 insertions(+), 733 deletions(-) delete mode 100644 include/asm-generic/hw_breakpoint.h create mode 100644 include/linux/hw_breakpoint.h
PATCH 1/6 perf/core: Provide a kernel-internal interface to get to performance counters
From: Arjan van de Ven <[email protected]>
There are reasons for kernel code to ask for, and use, performance counters.
For example, in CPU freq governors this tends to be a good idea, but there are other examples possible as well of course.
This patch adds the needed bits to do enable this functionality; they have been tested in an experimental cpufreq driver that I'm working on, and the changes are all that I needed to access counters properly.
[[email protected]: added pid to perf_event_create_kernel_counter so that we can profile a particular task too
TODO: Have a better error reporting, don't just return NULL in fail case.]
LKML-Reference: <[email protected]>
--- include/linux/perf_event.h | 6 ++++ kernel/perf_event.c | 72 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 77 insertions(+), 1 deletions(-)
include/linux/perf_event.h +6 -0
@@ -744,6 +744,12 @@ extern int hw_perf_group_sched_in(struct perf_event *group_leader,struct perf_cpu_context *cpuctx, struct perf_event_context *ctx, int cpu);extern void perf_event_update_userpage (struct perf_event *event); +extern int perf_event_release_kernel (struct perf_event *event); +extern struct perf_event * +perf_event_create_kernel_counter (struct perf_event_attr *attr, +int cpu, +pid_t pid); +extern u64 perf_event_read_value (struct perf_event *event);struct perf_sample_data {u64 type;
kernel/perf_event.c +71 -1
96 lines not on this page · raw message
Signature · 1 line
1.6.2.3
PATCH 2/6 x86/hw-breakpoints: Actually flush thread breakpoints in flush_thread().
In reply to Frederic WeisbeckerFrom: Paul Mundt <[email protected]>
flush_thread() tries to do a TIF_DEBUG check before calling in to flush_thread_hw_breakpoint() (which subsequently clears the thread flag), but for some reason, the x86 code is manually clearing TIF_DEBUG immediately before the test, so this path will never be taken.
This kills off the erroneous clear_tsk_thread_flag() and lets flush_thread_hw_breakpoint() actually get invoked.
Presumably folks were getting lucky with testing and the free_thread_info() -> free_thread_xstate() path was taking care of the flush there.
LKML-Reference: <[email protected]>
--- arch/x86/kernel/process.c | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-)
arch/x86/kernel/process.c +0 -2
@@ -107,8 +107,6 @@ void flush_thread(void) }#endif -clear_tsk_thread_flag (tsk,TIF_DEBUG ); -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG )))flush_thread_hw_breakpoint (tsk);memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array));
Signature · 1 line
1.6.2.3
PATCH 3/6 perf/core: Add a callback to perf events
In reply to Frederic WeisbeckerA simple callback in a perf event can be used for multiple purposes. For example it is useful for triggered based events like hardware breakpoints that need a callback to dispatch a triggered breakpoint event.
--- include/linux/perf_event.h | 7 ++++++- kernel/perf_event.c | 18 ++++++++++++++---- 2 files changed, 20 insertions(+), 5 deletions(-)
include/linux/perf_event.h +6 -1
@@ -544,6 +544,8 @@ struct perf_pending_entry {void (*func)(struct perf_pending_entry *); }; +typedef void (*perf_callback_t )(struct perf_event *,void *); +/** * struct perf_event - performance event kernel representation: */
@@ -639,6 +641,8 @@ struct perf_event {struct event_filter *filter;#endif +perf_callback_t callback; +#endif /* CONFIG_PERF_EVENTS */ };
@@ -748,7 +752,8 @@ extern int perf_event_release_kernel(struct perf_event *event);extern struct perf_event *perf_event_create_kernel_counter (struct perf_event_attr *attr,int cpu, -pid_t pid); +pid_t pid, +perf_callback_t callback);extern u64 perf_event_read_value (struct perf_event *event);struct perf_sample_data {
kernel/perf_event.c +14 -4
@@ -4293,6 +4293,7 @@ perf_event_alloc(struct perf_event_attr *attr,struct perf_event_context *ctx, struct perf_event *group_leader, struct perf_event *parent_event, + perf_callback_t callback, gfp_t gfpflags) { const struct pmu *pmu;
@@ -4335,6 +4336,15 @@ perf_event_alloc(struct perf_event_attr *attr, event->state =PERF_EVENT_STATE_INACTIVE ; +if (!callback) { +if (parent_event) + event->callback = parent_event->callback; +else + event->callback =NULL ; + }else { + event->callback = callback; + } +if (attr->disabled) event->state =PERF_EVENT_STATE_OFF ;
@@ -4611,7 +4621,7 @@ SYSCALL_DEFINE5(perf_event_open, } event =perf_event_alloc (&attr, cpu, ctx, group_leader, -NULL ,GFP_KERNEL ); +NULL ,NULL ,GFP_KERNEL ); err =PTR_ERR (event);if (IS_ERR (event)) goto err_put_context;
@@ -4665,7 +4675,7 @@ err_put_context:*/ struct perf_event *perf_event_create_kernel_counter (struct perf_event_attr *attr,int cpu, -pid_t pid) +pid_t pid,perf_callback_t callback) {struct perf_event *event;struct perf_event_context *ctx;
@@ -4680,7 +4690,7 @@ perf_event_create_kernel_counter(struct perf_event_attr *attr, int cpu,return NULL ; event =perf_event_alloc (attr, cpu, ctx,NULL , -NULL ,GFP_KERNEL ); +NULL , callback,GFP_KERNEL ); err =PTR_ERR (event);if (IS_ERR (event)) goto err_put_context;
@@ -4733,7 +4743,7 @@ inherit_event(struct perf_event *parent_event, child_event =perf_event_alloc (&parent_event->attr, parent_event->cpu, child_ctx, group_leader, parent_event, -GFP_KERNEL ); +NULL ,GFP_KERNEL );if (IS_ERR (child_event))return child_event;get_ctx (child_ctx);
Signature · 1 line
1.6.2.3
PATCH 4/6 hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
In reply to Frederic WeisbeckerThis patch rebase the implementation of the breakpoints API on top of perf events instances.
Each breakpoints are now perf events that handle the register scheduling, thread/cpu attachment, etc..
The new layering is now made as follows:
ptrace kgdb ftrace perf syscall
\ | / /
\ | / /
/
Core breakpoint API /
/
| /
| /
Breakpoints perf events
|
|
Breakpoints PMU ---- Debug Register constraints handling
(Part of core breakpoint API)
|
|
Hardware debug registers
Reasons of this rewrite:
- Use the centralized/optimized pmu registers scheduling, implying an easier arch integration
- More powerful register handling: perf attributes (pinned/flexible events, exclusive/non-exclusive, tunable period, etc...)
Impact:
- New perf ABI: the hardware breakpoints counters
- Ptrace breakpoints setting remains tricky and still needs some per thread breakpoints references.
Todo (in the order):
- Support breakpoints perf counter events for perf tools (ie: implement perf_bpcounter_event())
- Support from perf tools
Changes in v2:
- Follow the perf "event " rename
- The ptrace regression have been fixed (ptrace breakpoint perf events weren't released when a task ended)
- Drop the struct hw_breakpoint and store generic fields in perf_event_attr.
- Separate core and arch specific headers, drop asm-generic/hw_breakpoint.h and create linux/hw_breakpoint.h
- Use new generic len/type for breakpoint
- Handle off case: when breakpoints api is not supported by an arch
--- arch/Kconfig | 3 + arch/x86/include/asm/debugreg.h | 7 - arch/x86/include/asm/hw_breakpoint.h | 58 +++-- arch/x86/include/asm/processor.h | 12 +- arch/x86/kernel/hw_breakpoint.c | 376 +++++++++++++++++++----------- arch/x86/kernel/process.c | 7 +- arch/x86/kernel/process_32.c | 26 +-- arch/x86/kernel/process_64.c | 26 +-- arch/x86/kernel/ptrace.c | 182 ++++++++++----- arch/x86/kernel/smpboot.c | 3 - arch/x86/power/cpu.c | 6 - include/asm-generic/hw_breakpoint.h | 139 ----------- include/linux/hw_breakpoint.h | 131 +++++++++++ include/linux/perf_event.h | 26 ++- kernel/exit.c | 5 + kernel/hw_breakpoint.c | 424 ++++++++++++++-------------------- kernel/perf_event.c | 53 ++++- kernel/trace/trace_entries.h | 6 +- kernel/trace/trace_ksym.c | 126 +++++------ 19 files changed, 862 insertions(+), 754 deletions(-) delete mode 100644 include/asm-generic/hw_breakpoint.h create mode 100644 include/linux/hw_breakpoint.h
arch/Kconfig +3 -0
@@ -128,6 +128,9 @@ config HAVE_DEFAULT_NO_SPIN_MUTEXES config HAVE_HW_BREAKPOINT bool + depends on HAVE_PERF_EVENTS + select ANON_INODES + select PERF_EVENTS source "kernel/gcov/Kconfig"
arch/x86/include/asm/debugreg.h +0 -7
@@ -75,13 +75,6 @@*/ #ifdef __KERNEL__ -/* For process management */ -extern void flush_thread_hw_breakpoint (struct task_struct *tsk); -extern int copy_thread_hw_breakpoint (struct task_struct *tsk, -struct task_struct *child,unsigned long clone_flags); - -/* For CPU management */ -extern void load_debug_registers (void );static inline void hw_breakpoint_disable (void ) {/* Zero the control register for HW Breakpoint */
arch/x86/include/asm/hw_breakpoint.h +38 -20
88 lines not on this page · raw message
arch/x86/include/asm/processor.h +6 -6
@@ -423,6 +423,8 @@ extern unsigned int xstate_size;extern void free_thread_xstate (struct task_struct *);extern struct kmem_cache *task_xstate_cachep; +struct perf_event ; +struct thread_struct {/* Cached TLS descriptors: */ struct desc_struct tls_array[GDT_ENTRY_TLS_ENTRIES ];
@@ -444,12 +446,10 @@ struct thread_struct {unsigned long fs;#endif unsigned long gs; -/* Hardware debugging registers: */ -unsigned long debugreg[HBP_NUM ]; -unsigned long debugreg6; -unsigned long debugreg7; -/* Hardware breakpoint info */ -struct hw_breakpoint *hbp[HBP_NUM ]; +/* Save middle states of ptrace breakpoints */ +struct perf_event *ptrace_bps[HBP_NUM ]; +/* Debug status used for traps, single steps, etc... */ +unsigned long debugreg6;/* Fault info: */ unsigned long cr2;unsigned long trap_no;
arch/x86/kernel/hw_breakpoint.c +239 -137
532 lines not on this page · raw message
arch/x86/kernel/process.c +2 -5
@@ -10,6 +10,7 @@#include <linux/clockchips.h> #include <linux/random.h> #include <trace/events/power.h> +#include <linux/hw_breakpoint.h> #include <asm/system.h> #include <asm/apic.h> #include <asm/syscalls.h>
@@ -18,7 +19,6 @@#include <asm/i387.h> #include <asm/ds.h> #include <asm/debugreg.h> -#include <asm/hw_breakpoint.h> unsigned long idle_halt;EXPORT_SYMBOL (idle_halt);
@@ -47,8 +47,6 @@ void free_thread_xstate(struct task_struct *tsk)kmem_cache_free (task_xstate_cachep, tsk->thread.xstate); tsk->thread.xstate =NULL ; } -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG ))) -flush_thread_hw_breakpoint (tsk);WARN (tsk->thread.ds_ctx,"leaking DS context\n" ); }
@@ -107,8 +105,7 @@ void flush_thread(void) }#endif -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG ))) -flush_thread_hw_breakpoint (tsk); +flush_ptrace_hw_breakpoint (tsk);memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array));/* * Forget coprocessor state..
arch/x86/kernel/process_32.c +2 -24
@@ -59,7 +59,6 @@#include <asm/syscalls.h> #include <asm/ds.h> #include <asm/debugreg.h> -#include <asm/hw_breakpoint.h> asmlinkage void ret_from_fork(void) __asm__("ret_from_fork");
@@ -264,9 +263,8 @@ int copy_thread(unsigned long clone_flags, unsigned long sp, p->thread.io_bitmap_ptr =NULL ; tsk = current; err = -ENOMEM ; -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG ))) -if (copy_thread_hw_breakpoint (tsk, p, clone_flags)) - goto out; + +memset (p->thread.ptrace_bps,0 ,sizeof (p->thread.ptrace_bps));if (unlikely (test_tsk_thread_flag (tsk,TIF_IO_BITMAP ))) { p->thread.io_bitmap_ptr = kmemdup(tsk->thread.io_bitmap_ptr,
@@ -287,13 +285,10 @@ int copy_thread(unsigned long clone_flags, unsigned long sp, err =do_set_thread_area (p,-1 , (struct user_desc __user *)childregs->si,0 ); -out:if (err && p->thread.io_bitmap_ptr) {kfree (p->thread.io_bitmap_ptr); p->thread.io_bitmap_max =0 ; } -if (err) -flush_thread_hw_breakpoint (p);clear_tsk_thread_flag (p,TIF_DS_AREA_MSR ); p->thread.ds_ctx =NULL ;
@@ -437,23 +432,6 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)lazy_load_gs (next->gs);percpu_write (current_task, next_p); -/* -* There's a problem with moving the arch_install_thread_hw_breakpoint() -* call before current is updated. Suppose a kernel breakpoint is -* triggered in between the two, the hw-breakpoint handler will see that -* the 'current' task does not have TIF_DEBUG flag set and will think it -* is leftover from an old task (lazy switching) and will erase it. Then -* until the next context switch, no user-breakpoints will be installed. -* -* The real problem is that it's impossible to update both current and -* physical debug registers at the same instant, so there will always be -* a window in which they disagree and a breakpoint might get triggered. -* Since we use lazy switching, we are forced to assume that a -* disagreement means that current is correct and the exception is due -* to lazy debug register switching. -*/ -if (unlikely (test_tsk_thread_flag (next_p,TIF_DEBUG ))) -arch_install_thread_hw_breakpoint (next_p);return prev_p; }
arch/x86/kernel/process_64.c +1 -25
@@ -53,7 +53,6 @@#include <asm/syscalls.h> #include <asm/ds.h> #include <asm/debugreg.h> -#include <asm/hw_breakpoint.h> asmlinkage extern void ret_from_fork(void);
@@ -244,8 +243,6 @@ void release_thread(struct task_struct *dead_task)BUG (); } } -if (unlikely (dead_task->thread.debugreg7)) -flush_thread_hw_breakpoint (dead_task); }static inline void set_32bit_tls (struct task_struct *t,int tls,u32 addr)
@@ -309,9 +306,7 @@ int copy_thread(unsigned long clone_flags, unsigned long sp,savesegment (ds, p->thread.ds); err = -ENOMEM ; -if (unlikely (test_tsk_thread_flag (me,TIF_DEBUG ))) -if (copy_thread_hw_breakpoint (me, p, clone_flags)) - goto out; +memset (p->thread.ptrace_bps,0 ,sizeof (p->thread.ptrace_bps));if (unlikely (test_tsk_thread_flag (me,TIF_IO_BITMAP ))) { p->thread.io_bitmap_ptr =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL );
@@ -351,8 +346,6 @@ out:kfree (p->thread.io_bitmap_ptr); p->thread.io_bitmap_max =0 ; } -if (err) -flush_thread_hw_breakpoint (p);return err; }
@@ -508,23 +501,6 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)*/ if (preload_fpu)__math_state_restore (); -/* -* There's a problem with moving the arch_install_thread_hw_breakpoint() -* call before current is updated. Suppose a kernel breakpoint is -* triggered in between the two, the hw-breakpoint handler will see that -* the 'current' task does not have TIF_DEBUG flag set and will think it -* is leftover from an old task (lazy switching) and will erase it. Then -* until the next context switch, no user-breakpoints will be installed. -* -* The real problem is that it's impossible to update both current and -* physical debug registers at the same instant, so there will always be -* a window in which they disagree and a breakpoint might get triggered. -* Since we use lazy switching, we are forced to assume that a -* disagreement means that current is correct and the exception is due -* to lazy debug register switching. -*/ -if (unlikely (test_tsk_thread_flag (next_p,TIF_DEBUG ))) -arch_install_thread_hw_breakpoint (next_p);return prev_p; }
arch/x86/kernel/ptrace.c +125 -57
257 lines not on this page · raw message
arch/x86/kernel/smpboot.c +0 -3
@@ -64,7 +64,6 @@#include <asm/apic.h> #include <asm/setup.h> #include <asm/uv/uv.h> -#include <asm/debugreg.h> #include <linux/mc146818rtc.h> #include <asm/smpboot_hooks.h>
@@ -328,7 +327,6 @@ notrace static void __cpuinit start_secondary(void *unused) x86_cpuinit.setup_percpu_clockev ();wmb (); -load_debug_registers ();cpu_idle (); }
@@ -1269,7 +1267,6 @@ void cpu_disable_common(void)remove_cpu_from_maps (cpu);unlock_vector_lock ();fixup_irqs (); -hw_breakpoint_disable (); }int native_cpu_disable (void )
arch/x86/power/cpu.c +0 -6
@@ -105,7 +105,6 @@ static void __save_processor_state(struct saved_context *ctxt) ctxt->cr4 =read_cr4 (); ctxt->cr8 =read_cr8 ();#endif -hw_breakpoint_disable (); }/* Needed by apm.c */
@@ -144,11 +143,6 @@ static void fix_processor_context(void)#endif load_TR_desc ();/* This does ltr */ load_LDT (¤t->active_mm->context);/* This does lldt */ - -/* -* Now maybe reload the debug registers -*/ -load_debug_registers (); }/**
include/asm-generic/hw_breakpoint.h deleted +0 -139
139 lines not on this page · raw message
include/linux/hw_breakpoint.h new file +131 -0
131 lines not on this page · raw message
include/linux/perf_event.h +25 -1
@@ -18,6 +18,10 @@#include <linux/ioctl.h> #include <asm/byteorder.h> +#ifdef CONFIG_HAVE_HW_BREAKPOINT +#include <asm/hw_breakpoint.h> +#endif +/* * User-space ABI bits: */
@@ -31,6 +35,7 @@ enum perf_type_id {PERF_TYPE_TRACEPOINT =2 ,PERF_TYPE_HW_CACHE =3 ,PERF_TYPE_RAW =4 , +PERF_TYPE_BREAKPOINT =5 , PERF_TYPE_MAX, /* non-ABI */ };
@@ -207,6 +212,15 @@ struct perf_event_attr {__u32 wakeup_events;/* wakeup every n events */ __u32 wakeup_watermark;/* bytes before wakeup */ }; + +union { +struct {/* Hardware breakpoint info */ +__u64 bp_addr; +__u32 bp_type; +__u32 bp_len; + }; + }; +__u32 __reserved_2;__u64 __reserved_3;
@@ -476,6 +490,11 @@ struct hw_perf_event {atomic64_t count;struct hrtimer hrtimer; }; +#ifdef CONFIG_HAVE_HW_BREAKPOINT +union {/* breakpoint */ +struct arch_hw_breakpoint info; + }; +#endif };atomic64_t prev_count;u64 sample_period;
@@ -588,7 +607,7 @@ struct perf_event {u64 tstamp_running;u64 tstamp_stopped; -struct perf_event_attr attr; +struct perf_event_attr attr;struct hw_perf_event hw;struct perf_event_context *ctx;
@@ -643,6 +662,8 @@ struct perf_event {perf_callback_t callback; +perf_callback_t event_callback; +#endif /* CONFIG_PERF_EVENTS */ };
@@ -831,6 +852,7 @@ extern int sysctl_perf_event_sample_rate;extern void perf_event_init (void );extern void perf_tp_event (int event_id,u64 addr,u64 count,void *record,int entry_size); +extern void perf_bp_event (struct perf_event *event,void *data);#ifndef perf_misc_flags#define perf_misc_flags (regs) (user_mode(regs) ? PERF_RECORD_MISC_USER : \
@@ -865,6 +887,8 @@ static inline int perf_event_task_enable(void) { return -EINVAL; }static inline void perf_sw_event (u32 event_id,u64 nr,int nmi,struct pt_regs *regs,u64 addr) { } +static inline void +perf_bp_event (struct perf_event *event,void *data) { }static inline void perf_event_mmap (struct vm_area_struct *vma) { }static inline void perf_event_comm (struct task_struct *tsk) { }
kernel/exit.c +5 -0
@@ -49,6 +49,7 @@#include <linux/init_task.h> #include <linux/perf_event.h> #include <trace/events/sched.h> +#include <linux/hw_breakpoint.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -980,6 +981,10 @@ NORET_TYPE void do_exit(long code)proc_exit_connector (tsk);/* +* FIXME: do that only when needed, using sched_exit tracepoint +*/ +flush_ptrace_hw_breakpoint (tsk); +/* * Flush inherited counters to the parent - before the parent * gets woken up by child-exit notifications. */
kernel/hw_breakpoint.c +170 -254
515 lines not on this page · raw message
kernel/perf_event.c +52 -1
77 lines not on this page · raw message
kernel/trace/trace_entries.h +3 -3
@@ -372,11 +372,11 @@ FTRACE_ENTRY(ksym_trace, ksym_trace_entry, F_STRUCT( __field( unsigned long, ip ) __field( unsigned char, type ) - __array( char , ksym_name, KSYM_NAME_LEN ) __array( char , cmd, TASK_COMM_LEN ) + __field( unsigned long, addr ) ), - F_printk("ip: %pF type: %d ksym_name: %s cmd: %s", + F_printk("ip: %pF type: %d ksym_name: %pS cmd: %s", (void *)__entry->ip, (unsigned int )__entry->type, - __entry->ksym_name, __entry->cmd) + (void *)__entry->addr, __entry->cmd) );
kernel/trace/trace_ksym.c +60 -66
241 lines not on this page · raw message
Signature · 1 line
1.6.2.3
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 36 lines · Frederic Weisbecker
This patch rebase the implementation of the breakpoints API on top of perf events instances.
Each breakpoints are now perf events that handle the register scheduling, thread/cpu attachment, etc..
The new layering is now made as follows:
ptrace kgdb ftrace perf syscall
\ | / /
\ | / /
/
Core breakpoint API /
/
| /
| /Breakpoints perf events
|
|Breakpoints PMU ---- Debug Register constraints handling
(Part of core breakpoint API)
|
|Hardware debug registers
Reasons of this rewrite:
- Use the centralized/optimized pmu registers scheduling, implying an easier arch integration
- More powerful register handling: perf attributes (pinned/flexible events, exclusive/non-exclusive, tunable period, etc...)
Quoted 5 lines
Impact:
- New perf ABI: the hardware breakpoints counters
- Ptrace breakpoints setting remains tricky and still needs some per thread breakpoints references.
- Broken CONFIG_KVM
Quoted 3 lines
Todo (in the order):
- Unbreak CONFIG_KVM :)
Quoted 3 lines
- Support breakpoints perf counter events for perf tools (ie: implement
perf_bpcounter_event())
- Support from perf tools
Since commit 3d53c27d05, KVM uses current->thread.debugregs for restoring the host state in case the guest played with breakpoints. We need an equivalent interface to restore ptrace breakpoints and all others currently in use.
Jan
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
2009/10/24 Jan Kiszka <[email protected]>:
Quoted 6 lines
Since commit
3d53c27d05, KVM usescurrent->thread.debugregsfor restoring the host state in case the guest played with breakpoints. We need an equivalent interface to restore ptrace breakpoints and all others currently in use.Jan
Well, dr6 is still stored in the current thread.
dr7 has a per cpu variable containing its value.
So what remains is to have a per cpu variable for dr0-3 which is updated when perf schedules in/out a profiled context. I can do that in a v3.
Thanks.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 9 lines · Frederic Weisbecker
2009/10/24 Jan Kiszka <[email protected]>:
Quoted 6 lines
Since commit
3d53c27d05, KVM usescurrent->thread.debugregsfor restoring the host state in case the guest played with breakpoints. We need an equivalent interface to restore ptrace breakpoints and all others currently in use.Jan
Quoted 6 lines
Well, dr6 is still stored in the current thread.
dr7 has a per cpu variable containing its value.So what remains is to have a per cpu variable for dr0-3 which is updated when perf schedules in/out a profiled context. I can do that in a v3.
Sounds great. Maybe you can stuff all this into some function kvm can call to avoid that it has to peek into various internal structures.
Jan
Hi all,
This is the v3 of the hw-breakpoints API rewrite on top of perf events. You can find the previous version here:
http://lwn.net/Articles/358594/
Changes in v3:
- Fix broken CONFIG_KVM, propagate the breakpoint api changes to kvm when we exit the guest and restore the bp registers to the host. The only change is in the 4th patch.
Changes in v2:
- Follow the perf "event " rename
- The ptrace regression have been fixed (ptrace breakpoint perf events weren't released when a task ended)
- Drop the struct hw_breakpoint and store generic fields in perf_event_attr.
- Separate core and arch specific headers, drop asm-generic/hw_breakpoint.h and create linux/hw_breakpoint.h
- Use new generic len/type for breakpoint
- Handle off case: when breakpoints api is not supported by an arch
- Use proper in-kernel perf api provided by Arjan.
The following changes since commit 0f8f86c7bdd1c954fbe153af437a0d91a6c5721a:
Frederic Weisbecker (1):
Merge commit 'perf/core' into perf/hw-breakpoint
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
perfevents/hw-breakpoint
Arjan van de Ven (1):
perf/core: Provide a kernel-internal interface to get to performance counters
Frederic Weisbecker (3):
perf/core: Add a callback to perf events
hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
hw-breakpoints: Arbitrate access to pmu following registers constraints
Li Zefan (1):
ksym_tracer: Remove KSYM_SELFTEST_ENTRY
Paul Mundt (1):
x86/hw-breakpoints: Actually flush thread breakpoints in flush_thread().
arch/Kconfig | 3 + arch/x86/include/asm/debugreg.h | 13 +- arch/x86/include/asm/hw_breakpoint.h | 58 +++-- arch/x86/include/asm/processor.h | 12 +- arch/x86/kernel/hw_breakpoint.c | 390 +++++++++++++++-------- arch/x86/kernel/process.c | 9 +- arch/x86/kernel/process_32.c | 26 +-- arch/x86/kernel/process_64.c | 26 +-- arch/x86/kernel/ptrace.c | 182 +++++++---- arch/x86/kernel/smpboot.c | 3 - arch/x86/kvm/x86.c | 11 +- arch/x86/power/cpu.c | 6 - include/asm-generic/hw_breakpoint.h | 139 -------- include/linux/hw_breakpoint.h | 131 ++++++++ include/linux/perf_event.h | 37 ++- kernel/exit.c | 5 + kernel/hw_breakpoint.c | 595 +++++++++++++++++++++------------- kernel/perf_event.c | 137 ++++++++- kernel/trace/trace.h | 1 - kernel/trace/trace_entries.h | 6 +- kernel/trace/trace_ksym.c | 126 ++++---- kernel/trace/trace_selftest.c | 2 +- 22 files changed, 1178 insertions(+), 740 deletions(-) delete mode 100644 include/asm-generic/hw_breakpoint.h create mode 100644 include/linux/hw_breakpoint.h
PATCH 1/6 perf/core: Provide a kernel-internal interface to get to performance counters
In reply to Jan KiszkaFrom: Arjan van de Ven <[email protected]>
There are reasons for kernel code to ask for, and use, performance counters.
For example, in CPU freq governors this tends to be a good idea, but there are other examples possible as well of course.
This patch adds the needed bits to do enable this functionality; they have been tested in an experimental cpufreq driver that I'm working on, and the changes are all that I needed to access counters properly.
[[email protected]: added pid to perf_event_create_kernel_counter so that we can profile a particular task too
TODO: Have a better error reporting, don't just return NULL in fail case.]
LKML-Reference: <[email protected]>
--- include/linux/perf_event.h | 6 ++++ kernel/perf_event.c | 72 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 77 insertions(+), 1 deletions(-)
include/linux/perf_event.h +6 -0
@@ -744,6 +744,12 @@ extern int hw_perf_group_sched_in(struct perf_event *group_leader,struct perf_cpu_context *cpuctx, struct perf_event_context *ctx, int cpu);extern void perf_event_update_userpage (struct perf_event *event); +extern int perf_event_release_kernel (struct perf_event *event); +extern struct perf_event * +perf_event_create_kernel_counter (struct perf_event_attr *attr, +int cpu, +pid_t pid); +extern u64 perf_event_read_value (struct perf_event *event);struct perf_sample_data {u64 type;
kernel/perf_event.c +71 -1
96 lines not on this page · raw message
Signature · 1 line
1.6.2.3
PATCH 1/6 perf/core: Provide a kernel-internal interface to get to performance counters
Quoted 3 lines · Frederic Weisbecker
/*
+ *perf_event_create_kernel_counter
+ * MUST be called from a kernel thread.
Why? The reason for this requirement isn't obvious to me. It would be good to have the reason documented in the comment for the sake of people modifying the code in future.
Paul.
Re: [PATCH 1/6] perf/core: Provide a kernel-internal interface to get to performance counters
Quoted 9 lines · On Mon, 2 Nov 2009 14:46:55 +1100 Paul Mackerras <[email protected]> wrote:
Quoted 3 lines · Frederic Weisbecker
/*
+ *perf_event_create_kernel_counter
+ * MUST be called from a kernel thread.Why? The reason for this requirement isn't obvious to me. It would be good to have the reason documented in the comment for the sake of people modifying the code in future.
because if you call it from another context it will attach to that context... and go away when that context goes away...
Signature · 3 lines
Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org
Re: [PATCH 1/6] perf/core: Provide a kernel-internal interface to get to performance counters
Quoted 9 lines · Arjan van de Ven
Quoted 6 lines · On Mon, 2 Nov 2009 14:46:55 +1100 Paul Mackerras <[email protected]> wrote:
Quoted 3 lines · Frederic Weisbecker
/*
+ *perf_event_create_kernel_counter
+ * MUST be called from a kernel thread.
Quoted 6 lines
Quoted 3 lines
Why? The reason for this requirement isn't obvious to me. It would be good to have the reason documented in the comment for the sake of people modifying the code in future.
because if you call it from another context it will attach to that context... and go away when that context goes away...
I don't think that's right. When a task exits, that doesn't automatically kill all the perf_events it created. The perf_events each have a reference to their owner's task_struct, so the task_struct will hang around until all of the perf_events get released. Normally top-level perf_events have an associated filp and we use its reference count to control the perf_event lifetime, but with these new kernel perf_events there is no filp, so the caller will have to do any refcounting required.
Bottom line is that a perf_event created by perf_event_create_kernel_counter will exist until someone calls perf_event_release_kernel on it, whether or not the owner task exits (and whether or not that task is a kernel thread or a usermode process). So I see no need to prohibit creating kernel perf_events in the context of a user task, though callers need to be aware that doing so could potentially mean the user task's task_struct has to hang around for a long time after the task exits.
Paul.
Re: [PATCH 1/6] perf/core: Provide a kernel-internal interface to get to performance counters
Quoted 36 lines · Paul Mackerras
Quoted 9 lines · Arjan van de Ven
Quoted 6 lines · On Mon, 2 Nov 2009 14:46:55 +1100 Paul Mackerras <[email protected]> wrote:
Quoted 3 lines · Frederic Weisbecker
/*
+ *perf_event_create_kernel_counter
+ * MUST be called from a kernel thread.Quoted 6 lines
Quoted 3 lines
Why? The reason for this requirement isn't obvious to me. It would be good to have the reason documented in the comment for the sake of people modifying the code in future.
because if you call it from another context it will attach to that context... and go away when that context goes away...
I don't think that's right. When a task exits, that doesn't automatically kill all the
perf_eventsit created. Theperf_eventseach have a reference to their owner'stask_struct, so thetask_structwill hang around until all of theperf_eventsget released. Normally top-levelperf_eventshave an associated filp and we use its reference count to control theperf_eventlifetime, but with these new kernelperf_eventsthere is no filp, so the caller will have to do any refcounting required.Bottom line is that a
perf_eventcreated byperf_event_create_kernel_counterwill exist until someone callsperf_event_release_kernelon it, whether or not the owner task exits (and whether or not that task is a kernel thread or a usermode process). So I see no need to prohibit creating kernelperf_eventsin the context of a user task, though callers need to be aware that doing so could potentially mean the user task'stask_structhas to hang around for a long time after the task exits.
Quoted 1 line
Paul.
Yeah. For example while creating a breakpoint through ptrace, the owner of an event created by perf_event_create_kernel_counter is the user task that does the ptrace call.
But we need to explicitly release some disabled/pending ptrace breakpoints using perf_event_release_kernel when the task exits. I should simplify this part in the future.
That said, it's all fine to set the owner as a user task like in this example.
Should I remove this comment?
PATCH 2/6 x86/hw-breakpoints: Actually flush thread breakpoints in flush_thread().
In reply to Jan KiszkaFrom: Paul Mundt <[email protected]>
flush_thread() tries to do a TIF_DEBUG check before calling in to flush_thread_hw_breakpoint() (which subsequently clears the thread flag), but for some reason, the x86 code is manually clearing TIF_DEBUG immediately before the test, so this path will never be taken.
This kills off the erroneous clear_tsk_thread_flag() and lets flush_thread_hw_breakpoint() actually get invoked.
Presumably folks were getting lucky with testing and the free_thread_info() -> free_thread_xstate() path was taking care of the flush there.
LKML-Reference: <[email protected]>
--- arch/x86/kernel/process.c | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-)
arch/x86/kernel/process.c +0 -2
@@ -107,8 +107,6 @@ void flush_thread(void) }#endif -clear_tsk_thread_flag (tsk,TIF_DEBUG ); -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG )))flush_thread_hw_breakpoint (tsk);memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array));
Signature · 1 line
1.6.2.3
PATCH 3/6 perf/core: Add a callback to perf events
In reply to Jan KiszkaA simple callback in a perf event can be used for multiple purposes. For example it is useful for triggered based events like hardware breakpoints that need a callback to dispatch a triggered breakpoint event.
--- include/linux/perf_event.h | 7 ++++++- kernel/perf_event.c | 18 ++++++++++++++---- 2 files changed, 20 insertions(+), 5 deletions(-)
include/linux/perf_event.h +6 -1
@@ -544,6 +544,8 @@ struct perf_pending_entry {void (*func)(struct perf_pending_entry *); }; +typedef void (*perf_callback_t )(struct perf_event *,void *); +/** * struct perf_event - performance event kernel representation: */
@@ -639,6 +641,8 @@ struct perf_event {struct event_filter *filter;#endif +perf_callback_t callback; +#endif /* CONFIG_PERF_EVENTS */ };
@@ -748,7 +752,8 @@ extern int perf_event_release_kernel(struct perf_event *event);extern struct perf_event *perf_event_create_kernel_counter (struct perf_event_attr *attr,int cpu, -pid_t pid); +pid_t pid, +perf_callback_t callback);extern u64 perf_event_read_value (struct perf_event *event);struct perf_sample_data {
kernel/perf_event.c +14 -4
@@ -4293,6 +4293,7 @@ perf_event_alloc(struct perf_event_attr *attr,struct perf_event_context *ctx, struct perf_event *group_leader, struct perf_event *parent_event, + perf_callback_t callback, gfp_t gfpflags) { const struct pmu *pmu;
@@ -4335,6 +4336,15 @@ perf_event_alloc(struct perf_event_attr *attr, event->state =PERF_EVENT_STATE_INACTIVE ; +if (!callback) { +if (parent_event) + event->callback = parent_event->callback; +else + event->callback =NULL ; + }else { + event->callback = callback; + } +if (attr->disabled) event->state =PERF_EVENT_STATE_OFF ;
@@ -4611,7 +4621,7 @@ SYSCALL_DEFINE5(perf_event_open, } event =perf_event_alloc (&attr, cpu, ctx, group_leader, -NULL ,GFP_KERNEL ); +NULL ,NULL ,GFP_KERNEL ); err =PTR_ERR (event);if (IS_ERR (event)) goto err_put_context;
@@ -4665,7 +4675,7 @@ err_put_context:*/ struct perf_event *perf_event_create_kernel_counter (struct perf_event_attr *attr,int cpu, -pid_t pid) +pid_t pid,perf_callback_t callback) {struct perf_event *event;struct perf_event_context *ctx;
@@ -4680,7 +4690,7 @@ perf_event_create_kernel_counter(struct perf_event_attr *attr, int cpu,return NULL ; event =perf_event_alloc (attr, cpu, ctx,NULL , -NULL ,GFP_KERNEL ); +NULL , callback,GFP_KERNEL ); err =PTR_ERR (event);if (IS_ERR (event)) goto err_put_context;
@@ -4733,7 +4743,7 @@ inherit_event(struct perf_event *parent_event, child_event =perf_event_alloc (&parent_event->attr, parent_event->cpu, child_ctx, group_leader, parent_event, -GFP_KERNEL ); +NULL ,GFP_KERNEL );if (IS_ERR (child_event))return child_event;get_ctx (child_ctx);
Signature · 1 line
1.6.2.3
PATCH 3/6 perf/core: Add a callback to perf events
kernel/perf_event.c perf_event_alloc(struct perf_event_attr *attr, · line 4346
Quoted 12 lines · Frederic Weisbecker
kernel/perf_event.c +8
-0@@ -4335,6 +4336,15 @@ perf_event_alloc(struct perf_event_attr *attr, event->state =PERF_EVENT_STATE_INACTIVE ; +if (!callback) { +if (parent_event) + event->callback = parent_event->callback; +else + event->callback =NULL ; + }else { + event->callback = callback; + }
Wouldn't this be simpler and clearer as:
if (!callback && parent_event) callback = parent_event->callback; event->callback = callback;
?
Paul.
Re: [PATCH 3/6] perf/core: Add a callback to perf events
Quoted 17 lines · Paul Mackerras
kernel/perf_event.c perf_event_alloc(struct perf_event_attr *attr, · line 4346
Quoted 12 lines · Frederic Weisbecker
kernel/perf_event.c +8
-0@@ -4335,6 +4336,15 @@ perf_event_alloc(struct perf_event_attr *attr, event->state =PERF_EVENT_STATE_INACTIVE ; +if (!callback) { +if (parent_event) + event->callback = parent_event->callback; +else + event->callback =NULL ; + }else { + event->callback = callback; + }Wouldn't this be simpler and clearer as:
Quoted 7 lines
if (!callback && parent_event) callback = parent_event->callback; event->callback = callback;?
Paul.
Yep, I'm going to change that.
Thanks.
PATCH 4/6 hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
In reply to Jan KiszkaThis patch rebase the implementation of the breakpoints API on top of perf events instances.
Each breakpoints are now perf events that handle the register scheduling, thread/cpu attachment, etc..
The new layering is now made as follows:
ptrace kgdb ftrace perf syscall
\ | / /
\ | / /
/
Core breakpoint API /
/
| /
| /
Breakpoints perf events
|
|
Breakpoints PMU ---- Debug Register constraints handling
(Part of core breakpoint API)
|
|
Hardware debug registers
Reasons of this rewrite:
- Use the centralized/optimized pmu registers scheduling, implying an easier arch integration
- More powerful register handling: perf attributes (pinned/flexible events, exclusive/non-exclusive, tunable period, etc...)
Impact:
- New perf ABI: the hardware breakpoints counters
- Ptrace breakpoints setting remains tricky and still needs some per thread breakpoints references.
Todo (in the order):
- Support breakpoints perf counter events for perf tools (ie: implement perf_bpcounter_event())
- Support from perf tools
Changes in v2:
- Follow the perf "event " rename
- The ptrace regression have been fixed (ptrace breakpoint perf events weren't released when a task ended)
- Drop the struct hw_breakpoint and store generic fields in perf_event_attr.
- Separate core and arch specific headers, drop asm-generic/hw_breakpoint.h and create linux/hw_breakpoint.h
- Use new generic len/type for breakpoint
- Handle off case: when breakpoints api is not supported by an arch
Changes in v3:
- Fix broken CONFIG_KVM, we need to propagate the breakpoint api changes to kvm when we exit the guest and restore the bp registers to the host.
--- arch/Kconfig | 3 + arch/x86/include/asm/debugreg.h | 13 +- arch/x86/include/asm/hw_breakpoint.h | 58 +++-- arch/x86/include/asm/processor.h | 12 +- arch/x86/kernel/hw_breakpoint.c | 390 ++++++++++++++++++++----------- arch/x86/kernel/process.c | 7 +- arch/x86/kernel/process_32.c | 26 +-- arch/x86/kernel/process_64.c | 26 +-- arch/x86/kernel/ptrace.c | 182 ++++++++++----- arch/x86/kernel/smpboot.c | 3 - arch/x86/kvm/x86.c | 11 +- arch/x86/power/cpu.c | 6 - include/asm-generic/hw_breakpoint.h | 139 ----------- include/linux/hw_breakpoint.h | 131 +++++++++++ include/linux/perf_event.h | 26 ++- kernel/exit.c | 5 + kernel/hw_breakpoint.c | 424 ++++++++++++++-------------------- kernel/perf_event.c | 53 ++++- kernel/trace/trace_entries.h | 6 +- kernel/trace/trace_ksym.c | 126 +++++------ 20 files changed, 886 insertions(+), 761 deletions(-) delete mode 100644 include/asm-generic/hw_breakpoint.h create mode 100644 include/linux/hw_breakpoint.h
arch/Kconfig +3 -0
@@ -128,6 +128,9 @@ config HAVE_DEFAULT_NO_SPIN_MUTEXES config HAVE_HW_BREAKPOINT bool + depends on HAVE_PERF_EVENTS + select ANON_INODES + select PERF_EVENTS source "kernel/gcov/Kconfig"
arch/x86/include/asm/debugreg.h +6 -7
@@ -75,13 +75,6 @@*/ #ifdef __KERNEL__ -/* For process management */ -extern void flush_thread_hw_breakpoint (struct task_struct *tsk); -extern int copy_thread_hw_breakpoint (struct task_struct *tsk, -struct task_struct *child,unsigned long clone_flags); - -/* For CPU management */ -extern void load_debug_registers (void );static inline void hw_breakpoint_disable (void ) {/* Zero the control register for HW Breakpoint */
@@ -94,6 +87,12 @@ static inline void hw_breakpoint_disable(void)set_debugreg (0UL ,3 ); } +#ifdef CONFIG_KVM +extern void hw_breakpoint_restore (void ); +#else +static inline void hw_breakpoint_restore (void ) { } +#endif +#endif /* __KERNEL__ */ #endif /* _ASM_X86_DEBUGREG_H */
arch/x86/include/asm/hw_breakpoint.h +38 -20
88 lines not on this page · raw message
arch/x86/include/asm/processor.h +6 -6
@@ -423,6 +423,8 @@ extern unsigned int xstate_size;extern void free_thread_xstate (struct task_struct *);extern struct kmem_cache *task_xstate_cachep; +struct perf_event ; +struct thread_struct {/* Cached TLS descriptors: */ struct desc_struct tls_array[GDT_ENTRY_TLS_ENTRIES ];
@@ -444,12 +446,10 @@ struct thread_struct {unsigned long fs;#endif unsigned long gs; -/* Hardware debugging registers: */ -unsigned long debugreg[HBP_NUM ]; -unsigned long debugreg6; -unsigned long debugreg7; -/* Hardware breakpoint info */ -struct hw_breakpoint *hbp[HBP_NUM ]; +/* Save middle states of ptrace breakpoints */ +struct perf_event *ptrace_bps[HBP_NUM ]; +/* Debug status used for traps, single steps, etc... */ +unsigned long debugreg6;/* Fault info: */ unsigned long cr2;unsigned long trap_no;
arch/x86/kernel/hw_breakpoint.c +254 -136
548 lines not on this page · raw message
arch/x86/kernel/process.c +2 -5
@@ -10,6 +10,7 @@#include <linux/clockchips.h> #include <linux/random.h> #include <trace/events/power.h> +#include <linux/hw_breakpoint.h> #include <asm/system.h> #include <asm/apic.h> #include <asm/syscalls.h>
@@ -18,7 +19,6 @@#include <asm/i387.h> #include <asm/ds.h> #include <asm/debugreg.h> -#include <asm/hw_breakpoint.h> unsigned long idle_halt;EXPORT_SYMBOL (idle_halt);
@@ -47,8 +47,6 @@ void free_thread_xstate(struct task_struct *tsk)kmem_cache_free (task_xstate_cachep, tsk->thread.xstate); tsk->thread.xstate =NULL ; } -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG ))) -flush_thread_hw_breakpoint (tsk);WARN (tsk->thread.ds_ctx,"leaking DS context\n" ); }
@@ -107,8 +105,7 @@ void flush_thread(void) }#endif -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG ))) -flush_thread_hw_breakpoint (tsk); +flush_ptrace_hw_breakpoint (tsk);memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array));/* * Forget coprocessor state..
arch/x86/kernel/process_32.c +2 -24
@@ -59,7 +59,6 @@#include <asm/syscalls.h> #include <asm/ds.h> #include <asm/debugreg.h> -#include <asm/hw_breakpoint.h> asmlinkage void ret_from_fork(void) __asm__("ret_from_fork");
@@ -264,9 +263,8 @@ int copy_thread(unsigned long clone_flags, unsigned long sp, p->thread.io_bitmap_ptr =NULL ; tsk = current; err = -ENOMEM ; -if (unlikely (test_tsk_thread_flag (tsk,TIF_DEBUG ))) -if (copy_thread_hw_breakpoint (tsk, p, clone_flags)) - goto out; + +memset (p->thread.ptrace_bps,0 ,sizeof (p->thread.ptrace_bps));if (unlikely (test_tsk_thread_flag (tsk,TIF_IO_BITMAP ))) { p->thread.io_bitmap_ptr = kmemdup(tsk->thread.io_bitmap_ptr,
@@ -287,13 +285,10 @@ int copy_thread(unsigned long clone_flags, unsigned long sp, err =do_set_thread_area (p,-1 , (struct user_desc __user *)childregs->si,0 ); -out:if (err && p->thread.io_bitmap_ptr) {kfree (p->thread.io_bitmap_ptr); p->thread.io_bitmap_max =0 ; } -if (err) -flush_thread_hw_breakpoint (p);clear_tsk_thread_flag (p,TIF_DS_AREA_MSR ); p->thread.ds_ctx =NULL ;
@@ -437,23 +432,6 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)lazy_load_gs (next->gs);percpu_write (current_task, next_p); -/* -* There's a problem with moving the arch_install_thread_hw_breakpoint() -* call before current is updated. Suppose a kernel breakpoint is -* triggered in between the two, the hw-breakpoint handler will see that -* the 'current' task does not have TIF_DEBUG flag set and will think it -* is leftover from an old task (lazy switching) and will erase it. Then -* until the next context switch, no user-breakpoints will be installed. -* -* The real problem is that it's impossible to update both current and -* physical debug registers at the same instant, so there will always be -* a window in which they disagree and a breakpoint might get triggered. -* Since we use lazy switching, we are forced to assume that a -* disagreement means that current is correct and the exception is due -* to lazy debug register switching. -*/ -if (unlikely (test_tsk_thread_flag (next_p,TIF_DEBUG ))) -arch_install_thread_hw_breakpoint (next_p);return prev_p; }
arch/x86/kernel/process_64.c +1 -25
@@ -53,7 +53,6 @@#include <asm/syscalls.h> #include <asm/ds.h> #include <asm/debugreg.h> -#include <asm/hw_breakpoint.h> asmlinkage extern void ret_from_fork(void);
@@ -244,8 +243,6 @@ void release_thread(struct task_struct *dead_task)BUG (); } } -if (unlikely (dead_task->thread.debugreg7)) -flush_thread_hw_breakpoint (dead_task); }static inline void set_32bit_tls (struct task_struct *t,int tls,u32 addr)
@@ -309,9 +306,7 @@ int copy_thread(unsigned long clone_flags, unsigned long sp,savesegment (ds, p->thread.ds); err = -ENOMEM ; -if (unlikely (test_tsk_thread_flag (me,TIF_DEBUG ))) -if (copy_thread_hw_breakpoint (me, p, clone_flags)) - goto out; +memset (p->thread.ptrace_bps,0 ,sizeof (p->thread.ptrace_bps));if (unlikely (test_tsk_thread_flag (me,TIF_IO_BITMAP ))) { p->thread.io_bitmap_ptr =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL );
@@ -351,8 +346,6 @@ out:kfree (p->thread.io_bitmap_ptr); p->thread.io_bitmap_max =0 ; } -if (err) -flush_thread_hw_breakpoint (p);return err; }
@@ -508,23 +501,6 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)*/ if (preload_fpu)__math_state_restore (); -/* -* There's a problem with moving the arch_install_thread_hw_breakpoint() -* call before current is updated. Suppose a kernel breakpoint is -* triggered in between the two, the hw-breakpoint handler will see that -* the 'current' task does not have TIF_DEBUG flag set and will think it -* is leftover from an old task (lazy switching) and will erase it. Then -* until the next context switch, no user-breakpoints will be installed. -* -* The real problem is that it's impossible to update both current and -* physical debug registers at the same instant, so there will always be -* a window in which they disagree and a breakpoint might get triggered. -* Since we use lazy switching, we are forced to assume that a -* disagreement means that current is correct and the exception is due -* to lazy debug register switching. -*/ -if (unlikely (test_tsk_thread_flag (next_p,TIF_DEBUG ))) -arch_install_thread_hw_breakpoint (next_p);return prev_p; }
arch/x86/kernel/ptrace.c +125 -57
257 lines not on this page · raw message
arch/x86/kernel/smpboot.c +0 -3
@@ -64,7 +64,6 @@#include <asm/apic.h> #include <asm/setup.h> #include <asm/uv/uv.h> -#include <asm/debugreg.h> #include <linux/mc146818rtc.h> #include <asm/smpboot_hooks.h>
@@ -328,7 +327,6 @@ notrace static void __cpuinit start_secondary(void *unused) x86_cpuinit.setup_percpu_clockev ();wmb (); -load_debug_registers ();cpu_idle (); }
@@ -1269,7 +1267,6 @@ void cpu_disable_common(void)remove_cpu_from_maps (cpu);unlock_vector_lock ();fixup_irqs (); -hw_breakpoint_disable (); }int native_cpu_disable (void )
arch/x86/kvm/x86.c +3 -8
@@ -42,6 +42,7 @@#define CREATE_TRACE_POINTS #include "trace.h" +#include <asm/debugreg.h> #include <asm/uaccess.h> #include <asm/msr.h> #include <asm/desc.h>
@@ -3643,14 +3644,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)trace_kvm_entry (vcpu->vcpu_id); kvm_x86_ops->run (vcpu, kvm_run); -if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) { -set_debugreg (current->thread.debugreg[0 ],0 ); -set_debugreg (current->thread.debugreg[1 ],1 ); -set_debugreg (current->thread.debugreg[2 ],2 ); -set_debugreg (current->thread.debugreg[3 ],3 ); -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } +if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) +hw_breakpoint_restore ();set_bit (KVM_REQ_KICK , &vcpu->requests);local_irq_enable ();
arch/x86/power/cpu.c +0 -6
@@ -105,7 +105,6 @@ static void __save_processor_state(struct saved_context *ctxt) ctxt->cr4 =read_cr4 (); ctxt->cr8 =read_cr8 ();#endif -hw_breakpoint_disable (); }/* Needed by apm.c */
@@ -144,11 +143,6 @@ static void fix_processor_context(void)#endif load_TR_desc ();/* This does ltr */ load_LDT (¤t->active_mm->context);/* This does lldt */ - -/* -* Now maybe reload the debug registers -*/ -load_debug_registers (); }/**
include/asm-generic/hw_breakpoint.h deleted +0 -139
139 lines not on this page · raw message
include/linux/hw_breakpoint.h new file +131 -0
131 lines not on this page · raw message
include/linux/perf_event.h +25 -1
@@ -18,6 +18,10 @@#include <linux/ioctl.h> #include <asm/byteorder.h> +#ifdef CONFIG_HAVE_HW_BREAKPOINT +#include <asm/hw_breakpoint.h> +#endif +/* * User-space ABI bits: */
@@ -31,6 +35,7 @@ enum perf_type_id {PERF_TYPE_TRACEPOINT =2 ,PERF_TYPE_HW_CACHE =3 ,PERF_TYPE_RAW =4 , +PERF_TYPE_BREAKPOINT =5 , PERF_TYPE_MAX, /* non-ABI */ };
@@ -207,6 +212,15 @@ struct perf_event_attr {__u32 wakeup_events;/* wakeup every n events */ __u32 wakeup_watermark;/* bytes before wakeup */ }; + +union { +struct {/* Hardware breakpoint info */ +__u64 bp_addr; +__u32 bp_type; +__u32 bp_len; + }; + }; +__u32 __reserved_2;__u64 __reserved_3;
@@ -476,6 +490,11 @@ struct hw_perf_event {atomic64_t count;struct hrtimer hrtimer; }; +#ifdef CONFIG_HAVE_HW_BREAKPOINT +union {/* breakpoint */ +struct arch_hw_breakpoint info; + }; +#endif };atomic64_t prev_count;u64 sample_period;
@@ -588,7 +607,7 @@ struct perf_event {u64 tstamp_running;u64 tstamp_stopped; -struct perf_event_attr attr; +struct perf_event_attr attr;struct hw_perf_event hw;struct perf_event_context *ctx;
@@ -643,6 +662,8 @@ struct perf_event {perf_callback_t callback; +perf_callback_t event_callback; +#endif /* CONFIG_PERF_EVENTS */ };
@@ -831,6 +852,7 @@ extern int sysctl_perf_event_sample_rate;extern void perf_event_init (void );extern void perf_tp_event (int event_id,u64 addr,u64 count,void *record,int entry_size); +extern void perf_bp_event (struct perf_event *event,void *data);#ifndef perf_misc_flags#define perf_misc_flags (regs) (user_mode(regs) ? PERF_RECORD_MISC_USER : \
@@ -865,6 +887,8 @@ static inline int perf_event_task_enable(void) { return -EINVAL; }static inline void perf_sw_event (u32 event_id,u64 nr,int nmi,struct pt_regs *regs,u64 addr) { } +static inline void +perf_bp_event (struct perf_event *event,void *data) { }static inline void perf_event_mmap (struct vm_area_struct *vma) { }static inline void perf_event_comm (struct task_struct *tsk) { }
kernel/exit.c +5 -0
@@ -49,6 +49,7 @@#include <linux/init_task.h> #include <linux/perf_event.h> #include <trace/events/sched.h> +#include <linux/hw_breakpoint.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -980,6 +981,10 @@ NORET_TYPE void do_exit(long code)proc_exit_connector (tsk);/* +* FIXME: do that only when needed, using sched_exit tracepoint +*/ +flush_ptrace_hw_breakpoint (tsk); +/* * Flush inherited counters to the parent - before the parent * gets woken up by child-exit notifications. */
kernel/hw_breakpoint.c +170 -254
515 lines not on this page · raw message
kernel/perf_event.c +52 -1
77 lines not on this page · raw message
kernel/trace/trace_entries.h +3 -3
@@ -372,11 +372,11 @@ FTRACE_ENTRY(ksym_trace, ksym_trace_entry, F_STRUCT( __field( unsigned long, ip ) __field( unsigned char, type ) - __array( char , ksym_name, KSYM_NAME_LEN ) __array( char , cmd, TASK_COMM_LEN ) + __field( unsigned long, addr ) ), - F_printk("ip: %pF type: %d ksym_name: %s cmd: %s", + F_printk("ip: %pF type: %d ksym_name: %pS cmd: %s", (void *)__entry->ip, (unsigned int )__entry->type, - __entry->ksym_name, __entry->cmd) + (void *)__entry->addr, __entry->cmd) );
kernel/trace/trace_ksym.c +60 -66
241 lines not on this page · raw message
Signature · 1 line
1.6.2.3
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Frederic Weisbecker wrote:
[...]
arch/x86/include/asm/debugreg.h static inline void hw_breakpoint_disable(void) · line 95
Quoted 10 lines
arch/x86/include/asm/debugreg.h +6
-0@@ -94,6 +87,12 @@ static inline void hw_breakpoint_disable(void)set_debugreg (0UL ,3 ); } +#ifdef CONFIG_KVM +extern void hw_breakpoint_restore (void ); +#else +static inline void hw_breakpoint_restore (void ) { } +#endif +
Hmm, if this shall become a KVM-only API, why the empty wrapper?
[...]
Quoted 8 lines
arch/x86/kernel/hw_breakpoint.c +2
-5-void arch_flush_thread_hw_breakpoint (struct task_struct *tsk) +#ifdef CONFIG_KVM +void hw_breakpoint_restore (void ) { -int i; -struct thread_struct *thread = &(tsk->thread); - - thread->debugreg7 =0 ;
arch/x86/kernel/hw_breakpoint.c int arch_validate_hwbkpt_settings(struct hw_breakpoint *bp, · line 402
Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7
-2-for (i =0 ; i <HBP_NUM ; i++) - thread->debugreg[i] =0 ; +set_debugreg (__get_cpu_var (cpu_debugreg[0 ]),0 ); +set_debugreg (__get_cpu_var (cpu_debugreg[1 ]),1 ); +set_debugreg (__get_cpu_var (cpu_debugreg[2 ]),2 ); +set_debugreg (__get_cpu_var (cpu_debugreg[3 ]),3 ); +set_debugreg (current->thread.debugreg6,6 ); + set_debugreg(__get_cpu_var(dr7), 7); } +#endif
EXPORT_SYMBOL_GPL(hw_breakpoint_restore);
as KVM can be built as module.
[...]
Quoted 16 lines
arch/x86/kvm/x86.c +1
-0@@ -42,6 +42,7 @@#define CREATE_TRACE_POINTS #include "trace.h" +#include <asm/debugreg.h> #include <asm/uaccess.h> #include <asm/msr.h> #include <asm/desc.h> @@ -3643,14 +3644,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)trace_kvm_entry (vcpu->vcpu_id); kvm_x86_ops->run (vcpu, kvm_run);
arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) · line 3648
Quoted 10 lines
arch/x86/kvm/x86.c +2
-8-if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) { -set_debugreg (current->thread.debugreg[0 ],0 ); -set_debugreg (current->thread.debugreg[1 ],1 ); -set_debugreg (current->thread.debugreg[2 ],2 ); -set_debugreg (current->thread.debugreg[3 ],3 ); -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } +if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) +hw_breakpoint_restore ();
TIF_DEBUG is only set on active ptrace hw-breakpoints, thus we miss other types here, right? (Note: arch.switch_db_regs is guest-related, thus does not help in this regard.)
Jan
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 6 lines · Jan Kiszka
Frederic Weisbecker wrote:
[...]Quoted 4 lines
arch/x86/include/asm/debugreg.h +0
-0@@ -94,6 +87,12 @@ static inline void hw_breakpoint_disable(void)set_debugreg (0UL ,3 ); }
Quoted 8 lines
arch/x86/include/asm/debugreg.h static inline void hw_breakpoint_disable(void) · line 95
Quoted 6 lines
arch/x86/include/asm/debugreg.h +6
-0+#ifdef CONFIG_KVM +extern void hw_breakpoint_restore (void ); +#else +static inline void hw_breakpoint_restore (void ) { } +#endif +Hmm, if this shall become a KVM-only API, why the empty wrapper?
Right, this is an odd reflex.
Quoted 22 lines
[...]
Quoted 8 lines
arch/x86/kernel/hw_breakpoint.c +2
-5-void arch_flush_thread_hw_breakpoint (struct task_struct *tsk) +#ifdef CONFIG_KVM +void hw_breakpoint_restore (void ) { -int i; -struct thread_struct *thread = &(tsk->thread); - - thread->debugreg7 =0 ;arch/x86/kernel/hw_breakpoint.c int arch_validate_hwbkpt_settings(struct hw_breakpoint *bp, · line 402
Quoted 10 lines
arch/x86/kernel/hw_breakpoint.c +7
-2-for (i =0 ; i <HBP_NUM ; i++) - thread->debugreg[i] =0 ; +set_debugreg (__get_cpu_var (cpu_debugreg[0 ]),0 ); +set_debugreg (__get_cpu_var (cpu_debugreg[1 ]),1 ); +set_debugreg (__get_cpu_var (cpu_debugreg[2 ]),2 ); +set_debugreg (__get_cpu_var (cpu_debugreg[3 ]),3 ); +set_debugreg (current->thread.debugreg6,6 ); + set_debugreg(__get_cpu_var(dr7), 7); } +#endif
EXPORT_SYMBOL_GPL(hw_breakpoint_restore);
Quoted 1 line
as KVM can be built as module.
Indeed.
Quoted 32 lines
[...]
Quoted 16 lines
arch/x86/kvm/x86.c +1
-0@@ -42,6 +42,7 @@#define CREATE_TRACE_POINTS #include "trace.h" +#include <asm/debugreg.h> #include <asm/uaccess.h> #include <asm/msr.h> #include <asm/desc.h> @@ -3643,14 +3644,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)trace_kvm_entry (vcpu->vcpu_id); kvm_x86_ops->run (vcpu, kvm_run);arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) · line 3648
Quoted 10 lines
arch/x86/kvm/x86.c +2
-8-if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) { -set_debugreg (current->thread.debugreg[0 ],0 ); -set_debugreg (current->thread.debugreg[1 ],1 ); -set_debugreg (current->thread.debugreg[2 ],2 ); -set_debugreg (current->thread.debugreg[3 ],3 ); -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } +if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) +hw_breakpoint_restore ();
TIF_DEBUGis only set on active ptrace hw-breakpoints, thus we miss other types here, right? (Note: arch.switch_db_regs is guest-related, thus does not help in this regard.)
Quoted 2 lines
Jan
True :)
Thanks a lot for this review, I'm going to fix that quickly.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
In reply to Jan KiszkaQuoted 19 lines · Jan Kiszka
Quoted 4 lines
arch/x86/kvm/x86.c +0
-0@@ -3643,14 +3644,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)trace_kvm_entry (vcpu->vcpu_id); kvm_x86_ops->run (vcpu, kvm_run);arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) · line 3648
Quoted 10 lines
arch/x86/kvm/x86.c +2
-8-if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) { -set_debugreg (current->thread.debugreg[0 ],0 ); -set_debugreg (current->thread.debugreg[1 ],1 ); -set_debugreg (current->thread.debugreg[2 ],2 ); -set_debugreg (current->thread.debugreg[3 ],3 ); -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } +if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) +hw_breakpoint_restore ();
TIF_DEBUGis only set on active ptrace hw-breakpoints, thus we miss other types here, right? (Note: arch.switch_db_regs is guest-related, thus does not help in this regard.)
Quoted 2 lines
Jan
About this. vcpu->arch.switch_db_regs is guest related but it looks like the only thing I need to check.
I'm not sure when it is activated. Is it always done once the guest changes its debug registers? I suspect there is a corner case.
Because since I can't anymore assume TIF_DEBUG covers every breakpoints uses, it means I'll need to maintain a refcount of breakpoints in use.
Well, I have one already, but it is splitted into several refcounts (per task events, per cpu, non-pinned, etc...). And since vcpu_enter_guest() is a fast path, I'll need to maintain another global per cpu one, without lock or further operations to know if we need to save the debug registers, just a simple check.
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 23 lines · Frederic Weisbecker
Quoted 18 lines · Jan Kiszka
Quoted 4 lines
arch/x86/kvm/x86.c +0
-0@@ -3643,14 +3644,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)trace_kvm_entry (vcpu->vcpu_id); kvm_x86_ops->run (vcpu, kvm_run);arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) · line 3648
Quoted 10 lines
arch/x86/kvm/x86.c +2
-8-if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) { -set_debugreg (current->thread.debugreg[0 ],0 ); -set_debugreg (current->thread.debugreg[1 ],1 ); -set_debugreg (current->thread.debugreg[2 ],2 ); -set_debugreg (current->thread.debugreg[3 ],3 ); -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } +if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) +hw_breakpoint_restore ();
TIF_DEBUGis only set on active ptrace hw-breakpoints, thus we miss other types here, right? (Note: arch.switch_db_regs is guest-related, thus does not help in this regard.)Quoted 2 lines
Jan
Quoted 5 lines
About this.
vcpu->arch.switch_db_regsis guest related but it looks like the only thing I need to check.I'm not sure when it is activated. Is it always done once the guest changes its debug registers? I suspect there is a corner case.
It's set when we had to write to some debugreg[0..4], either for use by the guest itself or for debugging it from the host. It used to be the only condition for switching on exit as we saved the registers on entry (under the same condition). This was reworked recently to avoid the entry saving.
Quoted 10 lines
Because since I can't anymore assume
TIF_DEBUGcovers every breakpoints uses, it means I'll need to maintain a refcount of breakpoints in use.
Well, I have one already, but it is splitted into several refcounts (per task events, per cpu, non-pinned, etc...). And sincevcpu_enter_guest()is a fast path, I'll need to maintain another global per cpu one, without lock or further operations to know if we need to save the debug registers, just a simple check.
I'm not 100% sure right now if we still need "switch_db_reg" in case we have a reliable indicator that the host requires properly set registers. ATM I would dare to say, we don't, but I need to think about this again.
Jan
Re: [PATCH 4/6] hw-breakpoints: Rewrite the hw-breakpoints layer on top of perf events
Quoted 47 lines · Jan Kiszka
Quoted 23 lines · Frederic Weisbecker
Quoted 18 lines · Jan Kiszka
Quoted 4 lines
arch/x86/kvm/x86.c +0
-0@@ -3643,14 +3644,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)trace_kvm_entry (vcpu->vcpu_id); kvm_x86_ops->run (vcpu, kvm_run);arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) · line 3648
Quoted 10 lines
arch/x86/kvm/x86.c +2
-8-if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) { -set_debugreg (current->thread.debugreg[0 ],0 ); -set_debugreg (current->thread.debugreg[1 ],1 ); -set_debugreg (current->thread.debugreg[2 ],2 ); -set_debugreg (current->thread.debugreg[3 ],3 ); -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } +if (unlikely (vcpu->arch.switch_db_regs ||test_thread_flag (TIF_DEBUG ))) +hw_breakpoint_restore ();
TIF_DEBUGis only set on active ptrace hw-breakpoints, thus we miss other types here, right? (Note: arch.switch_db_regs is guest-related, thus does not help in this regard.)Quoted 2 lines
Jan
Quoted 5 lines
About this.
vcpu->arch.switch_db_regsis guest related but it looks like the only thing I need to check.I'm not sure when it is activated. Is it always done once the guest changes its debug registers? I suspect there is a corner case.
It's set when we had to write to some debugreg[0..4], either for use by the guest itself or for debugging it from the host. It used to be the only condition for switching on exit as we saved the registers on entry (under the same condition). This was reworked recently to avoid the entry saving.
Quoted 10 lines
Because since I can't anymore assume
TIF_DEBUGcovers every breakpoints uses, it means I'll need to maintain a refcount of breakpoints in use.
Well, I have one already, but it is splitted into several refcounts (per task events, per cpu, non-pinned, etc...). And sincevcpu_enter_guest()is a fast path, I'll need to maintain another global per cpu one, without lock or further operations to know if we need to save the debug registers, just a simple check.
Quoted 6 lines
I'm not 100% sure right now if we still need "
switch_db_reg" in case we have a reliable indicator that the host requires properly set registers. ATM I would dare to say, we don't, but I need to think about this again.Jan
Ok. I'm going to just check if the host has active breakpoints pending and if so, I'll restore them while exiting the guest.
Thanks.
PATCH 5/6 hw-breakpoints: Arbitrate access to pmu following registers constraints
In reply to Jan KiszkaAllow or refuse to build a counter using the breakpoints pmu following given constraints.
We keep track of the pmu users by using three per cpu variables:
- nr_cpu_bp_pinned stores the number of pinned cpu breakpoints counters in the given cpu
- nr_bp_flexible stores the number of non-pinned breakpoints counters in the given cpu.
- task_bp_pinned stores the number of pinned task breakpoints in a cpu
The latter is not a simple counter but gathers the number of tasks that have n pinned breakpoints.
Considering HBP_NUM the number of available breakpoint address registers:
task_bp_pinned[0] is the number of tasks having 1 breakpoint task_bp_pinned[1] is the number of tasks having 2 breakpoints [...]
task_bp_pinned[HBP_NUM - 1] is the number of tasks having the maximum number of registers (HBP_NUM).
When a breakpoint counter is created and wants an access to the pmu, we evaluate the following constraints:
== Non-pinned counter ==
- If attached to a single cpu, check:
(per_cpu(nr_bp_flexible, cpu) || (per_cpu(nr_cpu_bp_pinned, cpu)
+ max(per_cpu(task_bp_pinned, cpu)))) < HBP_NUM
-> If there are already non-pinned counters in this cpu, it
means there is already a free slot for them. Otherwise, we check that the maximum number of per task breakpoints (for this cpu) plus the number of per cpu breakpoint (for this cpu) doesn't cover every registers.
- If attached to every cpus, check:
(per_cpu(nr_bp_flexible, *) || (max(per_cpu(nr_cpu_bp_pinned, *))
+ max(per_cpu(task_bp_pinned, *)))) < HBP_NUM
-> This is roughly the same, except we check the number of per
cpu bp for every cpu and we keep the max one. Same for the per tasks breakpoints.
== Pinned counter ==
- If attached to a single cpu, check:
((per_cpu(nr_bp_flexible, cpu) > 1)
+ per_cpu(nr_cpu_bp_pinned, cpu)
+ max(per_cpu(task_bp_pinned, cpu))) < HBP_NUM
-> Same checks as before. But now the nr_bp_flexible, if any,
must keep one register at least (or flexible breakpoints will never be be fed).
- If attached to every cpus, check:
((per_cpu(nr_bp_flexible, *) > 1)
+ max(per_cpu(nr_cpu_bp_pinned, *))
+ max(per_cpu(task_bp_pinned, *))) < HBP_NUM
Changes in v2:
- Counter -> event rename
--- kernel/hw_breakpoint.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 231 insertions(+), 6 deletions(-)
kernel/hw_breakpoint.c +231 -6
@@ -16,6 +16,8 @@* Copyright (C) 2007 Alan Stern * Copyright (C) IBM Corporation, 2009 * Copyright (C) 2009, Frederic Weisbecker <[email protected]> +* +* Thanks to Ingo Molnar for his many suggestions. */ /*
@@ -44,24 +46,247 @@#include <asm/debugreg.h> #endif -static atomic_t bp_slot; +/* +* Constraints data +*/ + +/* Number of pinned cpu breakpoints in a cpu */ +static DEFINE_PER_CPU(unsigned int, nr_cpu_bp_pinned); -int reserve_bp_slot (struct perf_event *bp) +/* Number of pinned task breakpoints in a cpu */ +static DEFINE_PER_CPU(unsigned int, task_bp_pinned[HBP_NUM]); + +/* Number of non-pinned cpu/task breakpoints in a cpu */ +static DEFINE_PER_CPU(unsigned int, nr_bp_flexible); + +/* Gather the number of total pinned and un-pinned bp in a cpuset */ +struct bp_busy_slots { +unsigned int pinned; +unsigned int flexible; +}; + +/* Serialize accesses to the above constraints */ +static DEFINE_MUTEX (nr_bp_mutex); + +/* +* Report the maximum number of pinned breakpoints a task +* have in this cpu +*/ +static unsigned int max_task_bp_pinned (int cpu) { -if (atomic_inc_return (&bp_slot) ==HBP_NUM ) { -atomic_dec (&bp_slot); +int i; +unsigned int *tsk_pinned =per_cpu (task_bp_pinned, cpu); -return -ENOSPC ; +for (i =HBP_NUM -1 ; i >=0 ; i--) { +if (tsk_pinned[i] >0 ) +return i +1 ; }return 0 ; } +/* +* Report the number of pinned/un-pinned breakpoints we have in +* a given cpu (cpu > -1) or in all of them (cpu = -1). +*/ +static void fetch_bp_busy_slots (struct bp_busy_slots *slots,int cpu) +{ +if (cpu >=0 ) { + slots->pinned =per_cpu (nr_cpu_bp_pinned, cpu); + slots->pinned +=max_task_bp_pinned (cpu); + slots->flexible =per_cpu (nr_bp_flexible, cpu); + +return ; + } + +for_each_online_cpu (cpu) { +unsigned int nr; + + nr =per_cpu (nr_cpu_bp_pinned, cpu); + nr +=max_task_bp_pinned (cpu); + +if (nr > slots->pinned) + slots->pinned = nr; + + nr =per_cpu (nr_bp_flexible, cpu); + +if (nr > slots->flexible) + slots->flexible = nr; + } +} + +/* +* Add a pinned breakpoint for the given task in our constraint table +*/ +static void toggle_bp_task_slot (struct task_struct *tsk,int cpu,bool enable) +{ +int count =0 ; +struct perf_event *bp; +struct perf_event_context *ctx = tsk->perf_event_ctxp; +unsigned int *task_bp_pinned; +struct list_head *list; +unsigned long flags; + +if (WARN_ONCE (!ctx,"No perf context for this task" )) +return ; + + list = &ctx->event_list; + +spin_lock_irqsave (&ctx->lock, flags); + +/* +* The current breakpoint counter is not included in the list +* at the open() callback time +*/ +list_for_each_entry (bp, list, event_entry) { +if (bp->attr.type ==PERF_TYPE_BREAKPOINT ) + count++; + } + +spin_unlock_irqrestore (&ctx->lock, flags); + +if (WARN_ONCE (count <0 ,"No breakpoint counter found in the counter list" )) +return ; + + task_bp_pinned =per_cpu (task_bp_pinned, cpu); +if (enable) { + task_bp_pinned[count]++; +if (count >0 ) + task_bp_pinned[count-1 ]--; + }else { + task_bp_pinned[count]--; +if (count >0 ) + task_bp_pinned[count-1 ]++; + } +} + +/* +* Add/remove the given breakpoint in our constraint table +*/ +static void toggle_bp_slot (struct perf_event *bp,bool enable) +{ +int cpu = bp->cpu; +unsigned int *nr; +struct task_struct *tsk = bp->ctx->task; + +/* Flexible */ +if (!bp->attr.pinned) { +if (cpu >=0 ) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle; + } + +for_each_online_cpu (cpu) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle; + } + } +/* Pinned counter task profiling */ +if (tsk) { +if (cpu >=0 ) { +toggle_bp_task_slot (tsk, cpu, enable); +return ; + } + +for_each_online_cpu (cpu) +toggle_bp_task_slot (tsk, cpu, enable); +return ; + } + +/* Pinned counter cpu profiling */ + nr = &per_cpu (nr_cpu_bp_pinned, bp->cpu); + +toggle: + *nr = enable ? *nr +1 : *nr -1 ; +} + +/* +* Contraints to check before allowing this new breakpoint counter: +* +* == Non-pinned counter == +* +* - If attached to a single cpu, check: +* +* (per_cpu(nr_bp_flexible, cpu) || (per_cpu(nr_cpu_bp_pinned, cpu) +* + max(per_cpu(task_bp_pinned, cpu)))) < HBP_NUM +* +* -> If there are already non-pinned counters in this cpu, it means +* there is already a free slot for them. +* Otherwise, we check that the maximum number of per task +* breakpoints (for this cpu) plus the number of per cpu breakpoint +* (for this cpu) doesn't cover every registers. +* +* - If attached to every cpus, check: +* +* (per_cpu(nr_bp_flexible, *) || (max(per_cpu(nr_cpu_bp_pinned, *)) +* + max(per_cpu(task_bp_pinned, *)))) < HBP_NUM +* +* -> This is roughly the same, except we check the number of per cpu +* bp for every cpu and we keep the max one. Same for the per tasks +* breakpoints. +* +* +* == Pinned counter == +* +* - If attached to a single cpu, check: +* +* ((per_cpu(nr_bp_flexible, cpu) > 1) + per_cpu(nr_cpu_bp_pinned, cpu) +* + max(per_cpu(task_bp_pinned, cpu))) < HBP_NUM +* +* -> Same checks as before. But now the nr_bp_flexible, if any, must keep +* one register at least (or they will never be fed). +* +* - If attached to every cpus, check: +* +* ((per_cpu(nr_bp_flexible, *) > 1) + max(per_cpu(nr_cpu_bp_pinned, *)) +* + max(per_cpu(task_bp_pinned, *))) < HBP_NUM +*/ +int reserve_bp_slot (struct perf_event *bp) +{ +struct bp_busy_slots slots = {0 }; +int ret =0 ; + +mutex_lock (&nr_bp_mutex); + +fetch_bp_busy_slots (&slots, bp->cpu); + +if (!bp->attr.pinned) { +/* +* If there are already flexible counters here, +* there is at least one slot reserved for all +* of them. Just join the party. +* +* Otherwise, check there is at least one free slot +*/ +if (!slots.flexible && slots.pinned ==HBP_NUM ) { + ret = -ENOSPC ; + goto end; + } + +/* Flexible counters need to keep at least one slot */ + }else if (slots.pinned + (!!slots.flexible) ==HBP_NUM ) { + ret = -ENOSPC ; + goto end; + } + +toggle_bp_slot (bp, true); + +end: +mutex_unlock (&nr_bp_mutex); + +return ret; +} +void release_bp_slot (struct perf_event *bp) { -atomic_dec (&bp_slot); +mutex_lock (&nr_bp_mutex); + +toggle_bp_slot (bp, false); + +mutex_unlock (&nr_bp_mutex); } +int __register_perf_hw_breakpoint (struct perf_event *bp) {int ret;
Signature · 1 line
1.6.2.3
PATCH 6/6 ksym_tracer: Remove KSYM_SELFTEST_ENTRY
In reply to Jan KiszkaFrom: Li Zefan <[email protected]>
The macro used to be used in both trace_selftest.c and trace_ksym.c, but no longer, so remove it from header file.
--- kernel/trace/trace.h | 1 - kernel/trace/trace_selftest.c | 2 +- 2 files changed, 1 insertions(+), 2 deletions(-)
kernel/trace/trace.h +0 -1
@@ -370,7 +370,6 @@ int register_tracer(struct tracer *type);void unregister_tracer (struct tracer *type);int is_tracing_stopped (void ); -#define KSYM_SELFTEST_ENTRY "ksym_selftest_dummy"extern int process_new_ksym_entry (char *ksymname,int op,unsigned long addr);extern unsigned long nsecs_to_usecs (unsigned long nsecs);
kernel/trace/trace_selftest.c +1 -1
@@ -828,7 +828,7 @@ trace_selftest_startup_ksym(struct tracer *trace, struct trace_array *tr) ksym_selftest_dummy =0 ;/* Register the read-write tracing request */ - ret =process_new_ksym_entry (KSYM_SELFTEST_ENTRY ,HW_BREAKPOINT_RW , + ret =process_new_ksym_entry ("ksym_selftest_dummy" ,HW_BREAKPOINT_RW , (unsigned long )(&ksym_selftest_dummy));if (ret <0 ) {
Signature · 1 line
1.6.2.3
PATCH 5/6 hw-breakpoints: Arbitrate access to pmu following registers constraints
In reply to Frederic WeisbeckerAllow or refuse to build a counter using the breakpoints pmu following given constraints.
We keep track of the pmu users by using three per cpu variables:
- nr_cpu_bp_pinned stores the number of pinned cpu breakpoints counters in the given cpu
- nr_bp_flexible stores the number of non-pinned breakpoints counters in the given cpu.
- task_bp_pinned stores the number of pinned task breakpoints in a cpu
The latter is not a simple counter but gathers the number of tasks that have n pinned breakpoints.
Considering HBP_NUM the number of available breakpoint address registers:
task_bp_pinned[0] is the number of tasks having 1 breakpoint task_bp_pinned[1] is the number of tasks having 2 breakpoints [...]
task_bp_pinned[HBP_NUM - 1] is the number of tasks having the maximum number of registers (HBP_NUM).
When a breakpoint counter is created and wants an access to the pmu, we evaluate the following constraints:
== Non-pinned counter ==
- If attached to a single cpu, check:
(per_cpu(nr_bp_flexible, cpu) || (per_cpu(nr_cpu_bp_pinned, cpu)
+ max(per_cpu(task_bp_pinned, cpu)))) < HBP_NUM
-> If there are already non-pinned counters in this cpu, it
means there is already a free slot for them. Otherwise, we check that the maximum number of per task breakpoints (for this cpu) plus the number of per cpu breakpoint (for this cpu) doesn't cover every registers.
- If attached to every cpus, check:
(per_cpu(nr_bp_flexible, *) || (max(per_cpu(nr_cpu_bp_pinned, *))
+ max(per_cpu(task_bp_pinned, *)))) < HBP_NUM
-> This is roughly the same, except we check the number of per
cpu bp for every cpu and we keep the max one. Same for the per tasks breakpoints.
== Pinned counter ==
- If attached to a single cpu, check:
((per_cpu(nr_bp_flexible, cpu) > 1)
+ per_cpu(nr_cpu_bp_pinned, cpu)
+ max(per_cpu(task_bp_pinned, cpu))) < HBP_NUM
-> Same checks as before. But now the nr_bp_flexible, if any,
must keep one register at least (or flexible breakpoints will never be be fed).
- If attached to every cpus, check:
((per_cpu(nr_bp_flexible, *) > 1)
+ max(per_cpu(nr_cpu_bp_pinned, *))
+ max(per_cpu(task_bp_pinned, *))) < HBP_NUM
Changes in v2:
- Counter -> event rename
--- kernel/hw_breakpoint.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 231 insertions(+), 6 deletions(-)
kernel/hw_breakpoint.c +231 -6
@@ -16,6 +16,8 @@* Copyright (C) 2007 Alan Stern * Copyright (C) IBM Corporation, 2009 * Copyright (C) 2009, Frederic Weisbecker <[email protected]> +* +* Thanks to Ingo Molnar for his many suggestions. */ /*
@@ -44,24 +46,247 @@#include <asm/debugreg.h> #endif -static atomic_t bp_slot; +/* +* Constraints data +*/ + +/* Number of pinned cpu breakpoints in a cpu */ +static DEFINE_PER_CPU(unsigned int, nr_cpu_bp_pinned); -int reserve_bp_slot (struct perf_event *bp) +/* Number of pinned task breakpoints in a cpu */ +static DEFINE_PER_CPU(unsigned int, task_bp_pinned[HBP_NUM]); + +/* Number of non-pinned cpu/task breakpoints in a cpu */ +static DEFINE_PER_CPU(unsigned int, nr_bp_flexible); + +/* Gather the number of total pinned and un-pinned bp in a cpuset */ +struct bp_busy_slots { +unsigned int pinned; +unsigned int flexible; +}; + +/* Serialize accesses to the above constraints */ +static DEFINE_MUTEX (nr_bp_mutex); + +/* +* Report the maximum number of pinned breakpoints a task +* have in this cpu +*/ +static unsigned int max_task_bp_pinned (int cpu) { -if (atomic_inc_return (&bp_slot) ==HBP_NUM ) { -atomic_dec (&bp_slot); +int i; +unsigned int *tsk_pinned =per_cpu (task_bp_pinned, cpu); -return -ENOSPC ; +for (i =HBP_NUM -1 ; i >=0 ; i--) { +if (tsk_pinned[i] >0 ) +return i +1 ; }return 0 ; } +/* +* Report the number of pinned/un-pinned breakpoints we have in +* a given cpu (cpu > -1) or in all of them (cpu = -1). +*/ +static void fetch_bp_busy_slots (struct bp_busy_slots *slots,int cpu) +{ +if (cpu >=0 ) { + slots->pinned =per_cpu (nr_cpu_bp_pinned, cpu); + slots->pinned +=max_task_bp_pinned (cpu); + slots->flexible =per_cpu (nr_bp_flexible, cpu); + +return ; + } + +for_each_online_cpu (cpu) { +unsigned int nr; + + nr =per_cpu (nr_cpu_bp_pinned, cpu); + nr +=max_task_bp_pinned (cpu); + +if (nr > slots->pinned) + slots->pinned = nr; + + nr =per_cpu (nr_bp_flexible, cpu); + +if (nr > slots->flexible) + slots->flexible = nr; + } +} + +/* +* Add a pinned breakpoint for the given task in our constraint table +*/ +static void toggle_bp_task_slot (struct task_struct *tsk,int cpu,bool enable) +{ +int count =0 ; +struct perf_event *bp; +struct perf_event_context *ctx = tsk->perf_event_ctxp; +unsigned int *task_bp_pinned; +struct list_head *list; +unsigned long flags; + +if (WARN_ONCE (!ctx,"No perf context for this task" )) +return ; + + list = &ctx->event_list; + +spin_lock_irqsave (&ctx->lock, flags); + +/* +* The current breakpoint counter is not included in the list +* at the open() callback time +*/ +list_for_each_entry (bp, list, event_entry) { +if (bp->attr.type ==PERF_TYPE_BREAKPOINT ) + count++; + } + +spin_unlock_irqrestore (&ctx->lock, flags); + +if (WARN_ONCE (count <0 ,"No breakpoint counter found in the counter list" )) +return ; + + task_bp_pinned =per_cpu (task_bp_pinned, cpu); +if (enable) { + task_bp_pinned[count]++; +if (count >0 ) + task_bp_pinned[count-1 ]--; + }else { + task_bp_pinned[count]--; +if (count >0 ) + task_bp_pinned[count-1 ]++; + } +} + +/* +* Add/remove the given breakpoint in our constraint table +*/ +static void toggle_bp_slot (struct perf_event *bp,bool enable) +{ +int cpu = bp->cpu; +unsigned int *nr; +struct task_struct *tsk = bp->ctx->task; + +/* Flexible */ +if (!bp->attr.pinned) { +if (cpu >=0 ) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle; + } + +for_each_online_cpu (cpu) { + nr = &per_cpu (nr_bp_flexible, cpu); + goto toggle; + } + } +/* Pinned counter task profiling */ +if (tsk) { +if (cpu >=0 ) { +toggle_bp_task_slot (tsk, cpu, enable); +return ; + } + +for_each_online_cpu (cpu) +toggle_bp_task_slot (tsk, cpu, enable); +return ; + } + +/* Pinned counter cpu profiling */ + nr = &per_cpu (nr_cpu_bp_pinned, bp->cpu); + +toggle: + *nr = enable ? *nr +1 : *nr -1 ; +} + +/* +* Contraints to check before allowing this new breakpoint counter: +* +* == Non-pinned counter == +* +* - If attached to a single cpu, check: +* +* (per_cpu(nr_bp_flexible, cpu) || (per_cpu(nr_cpu_bp_pinned, cpu) +* + max(per_cpu(task_bp_pinned, cpu)))) < HBP_NUM +* +* -> If there are already non-pinned counters in this cpu, it means +* there is already a free slot for them. +* Otherwise, we check that the maximum number of per task +* breakpoints (for this cpu) plus the number of per cpu breakpoint +* (for this cpu) doesn't cover every registers. +* +* - If attached to every cpus, check: +* +* (per_cpu(nr_bp_flexible, *) || (max(per_cpu(nr_cpu_bp_pinned, *)) +* + max(per_cpu(task_bp_pinned, *)))) < HBP_NUM +* +* -> This is roughly the same, except we check the number of per cpu +* bp for every cpu and we keep the max one. Same for the per tasks +* breakpoints. +* +* +* == Pinned counter == +* +* - If attached to a single cpu, check: +* +* ((per_cpu(nr_bp_flexible, cpu) > 1) + per_cpu(nr_cpu_bp_pinned, cpu) +* + max(per_cpu(task_bp_pinned, cpu))) < HBP_NUM +* +* -> Same checks as before. But now the nr_bp_flexible, if any, must keep +* one register at least (or they will never be fed). +* +* - If attached to every cpus, check: +* +* ((per_cpu(nr_bp_flexible, *) > 1) + max(per_cpu(nr_cpu_bp_pinned, *)) +* + max(per_cpu(task_bp_pinned, *))) < HBP_NUM +*/ +int reserve_bp_slot (struct perf_event *bp) +{ +struct bp_busy_slots slots = {0 }; +int ret =0 ; + +mutex_lock (&nr_bp_mutex); + +fetch_bp_busy_slots (&slots, bp->cpu); + +if (!bp->attr.pinned) { +/* +* If there are already flexible counters here, +* there is at least one slot reserved for all +* of them. Just join the party. +* +* Otherwise, check there is at least one free slot +*/ +if (!slots.flexible && slots.pinned ==HBP_NUM ) { + ret = -ENOSPC ; + goto end; + } + +/* Flexible counters need to keep at least one slot */ + }else if (slots.pinned + (!!slots.flexible) ==HBP_NUM ) { + ret = -ENOSPC ; + goto end; + } + +toggle_bp_slot (bp, true); + +end: +mutex_unlock (&nr_bp_mutex); + +return ret; +} +void release_bp_slot (struct perf_event *bp) { -atomic_dec (&bp_slot); +mutex_lock (&nr_bp_mutex); + +toggle_bp_slot (bp, false); + +mutex_unlock (&nr_bp_mutex); } +int __register_perf_hw_breakpoint (struct perf_event *bp) {int ret;
Signature · 1 line
1.6.2.3
PATCH 6/6 ksym_tracer: Remove KSYM_SELFTEST_ENTRY
In reply to Frederic WeisbeckerFrom: Li Zefan <[email protected]>
The macro used to be used in both trace_selftest.c and trace_ksym.c, but no longer, so remove it from header file.
--- kernel/trace/trace.h | 1 - kernel/trace/trace_selftest.c | 2 +- 2 files changed, 1 insertions(+), 2 deletions(-)
kernel/trace/trace.h +0 -1
@@ -370,7 +370,6 @@ int register_tracer(struct tracer *type);void unregister_tracer (struct tracer *type);int is_tracing_stopped (void ); -#define KSYM_SELFTEST_ENTRY "ksym_selftest_dummy"extern int process_new_ksym_entry (char *ksymname,int op,unsigned long addr);extern unsigned long nsecs_to_usecs (unsigned long nsecs);
kernel/trace/trace_selftest.c +1 -1
@@ -828,7 +828,7 @@ trace_selftest_startup_ksym(struct tracer *trace, struct trace_array *tr) ksym_selftest_dummy =0 ;/* Register the read-write tracing request */ - ret =process_new_ksym_entry (KSYM_SELFTEST_ENTRY ,HW_BREAKPOINT_RW , + ret =process_new_ksym_entry ("ksym_selftest_dummy" ,HW_BREAKPOINT_RW , (unsigned long )(&ksym_selftest_dummy));if (ret <0 ) {
Signature · 1 line
1.6.2.3
Quoted 30 lines · Frederic Weisbecker
Hi all,
This is the v2 of the hw-breakpoints API rewrite on top of perf events. You can find the previous version here:
http://lwn.net/Articles/351922/Changes in v2:
- Follow the perf "event " rename
- The ptrace regression have been fixed (ptrace breakpoint perf events weren't released when a task ended)
- Drop the structhw_breakpointand store generic fields inperf_event_attr.
- Separate core and arch specific headers, dropasm-generic/hw_breakpoint.hand createlinux/hw_breakpoint.h
- Use new generic len/type for breakpoint
- Handle off case: when breakpoints api is not supported by an arch
- Use proper in-kernel perf api provided by Arjan.There are still a lot of things that need to be cleaned, simplified, improved (ptrace side, the bp api, etc....) I guess these things can be done incrementally if you agree.
I've also tried to get an arch-independent api. Generic fields for breakpoints are stored in
perf_event_attrstructure (type, len, addr). This needs to be discussed and improved before it becomes a perf userspace ABI. We need to find a generic enough structure to host the breakpoints parameters, something that can better fit to most arch (handling breakpoint ranges in powerpc, etc...).
Quoted 8 lines
Thanks.
---
The following patchset are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing.git
perfevents/hw-breakpoint
BTW, this is a branch based on tip:tracing/hw_breakpoint with tip:perf/core merged inside.
Quoted 23 lines · Frederic Weisbecker
Hi all,
This is the v2 of the hw-breakpoints API rewrite on top of perf events. You can find the previous version here:
http://lwn.net/Articles/351922/Changes in v2:
- Follow the perf "event " rename
- The ptrace regression have been fixed (ptrace breakpoint perf events weren't released when a task ended)
- Drop the structhw_breakpointand store generic fields inperf_event_attr.
- Separate core and arch specific headers, dropasm-generic/hw_breakpoint.hand createlinux/hw_breakpoint.h
- Use new generic len/type for breakpoint
- Handle off case: when breakpoints api is not supported by an arch
- Use proper in-kernel perf api provided by Arjan.There are still a lot of things that need to be cleaned, simplified, improved (ptrace side, the bp api, etc....) I guess these things can be done incrementally if you agree.
Quoted 7 lines
I've also tried to get an arch-independent api. Generic fields for breakpoints are stored in
perf_event_attrstructure (type, len, addr). This needs to be discussed and improved before it becomes a perf userspace ABI. We need to find a generic enough structure to host the breakpoints parameters, something that can better fit to most arch (handling breakpoint ranges in powerpc, etc...).
Outside the specific comments about the implementation here, I think the patchset begets a larger question about hw-breakpoint layer's integration with perf-events.
Upon being a witness to the proposed changes and after some exploration of perf_events' functionality, I'm afraid that hw-breakpoint integration with perf doesn't benefit the former as much as originally wished to be
(http://lkml.org/lkml/2009/8/26/149).
Some of the prevalent concerns (which have been raised in different threads earlier) are:
- While kernel-space breakpoints need to reside on every processor (irrespective of the process in user-space), perf-events' notion of a counter is always linked to a process context (although there could be workarounds by making it 'pinned', etc).
- HW Breakpoints register allocation mechanism is 'greedy', which in my opinion is more suitable for allocating a finite and contended resource such as debug register while that of perf-events can give rise to roll-backs (with side-effects such as stray exceptions and race conditions).
- Given that the notion of a per-process context for counters is well-ingrained into the design of perf-events (even system-wide counters are sometimes implemented through individual syscalls over nr_cpus as in builtin-stat.c), it requires huge re-design and user-space changes.
Trying to scoop out the hw-breakpoint layer off its book-keeping/register allocation features only to replace with that of perf-events leads to a poor retrofit. On the other hand, an implementation to enable perf to use hw-breakpoint layer (and its APIs) to profile memory accesses over kernel-space variables (in the context of a process) is very elegant, modular and fits cleanly within the frame-work of the perf-events as a new perf-type (refer http://lkml.org/lkml/2009/10/26/467). A working patchset (under development and containing bugs) is posted for RFC here:
http://lkml.org/lkml/2009/10/26/461
It is my opinion that enhancing perf-layer to profile memory accesses through hw-breakpoint layer should be preferred over merging them.
Thanks,
K.Prasad
2009/10/26 K.Prasad <[email protected]>:
Quoted 9 lines
Outside the specific comments about the implementation here, I think the patchset begets a larger question about hw-breakpoint layer's integration with perf-events.
Upon being a witness to the proposed changes and after some exploration of
perf_events' functionality, I'm afraid that hw-breakpoint integration with perf doesn't benefit the former as much as originally wished to be
(http://lkml.org/lkml/2009/8/26/149).
Quoted 7 lines
Some of the prevalent concerns (which have been raised in different threads earlier) are:
- While kernel-space breakpoints need to reside on every processor (irrespective of the process in user-space), perf-events' notion of a counter is always linked to a process context (although there could be workarounds by making it 'pinned', etc).
No. A counter (let's talk about an event profiling instance now) is not always attached to a single process.
It is attached to a context. Such contexts are defined by perf as gathering a group of tasks or it can be a whole cpu.
The breakpoint API only supports two kind of contexts: one task, or every cpus (or per cpu after your last patchset).
That said, perf events can be enhanced to support the context of a wide counter.
Quoted 6 lines
- HW Breakpoints register allocation mechanism is 'greedy', which in my opinion is more suitable for allocating a finite and contended resource such as debug register while that of perf-events can give rise to roll-backs (with side-effects such as stray exceptions and race conditions).
I don't get your point. The only possible rollback is when we allocate a wide breakpoint (then one per cpu).
If you worry about such races, we can register these breakpoints as being disabled
and enable them once we know the allocation succeeded for every cpu.
Quoted 6 lines
- Given that the notion of a per-process context for counters is well-ingrained into the design of perf-events (even system-wide counters are sometimes implemented through individual syscalls over
nr_cpusas inbuiltin-stat.c), it requires huge re-design and user-space changes.
It doesn't require a huge redesign to support wide perf events.
Quoted 9 lines
Trying to scoop out the hw-breakpoint layer off its book-keeping/register allocation features only to replace with that of perf-events leads to a poor retrofit. On the other hand, an implementation to enable perf to use hw-breakpoint layer (and its APIs) to profile memory accesses over kernel-space variables (in the context of a process) is very elegant, modular and fits cleanly within the frame-work of the perf-events as a new perf-type (refer http://lkml.org/lkml/2009/10/26/467). A working patchset (under development and containing bugs) is posted for RFC here:
http://lkml.org/lkml/2009/10/26/461
The non-perf based api is fine for ptrace, kgdb and ftrace uses. But it is too limited for perf use.
- It has an ad-hoc context binding (register scheduling) abstraction. Perf is able to manage
that already: binding to defined group of processes, cpu, etc...
- It doesn't allow non-pinned events, when a breakpoint is disabled (due to context schedule out), it is
only virtually disabled, it's slot is not freed.
Basically, the breakpoints are performance monitoring and debug events. Something
that perf can already handle.
The current breakpoint API does all that in an ad-hoc way (debug register scheduling when cpu get up/down, when we context switch, etc...).
It is also not powerful enough to support non-pinned events.
The only downside I can see in perf events: it does not support wide system contexts.
I don't think it requires a huge redesign. But instead of continuing this ad-hoc context-handling
to cover this hole in perf, why not enhance perf so that it can cover that?
Quoted 19 lines · Frederic Weisbecker
2009/10/26 K.Prasad <[email protected]>:
Quoted 9 lines
Outside the specific comments about the implementation here, I think the patchset begets a larger question about hw-breakpoint layer's integration with perf-events.
Upon being a witness to the proposed changes and after some exploration of
perf_events' functionality, I'm afraid that hw-breakpoint integration with perf doesn't benefit the former as much as originally wished to be
(http://lkml.org/lkml/2009/8/26/149).Quoted 7 lines
Some of the prevalent concerns (which have been raised in different threads earlier) are:
- While kernel-space breakpoints need to reside on every processor (irrespective of the process in user-space), perf-events' notion of a counter is always linked to a process context (although there could be workarounds by making it 'pinned', etc).
Quoted 5 lines
No. A counter (let's talk about an event profiling instance now) is not always attached to a single process.
It is attached to a context. Such contexts are defined by perf as gathering a group of tasks or it can be a whole cpu.
Okay.
Quoted 3 lines
The breakpoint API only supports two kind of contexts: one task, or every cpus (or per cpu after your last patchset).
Yes, and please see the replies to your concerns below.
Quoted 11 lines
That said, perf events can be enhanced to support the context of a wide counter.
Quoted 6 lines
- HW Breakpoints register allocation mechanism is 'greedy', which in my opinion is more suitable for allocating a finite and contended resource such as debug register while that of perf-events can give rise to roll-backs (with side-effects such as stray exceptions and race conditions).
Quoted 7 lines
I don't get your point. The only possible rollback is when we allocate a wide breakpoint (then one per cpu).
If you worry about such races, we can register these breakpoints as being disabled
and enable them once we know the allocation succeeded for every cpu.
Not just stray exceptions, as explained before here:
http://lkml.org/lkml/2009/10/1/76
- Races between the requests (also leading to temporary failure of all CPU requests) presenting an unclear picture about free debug registers (making it difficult to predict the need for a retry).
Quoted 11 lines
Quoted 6 lines
- Given that the notion of a per-process context for counters is well-ingrained into the design of perf-events (even system-wide counters are sometimes implemented through individual syscalls over
nr_cpusas inbuiltin-stat.c), it requires huge re-design and user-space changes.It doesn't require a huge redesign to support wide perf events.
I contest that :-)...and the sheer amount of code movement, re-design (including core data structures) in the patchset here:
http://lkml.org/lkml/2009/10/24/53.
And all this with a loss of a well-layered, modular code and a loss of true system-wide support for bkpt counters!
Quoted 11 lines
Quoted 9 lines
Trying to scoop out the hw-breakpoint layer off its book-keeping/register allocation features only to replace with that of perf-events leads to a poor retrofit. On the other hand, an implementation to enable perf to use hw-breakpoint layer (and its APIs) to profile memory accesses over kernel-space variables (in the context of a process) is very elegant, modular and fits cleanly within the frame-work of the perf-events as a new perf-type (refer http://lkml.org/lkml/2009/10/26/467). A working patchset (under development and containing bugs) is posted for RFC here:
http://lkml.org/lkml/2009/10/26/461
Quoted 7 lines
The non-perf based api is fine for ptrace, kgdb and ftrace uses. But it is too limited for perf use.
- It has an ad-hoc context binding (register scheduling) abstraction. Perf is able to manage
that already: binding to defined group of processes, cpu, etc...
I don't see what's ad-hoc in the scheduling behaviour of the hw-bkpt layer. Hw-breakpoint layer does the following with respect to register scheduling:
- User-space breakpoints are always tied to a thread (thread_info/task_struct) and are hence active when the corresponding thread is scheduled.
- Kernel-space addresses (requests from in-kernel sources) should be always active and aren't affected by process context-switches/schedule operations. Some of the sophisticated mechanisms for scheduling kernel vs user-space breakpoints (such as trapping syscalls to restore register context) were pre-empted by the community (as seen here:
http://lkml.org/lkml/2009/3/11/145).
Any further abstraction required by the end-user (as in the case of perf) can be well-implemented through the powerful breakpoint interfaces. For instance - perf-events with its unique requirement wherein a kernel-space breakpoint need to be active only when a given process is active. Hardware breakpoint layer handles them quite well as seen here: http://lkml.org/lkml/2009/10/29/300.
Quoted 4 lines
- It doesn't allow non-pinned events, when a breakpoint is disabled (due to context schedule out), it is
only virtually disabled, it's slot is not freed.
The <enable><disable>_hw_breakpoint() are designed such. If a user want the slot to be freed (which is ill-advised for a requirement here) it can invoke (un)register_kernel_hw_breakpoint() instead (would have very little overhead for the 1-CPU case without IPIs).
Quoted 9 lines
Basically, the breakpoints are performance monitoring and debug events. Something
that perf can already handle.The current breakpoint API does all that in an ad-hoc way (debug register scheduling when cpu get up/down, when we context switch, etc...).
It is also not powerful enough to support non-pinned events.
Quoted 5 lines
The only downside I can see in perf events: it does not support wide system contexts.
I don't think it requires a huge redesign. But instead of continuing this ad-hoc context-handling
to cover this hole in perf, why not enhance perf so that it can cover that?
The advantages of having perf-events to use hw-breakpoint layer is explained here and in many of my previous emails. It entails no loss of functionality for either perf-events of hw-breakpoints, while allowing users to harness the power of both.
Thanks,
K.Prasad
Quoted 8 lines · K.Prasad
Quoted 7 lines
I don't get your point. The only possible rollback is when we allocate a wide breakpoint (then one per cpu).
If you worry about such races, we can register these breakpoints as being disabled
and enable them once we know the allocation succeeded for every cpu.
Quoted 5 lines
Not just stray exceptions, as explained before here:
http://lkml.org/lkml/2009/10/1/76
- Races between the requests (also leading to temporary failure of all CPU requests) presenting an unclear picture about free debug registers (making it difficult to predict the need for a retry).
Ok. But say we have to set a wide breakpoint.
We can create a disabled set of per cpu breakpoint and then enable them once we are sure every cpus can host it (we have already reserved a slot for each of these while registering).
Then this race is not there anymore.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
- Given that the notion of a per-process context for counters is well-ingrained into the design of perf-events (even system-wide counters are sometimes implemented through individual syscalls over
nr_cpusas inbuiltin-stat.c), it requires huge re-design and user-space changes.
Quoted 7 lines
Quoted 3 lines
It doesn't require a huge redesign to support wide perf events.
I contest that :-)...and the sheer amount of code movement, re-design (including core data structures) in the patchset here:
http://lkml.org/lkml/2009/10/24/53.
This is about rebasing the hw-breakpoints on top of another profiling infrastructure.
So the fact we had to do a lot of changes looks fair.
Quoted 2 lines
And all this with a loss of a well-layered, modular code and a loss of true system-wide support for bkpt counters!
I don't get your point about the loss of a well-layered and modular code.
We are reusing an existing profiling infrastructure that looks pretty well layered and modular to me:
The scheduling of per task registers is centralized in the core and not in the arch like it was done with the hardware breakpoint api. The remaining arch bits are only a concern of writing these registers.
It is not sane to hook on arch switch_to(), cpu hotplug helpers, kexec, etc... to do an ad-hoc scheduling of perf task register whereas we already have a centralized profiling area that can take these decisions.
So, yes indeed it doesn't support the wide contexts yet.
Let's compare that to the tracing area. What if we hadn't the tracepoints and every tracers put their own tracing callbacks in the area they want to trace. We would have a proliferation of ad-hoc tracing functions calls.
But we have the tracepoints: a centralized feature that only requires just one callback somewhere in the kernel where we want to hook up and in which every tracers can subscribe.
That's more modular and well-layered.
The problem with the lack of a wide context support with perf events is pretty the same.
The hw-breakpoint api can implement its ad-hoc one. But it means every other profiling/debug features will lack it and need to implement their own.
So why not improving a centralized profiling subsystem instead of implementing an ad-hoc one for every profiling classes that need it?
Quoted 8 lines
Quoted 7 lines
The non-perf based api is fine for ptrace, kgdb and ftrace uses. But it is too limited for perf use.
- It has an ad-hoc context binding (register scheduling) abstraction. Perf is able to manage
that already: binding to defined group of processes, cpu, etc...
Quoted 7 lines
I don't see what's ad-hoc in the scheduling behaviour of the hw-bkpt layer. Hw-breakpoint layer does the following with respect to register scheduling:
- User-space breakpoints are always tied to a thread (thread_info/task_struct) and are hence active when the corresponding thread is scheduled.
This is what is ad-hoc. You need to hook on switch_to, cpu hotplug and kexec to update the breakpoints registers. And this is something that would need to be done in every archs. That looks insane considering the fact we have a core layer that can handle these decisions already.
Quoted 6 lines
- Kernel-space addresses (requests from in-kernel sources) should be always active and aren't affected by process context-switches/schedule operations. Some of the sophisticated mechanisms for scheduling kernel vs user-space breakpoints (such as trapping syscalls to restore register context) were pre-empted by the community (as seen here:
http://lkml.org/lkml/2009/3/11/145).
Sure. And things have evolved since then. We have a centralized profiling/event susbsystem now.
Quoted 6 lines
Any further abstraction required by the end-user (as in the case of perf) can be well-implemented through the powerful breakpoint interfaces. For instance - perf-events with its unique requirement wherein a kernel-space breakpoint need to be active only when a given process is active. Hardware breakpoint layer handles them quite well as seen here: http://lkml.org/lkml/2009/10/29/300.
It logically disables/enables the breakpoints but not physically. Which means a disabled breakpoint still keeps its slot, making it unavailable for another event, it i required for non-pinned events.
Quoted 9 lines
Quoted 4 lines
- It doesn't allow non-pinned events, when a breakpoint is disabled (due to context schedule out), it is
only virtually disabled, it's slot is not freed.The <enable><disable>_hw_breakpoint() are designed such. If a user want the slot to be freed (which is ill-advised for a requirement here) it can invoke (un)register_kernel_hw_breakpoint() instead (would have very little overhead for the 1-CPU case without IPIs).
This adds unnecessary overhead. All we want is to update arch registers when we schedule in/out an event.
We need to be able to free a slot for non-pinned counter because an undefined number of events must be able to be time shared in a single slot (in the worst case).
Calling unregister_kernel_breakpoint() each time we schedule out a non-pinned counter adds unnecessary overhead. And calling register_kernel_breakpoint() while enabling one is yet another unnecessary overhead.
You'd need to check the free slots constraints every time for them.
Quoted 17 lines · Frederic Weisbecker
Quoted 8 lines · K.Prasad
Quoted 7 lines
I don't get your point. The only possible rollback is when we allocate a wide breakpoint (then one per cpu).
If you worry about such races, we can register these breakpoints as being disabled
and enable them once we know the allocation succeeded for every cpu.Quoted 5 lines
Not just stray exceptions, as explained before here:
http://lkml.org/lkml/2009/10/1/76
- Races between the requests (also leading to temporary failure of all CPU requests) presenting an unclear picture about free debug registers (making it difficult to predict the need for a retry).
Quoted 7 lines
Ok. But say we have to set a wide breakpoint.
We can create a disabled set of per cpu breakpoint and then enable them once we are sure every cpus can host it (we have already reserved a slot for each of these while registering).Then this race is not there anymore.
Let me explain further with an illustration..assume two contenders for breakpoints say A and B on a machine with 4 CPUs (0-3). Due to several prior per-CPU requests CPU0 has one free debug register while CPU3 has none. 'A' asks for a system-wide breakpoint while 'B' requests for a per-cpu breakpoint on CPU0. Now think of a race between them! If 'A' begins first, it starts with CPU0 and proceeds by consuming debug registers on CPUs in ascending order, meanwhile 'B' would return thinking that its request cannot be serviced. However 'A' would eventually fail (since there are no free debug registers on CPU3) and relinquish the debug register on CPU0 also (during rollback), but 'B' has returned thinking that there are no free debug registers available. (registering breakpoint requests in disabled state only helps prevent stray exceptions).
With some book-keeping such races are prevented. Registration requests would fail with -ENOSPC only if there are genuine users of debug registers and not because some potential user (who might eventually rollback) is holding onto the register temporarily.
Such a limitation may not be of immediate worry, but shouldn't be a reason to throw away a feature that pre-empts such concerns. Are there any hw-registers managed by perf-events that are more than one (like debug registers which are 4 in x86) with such peculiar needs?
Quoted 34 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
- Given that the notion of a per-process context for counters is well-ingrained into the design of perf-events (even system-wide counters are sometimes implemented through individual syscalls over
nr_cpusas inbuiltin-stat.c), it requires huge re-design and user-space changes.Quoted 7 lines
Quoted 3 lines
It doesn't require a huge redesign to support wide perf events.
I contest that :-)...and the sheer amount of code movement, re-design (including core data structures) in the patchset here:
http://lkml.org/lkml/2009/10/24/53.This is about rebasing the hw-breakpoints on top of another profiling infrastructure.
So the fact we had to do a lot of changes looks fair.
Quoted 2 lines
And all this with a loss of a well-layered, modular code and a loss of true system-wide support for bkpt counters!
I don't get your point about the loss of a well-layered and modular code.
We are reusing an existing profiling infrastructure that looks pretty well layered and modular to me:
Quoted 9 lines
The scheduling of per task registers is centralized in the core and not in the arch like it was done with the hardware breakpoint api. The remaining arch bits are only a concern of writing these registers.
It is not sane to hook on archswitch_to(), cpu hotplug helpers, kexec, etc... to do an ad-hoc scheduling of perf task register whereas we already have a centralized profiling area that can take these decisions.So, yes indeed it doesn't support the wide contexts yet.
User-space scheduling of breakpoint register is the closest that comes to what perf-events already does...and that's just about a single-hook in __switch_to(). Have no delusions about huge duplication! and no concerns about arch-specific code being littered all over - writing onto debug registers is a processor-specific activity and hence the arch-specific invocation.
System-wide breakpoints, cpu hotplug helpers, kexec hooks as you mentioned have not been implemented for perf-events....and in a way it is of little help there other than for hw-breakpoints (are there any hw-registers managed by perf that have residual effect i.e. continue to generate exceptions like hw-breakpoints?)
Quoted 8 lines
Let's compare that to the tracing area. What if we hadn't the tracepoints and every tracers put their own tracing callbacks in the area they want to trace. We would have a proliferation of ad-hoc tracing functions calls.
But we have the tracepoints: a centralized feature that only requires just one callback somewhere in the kernel where we want to hook up and in which every tracers can subscribe.
That's more modular and well-layered.
Comparing this to tracepoints isn't apt here. My limited knowledge doesn't quickly provide me with an alternate analogy (how about kprobes
+ perf-events integration?...no, even that isn't close).
Quoted 6 lines
The problem with the lack of a wide context support with perf events is pretty the same.
The hw-breakpoint api can implement its ad-hoc one. But it means every other profiling/debug features will lack it and need to implement their own.
Can you cite the need for such features in general perf-events architecture with examples other than hw-breakpoints? In my opinion, if there had been a need, perf-events would have included them already (perf top is the only need that comes close to wanting system-wide support but even there, it is happy by making one syscall per-cpu).
Integrating the features required by hw-breakpoints with perf-events (with apologies for the out-of-context examples) like mixing oil with water, proverbial chalk-and-cheese....they stay together but are immiscible.
Citing some examples from your patchset, look at the addition of 'callback' function pointer or the addition of length, type fields in perf_event_attr. Do you find generic use-cases for them in perf-events (outside hw-breakpoints)? Merging structures to create a generic one, but only to be used for a specific use-case (hw-breakpoint) doesn't sound like a good idea, and speculating on future use-cases (not current ones) have never been welcomed.
Quoted 23 lines
So why not improving a centralized profiling subsystem instead of implementing an ad-hoc one for every profiling classes that need it?
Quoted 8 lines
Quoted 7 lines
The non-perf based api is fine for ptrace, kgdb and ftrace uses. But it is too limited for perf use.
- It has an ad-hoc context binding (register scheduling) abstraction. Perf is able to manage
that already: binding to defined group of processes, cpu, etc...Quoted 7 lines
I don't see what's ad-hoc in the scheduling behaviour of the hw-bkpt layer. Hw-breakpoint layer does the following with respect to register scheduling:
- User-space breakpoints are always tied to a thread (thread_info/task_struct) and are hence active when the corresponding thread is scheduled.
Quoted 6 lines
This is what is ad-hoc. You need to hook on
switch_to, cpu hotplug and kexec to update the breakpoints registers. And this is something that would need to be done in every archs. That looks insane considering the fact we have a core layer that can handle these decisions already.
As explained above.
Quoted 44 lines
Quoted 7 lines
- Kernel-space addresses (requests from in-kernel sources) should be always active and aren't affected by process context-switches/schedule operations. Some of the sophisticated mechanisms for scheduling kernel vs user-space breakpoints (such as trapping syscalls to restore register context) were pre-empted by the community (as seen here:
http://lkml.org/lkml/2009/3/11/145).Sure. And things have evolved since then. We have a centralized profiling/event susbsystem now.
Quoted 6 lines
Any further abstraction required by the end-user (as in the case of perf) can be well-implemented through the powerful breakpoint interfaces. For instance - perf-events with its unique requirement wherein a kernel-space breakpoint need to be active only when a given process is active. Hardware breakpoint layer handles them quite well as seen here: http://lkml.org/lkml/2009/10/29/300.
It logically disables/enables the breakpoints but not physically. Which means a disabled breakpoint still keeps its slot, making it unavailable for another event, it i required for non-pinned events.
Quoted 9 lines
Quoted 4 lines
- It doesn't allow non-pinned events, when a breakpoint is disabled (due to context schedule out), it is
only virtually disabled, it's slot is not freed.The <enable><disable>_hw_breakpoint() are designed such. If a user want the slot to be freed (which is ill-advised for a requirement here) it can invoke (un)register_kernel_hw_breakpoint() instead (would have very little overhead for the 1-CPU case without IPIs).
This adds unnecessary overhead. All we want is to update arch registers when we schedule in/out an event.
We need to be able to free a slot for non-pinned counter because an undefined number of events must be able to be time shared in a single slot (in the worst case).
Quoted 7 lines
Calling
unregister_kernel_breakpoint()each time we schedule out a non-pinned counter adds unnecessary overhead. And callingregister_kernel_breakpoint()while enabling one is yet another unnecessary overhead.You'd need to check the free slots constraints every time for them.
As I told you register/unregister combination is to be used, and I can get you some numbers about its overhead if that is of concern to you. It is the IPI constitues a large overhead (not needed for a 1-cpu request), and not the book-keeping work. A more elegant way would be to use the modify_kernel_hw_breakpoint() (interface submitted in one of my previous patches) to simply change the breakpoint address/type/len.
I don't see anything that is required by the perf-event that the hw-breakpoint layer doesn't provide.
I shall re-state these views in a response to Ingo's mail (that talks about concerns of duplicate code)...meanwhile I think I should begin reviewing your patchset (for perf-integration) lest the community insist that approach.
Thanks,
K.Prasad
Quoted 14 lines · K.Prasad
Let me explain further with an illustration..assume two contenders for breakpoints say A and B on a machine with 4 CPUs (0-3). Due to several prior per-CPU requests CPU0 has one free debug register while CPU3 has none. 'A' asks for a system-wide breakpoint while 'B' requests for a per-cpu breakpoint on CPU0. Now think of a race between them! If 'A' begins first, it starts with CPU0 and proceeds by consuming debug registers on CPUs in ascending order, meanwhile 'B' would return thinking that its request cannot be serviced. However 'A' would eventually fail (since there are no free debug registers on CPU3) and relinquish the debug register on CPU0 also (during rollback), but 'B' has returned thinking that there are no free debug registers available. (registering breakpoint requests in disabled state only helps prevent stray exceptions).
Quoted 4 lines
With some book-keeping such races are prevented. Registration requests would fail with
-ENOSPConly if there are genuine users of debug registers and not because some potential user (who might eventually rollback) is holding onto the register temporarily.
True. But such race would only happen in case of concurrent launching of perf and the ksym_tracer or kgdb in parallel.
I can't figure out such situation to happen easily.
But if that's really a worry, we can lock the register_breakpoint_* path (or implement wide perf events).
Quoted 2 lines
Such a limitation may not be of immediate worry, but shouldn't be a reason to throw away a feature that pre-empts such concerns.
When you migrate a feature on top of another subsystem, it happens you can lose something. If the migration induces nice new features then sometimes it's worth the migration and add some code to break the loss.
But if the loss is about such a tiny tight race that is unlikely to happen in the real world (and doesn't imply crash or something bad like that) then it's not necessary worth the effort.
Quoted 3 lines
Are there
any hw-registers managed by perf-events that are more than one (like debug registers which are 4 in x86) with such peculiar needs?
Perhaps. I don't know. If so, we may want to migrate the plural pmu constraints from hw_breakpoint.c to perf_event.c.
That said this is likely to happen because we don't need registers for that. Any profiling/tracing/debugging unit based on multiple sources and bound to tunable contexts may fit into that scheme.
Hardware registers are just a subset of what can be considered as a a source of "performance monitoring unit".
And perf event has been extended enough to expand the possible sources to "Event monitoring unit", so the possibilities are broad.
Quoted 6 lines
User-space scheduling of breakpoint register is the closest that comes to what perf-events already does...and that's just about a single-hook in
__switch_to(). Have no delusions about huge duplication! and no concerns about arch-specific code being littered all over - writing onto debug registers is a processor-specific activity and hence the arch-specific invocation.
That's because it's so close to perf event context scheduling that we want to unify it.
This is not only a single hook in __switch_to():
- This is a context bound ressource scheduling decision made from arch in spite of the existing optimized mechanisms implemented in a core profiling subsystem.
- This is a single hook in __switch_to() in x86. Now multiply that by $(ls arch/ | wc -l) = 26 (-1 because of arch/Kconfig) Also, expand that to cpu hotplug hooks, kexec hooks, etc... And also include the free_thread hooks.
- If we have a ptrace breakpoint, the scheduling decision is made by the hw_breakpoint API. Otherwise it's made by perf. Why should we maintain two versions of the scheduling decision?
This is a matter of maintainabilty.
Quoted 5 lines
System-wide breakpoints, cpu hotplug helpers, kexec hooks as you mentioned have not been implemented for perf-events....and in a way it is of little help there other than for hw-breakpoints (are there any hw-registers managed by perf that have residual effect i.e. continue to generate exceptions like hw-breakpoints?)
System wide events are not supported by perf because of a design decision for scalability ends.
Each event is bound to a private buffer (inherited to task childs in the case of task bound counters).
If we have an event that can trigger from every cpu, we can have multiple concurrent writes in the same buffer and the profiling would suffer from such contention if we have a lot of events from several cpus.
Having cpu bound events drop this contention as the events don't fight against other cpus.
(It the case of task bound events, the contention is there, but in the window of a task group only).
We manage a wide profiling using a collection of per cpu events, it scales way much better.
We could also implement the wide context if that becomes wanted but it should be used by knowing that it won't scale for high frequency events in SMP.
Concerning cpu hotplug helpers, it is implemented by perf events (perf_event_{exit/init}_cpu() notifiers).
Kexec is a corner case, but we might want to add a kexec callback to the pmu structure if needed.
Concerning residual effects of lazy breakpoint registers switching I don't how the migration to perf brings any problem.
Quoted 3 lines
Comparing this to tracepoints isn't apt here. My limited knowledge doesn't quickly provide me with an alternate analogy (how about kprobes
+ perf-events integration?...no, even that isn't close).
Why isn't the tracepoint analogy apt?
It's basically the same.
Say you have func1(), a function in which several subsystems want to hook:
func1()
{
// do something cool
subsys1_hook();
subsys2_hook();
subsys3_hook();
etc...
}Instead of that, we can use tracepoint, so that we can zap all these hooks and only provide one that will dispatch to the subsys:
func1()
{
trace_func1() -> will dispatch to subsys1_hook(), subsys2_hook() etc..
}The current situation with hw breakpoints and perf is somehow the same. Hw breakpoint is hooking at arch __switch_to(), cpu_hotplug thing, etc...
But perf too. And perf can act as a dispatcher there. It already hooks on the scheduler events and cpu hotplug and can take centralized decisions from these hooks, such as toggling pmus, and hardware breakpoints can fit there.
Quoted 5 lines
Can you cite the need for such features in general perf-events architecture with examples other than hw-breakpoints? In my opinion, if there had been a need, perf-events would have included them already (perf top is the only need that comes close to wanting system-wide support but even there, it is happy by making one syscall per-cpu).
It is happy doing so because it scales.
Quoted 5 lines
Integrating the features required by hw-breakpoints with perf-events (with apologies for the out-of-context examples) like mixing oil with water, proverbial chalk-and-cheese....they stay together but are immiscible.
Quoted 7 lines
Citing some examples from your patchset, look at the addition of 'callback' function pointer or the addition of length, type fields in
perf_event_attr. Do you find generic use-cases for them in perf-events (outside hw-breakpoints)? Merging structures to create a generic one, but only to be used for a specific use-case (hw-breakpoint) doesn't sound like a good idea, and speculating on future use-cases (not current ones) have never been welcomed.
We need to give the parameters for this specific event. The current struct perf_event_attr is not sufficient for that.
It needs to grow if needed to host new needs for new events. That's why it has a field for its size, that's why it has reserved fields, I don't see where is the problem with that. Nothing in the breakpoint API can help about that either. We still need a suitable userspace gate to define a breakpoint.
And while at it, we reuse it for in-kernel uses.
If you look at the struct hw_perf_event, you'll find a protean structure, considering the union inside, so that it can fit the needs for two different type of events.
Quoted 6 lines
As I told you register/unregister combination is to be used, and I can get you some numbers about its overhead if that is of concern to you. It is the IPI constitues a large overhead (not needed for a 1-cpu request), and not the book-keeping work. A more elegant way would be to use the
modify_kernel_hw_breakpoint()(interface submitted in one of my previous patches) to simply change the breakpoint address/type/len.
And btw the IPI that binds an event to a cpu is another example of something already managed by perf.
My real concern is about the fact that the hw-breakpoint api would act as an unnecessary midlayer there.
Perf is already able to talk directly to the pmu to enable/disable it, which in practice is just a write to the debug registers.
Why should we encumber with such midlayer?