Introducing generic hardware breakpoint handler interfaces
Subsystems (5)
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 1/11 Introducing generic hardware breakpoint handler interfaces
From: Alan Stern <[email protected]>
This patch introduces two new files hw_breakpoint.[ch] which defines the generic interfaces to use hardware breakpoint infrastructure of the system.
[K.Prasad: Re-based the original patch to newer kernel base, modified the
register_<kernel/user>_hw_breakpoint() interfaces and split the monolithic patch into smaller ones. Split-out from the bigger patch and minor changes following re-basing]
--- include/asm-generic/hw_breakpoint.h | 243 +++++++++++ kernel/Makefile | 2 kernel/hw_breakpoint.c | 772 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1016 insertions(+), 1 deletion(-)
kernel/hw_breakpoint.c new file +772 -0
@@ -0,0 +1,772 @@ +/* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +* +* Copyright (C) 2007 Alan Stern +* Copyright (C) IBM Corporation, 2009 +*/ + +/* +* HW_breakpoint: a unified kernel/user-space hardware breakpoint facility, +* using the CPU's debug registers. +* +* This file contains the arch-independent routines. It is not meant +* to be compiled as a standalone source file; rather it should be +* #include'd by the arch-specific implementation. +*/ + +#include <linux/init.h> +#include <linux/irqflags.h> +#include <linux/kdebug.h> +#include <linux/kernel.h> +#include <linux/kprobes.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/notifier.h> +#include <linux/rculist.h> +#include <linux/sched.h> +#include <linux/smp.h> +#include <linux/percpu.h> +#include <linux/kallsyms.h> + +#include <asm/debugreg.h> +#include <asm/hw_breakpoint.h> +#include <asm/processor.h> + +/* Global info */ +struct kernel_bp_data kbpdata[2 ];/* Old and new settings */ +int cur_kbpindex;/* Alternates 0, 1, ... */ +struct kernel_bp_data *cur_kbpdata = &kbpdata[0 ]; +/* Always equal to &kbpdata[cur_kbpindex] */ + +static u8 tprio[HB_NUM ];/* Thread bp max priorities */ +LIST_HEAD (kernel_bps);/* Kernel breakpoint list */ +static LIST_HEAD (thread_list);/* thread_hw_breakpoint list */ +DEFINE_PER_CPU(struct cpu_hw_breakpoint, cpu_bp); + +/* +* Install the debug register values for a new thread. +*/ +void switch_to_thread_hw_breakpoint (struct task_struct *tsk) +{ +struct thread_hw_breakpoint *thbi = tsk->thread.hw_breakpoint_info; +struct cpu_hw_breakpoint *chbi; +struct kernel_bp_data *thr_kbpdata; + +/* This routine is on the hot path; it gets called for every +* context switch into a task with active breakpoints. We +* must make sure that the common case executes as quickly as +* possible. +*/ + chbi = &per_cpu (cpu_bp,get_cpu ()); + chbi->bp_task = tsk; + +/* Use RCU to synchronize with external updates */ +rcu_read_lock (); + +/* Other CPUs might be making updates to the list of kernel +* breakpoints at this time. If they are, they will modify +* the other entry in kbpdata[] -- the one not pointed to +* by chbi->cur_kbpdata. So the update itself won't affect +* us directly. +* +* However when the update is finished, an IPI will arrive +* telling this CPU to change chbi->cur_kbpdata. We need +* to use a single consistent kbpdata[] entry, the present one. +* So we'll copy the pointer to a local variable, thr_kbpdata, +* and we must prevent the compiler from aliasing the two +* pointers. Only a compiler barrier is required, not a full +* memory barrier, because everything takes place on a single CPU. +*/ + restart: + thr_kbpdata =ACCESS_ONCE (chbi->cur_kbpdata); + +/* Normally we can keep the same debug register settings as the +* last time this task ran. But if the kernel breakpoints have +* changed or any user breakpoints have been registered or +* unregistered, we need to handle the updates and possibly +* send out some notifications. +*/ +if (unlikely (thbi->gennum != thr_kbpdata->gennum)) { +struct hw_breakpoint *bp; +int i; +int num; + + thbi->gennum = thr_kbpdata->gennum; +arch_update_thbi (thbi, thr_kbpdata); + num = thr_kbpdata->num_kbps; + +/* This code can be invoked while a debugger is actively +* updating the thread's breakpoint list. We use RCU to +* protect our access to the list pointers. */ + thbi->num_installed =0 ; + i =HB_NUM ; +list_for_each_entry_rcu (bp, &thbi->thread_bps, node) { + +/* If this register is allocated for kernel bps, +* don't install. Otherwise do. */ +if (--i < num) { +if (bp->status ==HW_BREAKPOINT_INSTALLED ) { +if (bp->uninstalled) + (bp->uninstalled)(bp); + bp->status =HW_BREAKPOINT_REGISTERED ; + } + }else { + ++thbi->num_installed; +if (bp->status !=HW_BREAKPOINT_INSTALLED ) { + bp->status =HW_BREAKPOINT_INSTALLED ; +if (bp->installed) + (bp->installed)(bp); + } + } + } + } + +/* Set the debug register */ +arch_install_thbi (thbi); + +/* Were there any kernel breakpoint changes while we were running? */ +if (unlikely (chbi->cur_kbpdata != thr_kbpdata)) { + +/* Some debug registers now be assigned to kernel bps and +* we might have messed them up. Reload all the kernel bps +* and then reload the thread bps. +*/ +arch_install_chbi (chbi); + goto restart; + } + +rcu_read_unlock (); +put_cpu_no_resched (); +} + +/* +* Install the debug register values for just the kernel, no thread. +*/ +void switch_to_none_hw_breakpoint (void ) +{ +struct cpu_hw_breakpoint *chbi; + + chbi = &per_cpu (cpu_bp,get_cpu ()); + chbi->bp_task =NULL ; + +/* This routine gets called from only two places. In one +* the caller holds the hw_breakpoint_mutex; in the other +* interrupts are disabled. In either case, no kernel +* breakpoint updates can arrive while the routine runs. +* So we don't need to use RCU. +*/ +arch_install_none (chbi); +put_cpu_no_resched (); +} + +/* +* Update the debug registers on this CPU. +*/ +static void update_this_cpu (void *unused) +{ +struct cpu_hw_breakpoint *chbi; +struct task_struct *tsk = current; + + chbi = &per_cpu (cpu_bp,get_cpu ()); + +rcu_read_lock (); +/* Install both the kernel and the user breakpoints */ +arch_install_chbi (chbi); +if (test_tsk_thread_flag (tsk,TIF_DEBUG )) +switch_to_thread_hw_breakpoint (tsk); + +rcu_read_unlock (); +put_cpu_no_resched (); +} + +/* +* Tell all CPUs to update their debug registers. +* +* The caller must hold hw_breakpoint_mutex. +*/ +static void update_all_cpus (void ) +{ +/* We don't need to use any sort of memory barrier. The IPI +* carried out by on_each_cpu() includes its own barriers. +*/ +on_each_cpu (update_this_cpu,NULL ,0 ); +synchronize_rcu (); +} + +/* +* Load the debug registers during startup of a CPU. +*/ +void load_debug_registers (void ) +{ +unsigned long flags; + +/* Prevent IPIs for new kernel breakpoint updates */ +local_irq_save (flags); +update_this_cpu (NULL ); +local_irq_restore (flags); +} + +/* +* Take the 4 highest-priority breakpoints in a thread and accumulate +* their priorities in tprio. Highest-priority entry is in tprio[3]. +*/ +static void accum_thread_tprio (struct thread_hw_breakpoint *thbi) +{ +int i; + +for (i =HB_NUM -1 ; i >=0 && thbi->bps[i]; --i) + tprio[i] =max (tprio[i], thbi->bps[i]->priority); +} + +/* +* Recalculate the value of the tprio array, the maximum priority levels +* requested by user breakpoints in all threads. +* +* Each thread has a list of registered breakpoints, kept in order of +* decreasing priority. We'll set tprio[0] to the maximum priority of +* the first entries in all the lists, tprio[1] to the maximum priority +* of the second entries in all the lists, etc. In the end, we'll know +* that no thread requires breakpoints with priorities higher than the +* values in tprio. +* +* The caller must hold hw_breakpoint_mutex. +*/ +static void recalc_tprio (void ) +{ +struct thread_hw_breakpoint *thbi; + +memset (tprio,0 ,sizeof tprio); + +/* Loop through all threads having registered breakpoints +* and accumulate the maximum priority levels in tprio. +*/ +list_for_each_entry (thbi, &thread_list, node) +accum_thread_tprio (thbi); +} + +/* +* Decide how many debug registers will be allocated to kernel breakpoints +* and consequently, how many remain available for user breakpoints. +* +* The priorities of the entries in the list of registered kernel bps +* are compared against the priorities stored in tprio[]. The 4 highest +* winners overall get to be installed in a debug register; num_kpbs +* keeps track of how many of those winners come from the kernel list. +* +* If num_kbps changes, or if a kernel bp changes its installation status, +* then call update_all_cpus() so that the debug registers will be set +* correctly on every CPU. If neither condition holds then the set of +* kernel bps hasn't changed, and nothing more needs to be done. +* +* The caller must hold hw_breakpoint_mutex. +*/ +static void balance_kernel_vs_user (void ) +{ +int k, u; +int changed =0 ; +struct hw_breakpoint *bp; +struct kernel_bp_data *new_kbpdata; + +/* Determine how many debug registers are available for kernel +* breakpoints as opposed to user breakpoints, based on the +* priorities. Ties are resolved in favor of user bps. +*/ + k =0 ;/* Next kernel bp to allocate */ + u =HB_NUM -1 ;/* Next user bp to allocate */ + + bp = list_entry(kernel_bps.next, struct hw_breakpoint, node); +while (k <= u) { +if (&bp->node == &kernel_bps || tprio[u] >= bp->priority) + --u;/* User bps win a slot */ +else { + ++k;/* Kernel bp wins a slot */ +if (bp->status !=HW_BREAKPOINT_INSTALLED ) + changed =1 ; + bp = list_entry(bp->node.next, struct hw_breakpoint, + node); + } + } +if (k != cur_kbpdata->num_kbps) + changed =1 ; + +/* Notify the remaining kernel breakpoints that they are about +* to be uninstalled. +*/ +list_for_each_entry_from (bp, &kernel_bps, node) { +if (bp->status ==HW_BREAKPOINT_INSTALLED ) { +if (bp->uninstalled) + (bp->uninstalled)(bp); + bp->status =HW_BREAKPOINT_REGISTERED ; + changed =1 ; + } + } + +if (changed) { + cur_kbpindex ^=1 ; + new_kbpdata = &kbpdata[cur_kbpindex]; + new_kbpdata->gennum = cur_kbpdata->gennum +1 ; + new_kbpdata->num_kbps = k; +arch_new_kbpdata (new_kbpdata); + u =0 ; +list_for_each_entry (bp, &kernel_bps, node) { +if (u >= k) +break ; + new_kbpdata->bps[u] = bp; + ++u; + } +rcu_assign_pointer (cur_kbpdata, new_kbpdata); + +/* Tell all the CPUs to update their debug registers */ +update_all_cpus (); + +/* Notify the breakpoints that just got installed */ +for (u =0 ; u < k; ++u) { + bp = new_kbpdata->bps[u]; +if (bp->status !=HW_BREAKPOINT_INSTALLED ) { + bp->status =HW_BREAKPOINT_INSTALLED ; +if (bp->installed) + (bp->installed)(bp); + } + } + } +} + +/* +* Return the pointer to a thread's hw_breakpoint info area, +* and try to allocate one if it doesn't exist. +* +* The caller must hold hw_breakpoint_mutex. +*/ +struct thread_hw_breakpoint *alloc_thread_hw_breakpoint ( +struct task_struct *tsk) +{ +if (!tsk->thread.hw_breakpoint_info && !(tsk->flags &PF_EXITING )) { +struct thread_hw_breakpoint *thbi; + + thbi =kzalloc (sizeof (struct thread_hw_breakpoint ), +GFP_KERNEL ); +if (thbi) { +INIT_LIST_HEAD (&thbi->node); +INIT_LIST_HEAD (&thbi->thread_bps); + +/* Force an update the next time tsk runs */ + thbi->gennum = cur_kbpdata->gennum -2 ; + tsk->thread.hw_breakpoint_info = thbi; + } + } +return tsk->thread.hw_breakpoint_info; +} + +/* +* Erase all the hardware breakpoint info associated with a thread. +* +* If tsk != current then tsk must not be usable (for example, a +* child being cleaned up from a failed fork). +*/ +void flush_thread_hw_breakpoint (struct task_struct *tsk) +{ +struct thread_hw_breakpoint *thbi = tsk->thread.hw_breakpoint_info; +struct hw_breakpoint *bp; + +if (!thbi) +return ; +mutex_lock (&hw_breakpoint_mutex); + +/* Let the breakpoints know they are being uninstalled */ +list_for_each_entry (bp, &thbi->thread_bps, node) { +if (bp->status ==HW_BREAKPOINT_INSTALLED && bp->uninstalled) + (bp->uninstalled)(bp); + bp->status =0 ; + } + +/* Remove tsk from the list of all threads with registered bps */ +list_del (&thbi->node); + +/* The thread no longer has any breakpoints associated with it */ +clear_tsk_thread_flag (tsk,TIF_DEBUG ); + tsk->thread.hw_breakpoint_info =NULL ; +kfree (thbi); + +/* Recalculate and rebalance the kernel-vs-user priorities */ +recalc_tprio (); +balance_kernel_vs_user (); + +/* Actually uninstall the breakpoints if necessary */ +if (tsk == current) +switch_to_none_hw_breakpoint (); +mutex_unlock (&hw_breakpoint_mutex); +} + +/* +* Copy the hardware breakpoint info from a thread to its cloned child. +*/ +int copy_thread_hw_breakpoint (struct task_struct *tsk, +struct task_struct *child,unsigned long clone_flags) +{ +/* We will assume that breakpoint settings are not inherited +* and the child starts out with no debug registers set. +* But what about CLONE_PTRACE? +*/ +clear_tsk_thread_flag (child,TIF_DEBUG ); +return 0 ; +} + +/* +* Store the highest-priority thread breakpoint entries in an array. +*/ +static void store_thread_bp_array (struct thread_hw_breakpoint *thbi) +{ +struct hw_breakpoint *bp; +int i; + + i =HB_NUM -1 ; +list_for_each_entry (bp, &thbi->thread_bps, node) { + thbi->bps[i] = bp; +arch_store_thread_bp_array (thbi, bp, i); +if (--i <0 ) +break ; + } +while (i >=0 ) + thbi->bps[i--] =NULL ; + +/* Force an update the next time this task runs */ + thbi->gennum = cur_kbpdata->gennum -2 ; +} + +/* +* Insert a new breakpoint in a priority-sorted list. +* Return the bp's index in the list. +* +* Thread invariants: +* tsk_thread_flag(tsk, TIF_DEBUG) set implies +* tsk->thread.hw_breakpoint_info is not NULL. +* tsk_thread_flag(tsk, TIF_DEBUG) set iff thbi->thread_bps is non-empty +* iff thbi->node is on thread_list. +*/ +static int insert_bp_in_list (struct hw_breakpoint *bp, +struct thread_hw_breakpoint *thbi,struct task_struct *tsk) +{ +struct list_head *head; +int pos; +struct hw_breakpoint *temp_bp; + +/* tsk and thbi are NULL for kernel bps, non-NULL for user bps */ +if (tsk) + head = &thbi->thread_bps; +else + head = &kernel_bps; + +/* Equal-priority breakpoints get listed first-come-first-served */ + pos =0 ; +list_for_each_entry (temp_bp, head, node) { +if (bp->priority > temp_bp->priority) +break ; + ++pos; + } + bp->status =HW_BREAKPOINT_REGISTERED ; +list_add_tail (&bp->node, &temp_bp->node); + +if (tsk) { +store_thread_bp_array (thbi); + +/* Is this the thread's first registered breakpoint? */ +if (list_empty (&thbi->node)) { +set_tsk_thread_flag (tsk,TIF_DEBUG ); +list_add (&thbi->node, &thread_list); + } + } +return pos; +} + +/* +* Remove a breakpoint from its priority-sorted list. +* +* See the invariants mentioned above. +*/ +static void remove_bp_from_list (struct hw_breakpoint *bp, +struct thread_hw_breakpoint *thbi,struct task_struct *tsk) +{ +/* Remove bp from the thread's/kernel's list. If the list is now +* empty we must clear the TIF_DEBUG flag. But keep the +* thread_hw_breakpoint structure, so that the virtualized debug +* register values will remain valid. +*/ +list_del (&bp->node); +if (tsk) { +store_thread_bp_array (thbi); + +if (list_empty (&thbi->thread_bps)) { +list_del_init (&thbi->node); +clear_tsk_thread_flag (tsk,TIF_DEBUG ); + } + } + +/* Tell the breakpoint it is being uninstalled */ +if (bp->status ==HW_BREAKPOINT_INSTALLED && bp->uninstalled) + (bp->uninstalled)(bp); + bp->status =0 ; +} + +/* +* Validate the settings in a hw_breakpoint structure. +*/ +static int validate_settings (struct hw_breakpoint *bp,struct task_struct *tsk) +{ +int ret; +unsigned int align; + + ret =arch_validate_hwbkpt_settings (bp, &align, tsk); +if (ret <0 ) + goto err; + +/* Check that the low-order bits of the address are appropriate +* for the alignment implied by len. +*/ +if (bp->info.address & align) +return -EINVAL ; + +/* Check that the virtual address is in the proper range */ +if (tsk) { +if (!arch_check_va_in_userspace (bp->info.address, tsk)) +return -EFAULT ; + }else { +if (!arch_check_va_in_kernelspace (bp->info.address)) +return -EFAULT ; + } + err: +return ret; +} + +/* +* Actual implementation of register_user_hw_breakpoint. +*/ +int __register_user_hw_breakpoint (struct task_struct *tsk, +struct hw_breakpoint *bp) +{ +int rc; +struct thread_hw_breakpoint *thbi; +int pos; + + bp->status =0 ; + rc =validate_settings (bp, tsk); +if (rc) +return rc; + + thbi =alloc_thread_hw_breakpoint (tsk); +if (!thbi) +return -ENOMEM ; + +/* Insert bp in the thread's list */ + pos =insert_bp_in_list (bp, thbi, tsk); +arch_register_user_hw_breakpoint (bp, thbi); + +/* Update and rebalance the priorities. We don't need to go through +* the list of all threads; adding a breakpoint can only cause the +* priorities for this thread to increase. +*/ +accum_thread_tprio (thbi); +balance_kernel_vs_user (); + +/* Did bp get allocated to a debug register? We can tell from its +* position in the list. The number of registers allocated to +* kernel breakpoints is num_kbps; all the others are available for +* user breakpoints. If bp's position in the priority-ordered list +* is low enough, it will get a register. +*/ +if (pos <HB_NUM - cur_kbpdata->num_kbps) { + rc =1 ; + +/* Does it need to be installed right now? */ +if (tsk == current) +switch_to_thread_hw_breakpoint (tsk); +/* Otherwise it will get installed the next time tsk runs */ + } + +return rc; +} + +/** +* register_user_hw_breakpoint - register a hardware breakpoint for user space +* @tsk: the task in whose memory space the breakpoint will be set +* @bp: the breakpoint structure to register +* @address: location (virtual address) of the breakpoint +* @len: encoded extent of the breakpoint address (1, 2, 4, or 8 bytes) +* @type: breakpoint type (read-only, write-only, read-write, or execute) +* +* This routine registers a breakpoint to be associated with @tsk's +* memory space and active only while @tsk is running. It does not +* guarantee that the breakpoint will be allocated to a debug register +* immediately; there may be other higher-priority breakpoints registered +* which require the use of all the debug registers. +* +* @tsk will normally be a process being debugged by the current process, +* but it may also be the current process. +* +* @bp->address, @bp->len, @bp->type, @bp->triggered and @bp->priority must be +* set properly before invocation +* +* Returns 1 if @bp is allocated to a debug register, 0 if @bp is +* registered but not allowed to be installed, otherwise a negative error +* code. +*/ +int register_user_hw_breakpoint (struct task_struct *tsk, +struct hw_breakpoint *bp) +{ +int rc; + +mutex_lock (&hw_breakpoint_mutex); + rc =__register_user_hw_breakpoint (tsk, bp); +mutex_unlock (&hw_breakpoint_mutex); +return rc; +} + +/* +* Actual implementation of unregister_user_hw_breakpoint. +*/ +void __unregister_user_hw_breakpoint (struct task_struct *tsk, +struct hw_breakpoint *bp) +{ +struct thread_hw_breakpoint *thbi = tsk->thread.hw_breakpoint_info; + +if (!bp->status) +return ;/* Not registered */ + +/* Remove bp from the thread's list */ +remove_bp_from_list (bp, thbi, tsk); +arch_unregister_user_hw_breakpoint (bp, thbi); + +/* Recalculate and rebalance the kernel-vs-user priorities, +* and actually uninstall bp if necessary. +*/ +recalc_tprio (); +balance_kernel_vs_user (); +if (tsk == current) +switch_to_thread_hw_breakpoint (tsk); +} + +/** +* unregister_user_hw_breakpoint - unregister a hardware breakpoint for user space +* @tsk: the task in whose memory space the breakpoint is registered +* @bp: the breakpoint structure to unregister +* +* Uninstalls and unregisters @bp. +*/ +void unregister_user_hw_breakpoint (struct task_struct *tsk, +struct hw_breakpoint *bp) +{ +mutex_lock (&hw_breakpoint_mutex); +__unregister_user_hw_breakpoint (tsk, bp); +mutex_unlock (&hw_breakpoint_mutex); +} + +/** +* register_kernel_hw_breakpoint - register a hardware breakpoint for kernel space +* @bp: the breakpoint structure to register +* +* This routine registers a breakpoint to be active at all times. It +* does not guarantee that the breakpoint will be allocated to a debug +* register immediately; there may be other higher-priority breakpoints +* registered which require the use of all the debug registers. +* +* @bp.info->name or @bp.info->address, @bp.info->len, @bp.info->type, +* @bp->triggered and @bp->priority must be set properly before invocation +* +* Returns 1 if @bp is allocated to a debug register, 0 if @bp is +* registered but not allowed to be installed, otherwise a negative error +* code. +*/ +int register_kernel_hw_breakpoint (struct hw_breakpoint *bp) +{ +int rc; +int pos; + + bp->status =0 ; + rc =validate_settings (bp,NULL ); +if (rc) +return rc; + +mutex_lock (&hw_breakpoint_mutex); + +/* Insert bp in the kernel's list */ + pos =insert_bp_in_list (bp,NULL ,NULL ); +arch_register_kernel_hw_breakpoint (bp); + +/* Rebalance the priorities. This will install bp if it +* was allocated a debug register. +*/ +balance_kernel_vs_user (); + +/* Did bp get allocated to a debug register? We can tell from its +* position in the list. The number of registers allocated to +* kernel breakpoints is num_kbps; all the others are available for +* user breakpoints. If bp's position in the priority-ordered list +* is low enough, it will get a register. +*/ +if (pos < cur_kbpdata->num_kbps) + rc =1 ; + +mutex_unlock (&hw_breakpoint_mutex); +return rc; +} +EXPORT_SYMBOL_GPL (register_kernel_hw_breakpoint); + +/** +* unregister_kernel_hw_breakpoint - unregister a hardware breakpoint for kernel space +* @bp: the breakpoint structure to unregister +* +* Uninstalls and unregisters @bp. +*/ +void unregister_kernel_hw_breakpoint (struct hw_breakpoint *bp) +{ +if (!bp->status) +return ;/* Not registered */ +mutex_lock (&hw_breakpoint_mutex); + +/* Remove bp from the kernel's list */ +remove_bp_from_list (bp,NULL ,NULL ); +arch_unregister_kernel_hw_breakpoint (bp); + +/* Rebalance the priorities. This will uninstall bp if it +* was allocated a debug register. +*/ +balance_kernel_vs_user (); + +mutex_unlock (&hw_breakpoint_mutex); +} +EXPORT_SYMBOL_GPL (unregister_kernel_hw_breakpoint); + +/* +* Handle debug exception notifications. +*/ +static int __kprobes hw_breakpoint_exceptions_notify ( +struct notifier_block *unused,unsigned long val,void *data) +{ +if (val !=DIE_DEBUG ) +return NOTIFY_DONE ; +return hw_breakpoint_handler (data); +} + +static struct notifier_block hw_breakpoint_exceptions_nb = { + .notifier_call = hw_breakpoint_exceptions_notify, + .priority =0x7fffffff /* we need to be notified first */ +}; + +static int __init init_hw_breakpoint (void ) +{ +load_debug_registers (); +return register_die_notifier (&hw_breakpoint_exceptions_nb); +} + +core_initcall (init_hw_breakpoint);
kernel/Makefile +1 -1
@@ -10,7 +10,7 @@ obj-y = sched.o fork.o exec_domain.o kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o mutex.o \ hrtimer.o rwsem.o nsproxy.o srcu.o semaphore.o \ notifier.o ksysfs.o pm_qos_params.o sched_clock.o cred.o \ - async.o + async.o hw_breakpoint.o ifdef CONFIG_FUNCTION_TRACER # Do not trace debug files and internal ftrace files
include/asm-generic/hw_breakpoint.h new file +243 -0
243 lines not on this page · raw message
PATCH 2/11 x86 architecture implementation of Hardware Breakpoint interfaces
In reply to a message not in the archiveFrom: Alan Stern <[email protected]>
This patch introduces two new files named hw_breakpoint.[ch] inside x86 specific directories. They contain functions which help validate and serve requests for using Hardware Breakpoint registers on x86 processors.
[K.Prasad: More declarations in hw_breakpoint.h to independently compile each
hw_breakpoint.c files. Split-out from the bigger patch and minor changes following re-basing]
--- arch/x86/include/asm/hw_breakpoint.h | 132 ++++++++++ arch/x86/kernel/Makefile | 2 arch/x86/kernel/hw_breakpoint.c | 437 +++++++++++++++++++++++++++++++++++ 3 files changed, 570 insertions(+), 1 deletion(-)
arch/x86/kernel/hw_breakpoint.c new file +437 -0
@@ -0,0 +1,437 @@ +/* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +* +* Copyright (C) 2007 Alan Stern +* Copyright (C) 2009 IBM Corporation +*/ + +/* +* HW_breakpoint: a unified kernel/user-space hardware breakpoint facility, +* using the CPU's debug registers. +*/ + +#include <linux/init.h> +#include <linux/irqflags.h> +#include <linux/kdebug.h> +#include <linux/kernel.h> +#include <linux/kprobes.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/notifier.h> +#include <linux/rculist.h> +#include <linux/sched.h> +#include <linux/smp.h> +#include <linux/percpu.h> +#include <linux/kallsyms.h> + +#include <asm/debugreg.h> +#include <asm/hw_breakpoint.h> +#include <asm/processor.h> + +static unsigned long kdr7;/* Unmasked kernel DR7 value */ + +/* Masks for the bits in DR7 related to kernel breakpoints, for various +* values of num_kbps. Entry n is the mask for when there are n kernel +* breakpoints, in debug registers 0 - (n-1). The DR_GLOBAL_SLOWDOWN bit +* (GE) is handled specially. +*/ +static const unsigned long kdr7_masks[HB_NUM +1 ] = { +0x00000000 , +0x000f0003 ,/* LEN0, R/W0, G0, L0 */ +0x00ff000f ,/* Same for 0,1 */ +0x0fff003f ,/* Same for 0,1,2 */ +0xffff00ff /* Same for 0,1,2,3 */ +}; + +/* +* Install the kernel breakpoints in their debug registers. +*/ +void arch_install_chbi (struct cpu_hw_breakpoint *chbi) +{ +struct hw_breakpoint **bps; + +/* Don't allow debug exceptions while we update the registers */ +set_debugreg (0UL ,7 ); + chbi->cur_kbpdata =rcu_dereference (cur_kbpdata); + +/* Kernel breakpoints are stored starting in DR0 and going up */ + bps = chbi->cur_kbpdata->bps; +switch (chbi->cur_kbpdata->num_kbps) { +case 4 : +set_debugreg (bps[3 ]->info.address,3 ); +case 3 : +set_debugreg (bps[2 ]->info.address,2 ); +case 2 : +set_debugreg (bps[1 ]->info.address,1 ); +case 1 : +set_debugreg (bps[0 ]->info.address,0 ); + } +/* No need to set DR6 */ +set_debugreg (chbi->cur_kbpdata->mkdr7,7 ); +} + +/* +* Update an out-of-date thread hw_breakpoint info structure. +*/ +void arch_update_thbi (struct thread_hw_breakpoint *thbi, +struct kernel_bp_data *thr_kbpdata) +{ +int num = thr_kbpdata->num_kbps; + + thbi->tkdr7 = thr_kbpdata->mkdr7 | (thbi->tdr7 & ~kdr7_masks[num]); +} + +/* +* Install the thread breakpoints in their debug registers. +*/ +void arch_install_thbi (struct thread_hw_breakpoint *thbi) +{ +/* Install the user breakpoints. Kernel breakpoints are stored +* starting in DR0 and going up; there are num_kbps of them. +* User breakpoints are stored starting in DR3 and going down, +* as many as we have room for. +*/ +switch (thbi->num_installed) { +case 4 : +set_debugreg (thbi->tdr[0 ],0 ); +case 3 : +set_debugreg (thbi->tdr[1 ],1 ); +case 2 : +set_debugreg (thbi->tdr[2 ],2 ); +case 1 : +set_debugreg (thbi->tdr[3 ],3 ); + } +/* No need to set DR6 */ +set_debugreg (thbi->tkdr7,7 ); +} + +/* +* Install the debug register values for just the kernel, no thread. +*/ +void arch_install_none (struct cpu_hw_breakpoint *chbi) +{ +set_debugreg (chbi->cur_kbpdata->mkdr7,7 ); +} + +/* +* Create a new kbpdata entry. +*/ +void arch_new_kbpdata (struct kernel_bp_data *new_kbpdata) +{ +int num = new_kbpdata->num_kbps; + + new_kbpdata->mkdr7 = kdr7 & (kdr7_masks[num] |DR_GLOBAL_SLOWDOWN ); +} + +/* +* Store a thread breakpoint array entry's address +*/ +void arch_store_thread_bp_array (struct thread_hw_breakpoint *thbi, +struct hw_breakpoint *bp,int i) +{ + thbi->tdr[i] = bp->info.address; +} + +/* +* Check for virtual address in user space. +*/ +int arch_check_va_in_userspace (unsigned long va,struct task_struct *tsk) +{ +return (va <TASK_SIZE ); +} + +/* +* Check for virtual address in kernel space. +*/ +int arch_check_va_in_kernelspace (unsigned long va) +{ +return (va >=TASK_SIZE ); +} + +/* +* Store a breakpoint's encoded address, length, and type. +*/ +void arch_store_info (struct hw_breakpoint *bp) +{ +/* +* User-space requests will always have the address field populated +* For kernel-addresses, either the address or symbol name can be +* specified. +*/ +if (bp->info.address) +return ; + bp->info.address = (unsigned long )kallsyms_lookup_name (bp->info.name); +} + +/* +* Validate the arch-specific HW Breakpoint register settings +*/ +int arch_validate_hwbkpt_settings (struct hw_breakpoint *bp, +unsigned int *align,struct task_struct *tsk) +{ +int ret = -EINVAL ; + +switch (bp->info.type) { + +/* Ptrace-refactoring code +* For now, we'll allow instruction breakpoint only for user-space +* addresses +*/ +case HW_BREAKPOINT_EXECUTE : +if ((!arch_check_va_in_userspace (bp->info.address, tsk)) && + bp->info.len !=HW_BREAKPOINT_LEN_EXECUTE ) +return ret; +break ; +case HW_BREAKPOINT_WRITE : +break ; +case HW_BREAKPOINT_RW : +break ; +default : +return ret; + } + +switch (bp->info.len) { +case HW_BREAKPOINT_LEN_1 : + *align =0 ; +break ; +case HW_BREAKPOINT_LEN_2 : + *align =1 ; +break ; +case HW_BREAKPOINT_LEN_4 : + *align =3 ; +break ; +default : +return ret; + } + +if (bp->triggered) { + ret =0 ; +arch_store_info (bp); + } +return ret; +} + +/* +* Encode the length, type, Exact, and Enable bits for a particular breakpoint +* as stored in debug register 7. +*/ +static unsigned long encode_dr7 (int drnum,unsigned len,unsigned type) +{ +unsigned long temp; + + temp = (len | type) &0xf ; + temp <<= (DR_CONTROL_SHIFT + drnum *DR_CONTROL_SIZE ); + temp |= (DR_GLOBAL_ENABLE << (drnum *DR_ENABLE_SIZE )) | +DR_GLOBAL_SLOWDOWN ; +return temp; +} + +/* +* Calculate the DR7 value for a list of kernel or user breakpoints. +*/ +static unsigned long calculate_dr7 (struct thread_hw_breakpoint *thbi) +{ +int is_user; +struct list_head *bp_list; +struct hw_breakpoint *bp; +int i; +int drnum; +unsigned long dr7; + +if (thbi) { + is_user =1 ; + bp_list = &thbi->thread_bps; + drnum =HB_NUM -1 ; + }else { + is_user =0 ; + bp_list = &kernel_bps; + drnum =0 ; + } + +/* Kernel bps are assigned from DR0 on up, and user bps are assigned +* from DR3 on down. Accumulate all 4 bps; the kernel DR7 mask will +* select the appropriate bits later. +*/ + dr7 =0 ; + i =0 ; +list_for_each_entry (bp, bp_list, node) { + +/* Get the debug register number and accumulate the bits */ + dr7 |=encode_dr7 (drnum, bp->info.len, bp->info.type); +if (++i >=HB_NUM ) +break ; +if (is_user) + --drnum; +else + ++drnum; + } +return dr7; +} + +/* +* Register a new user breakpoint structure. +*/ +void arch_register_user_hw_breakpoint (struct hw_breakpoint *bp, +struct thread_hw_breakpoint *thbi) +{ + thbi->tdr7 =calculate_dr7 (thbi); + +/* If this is an execution breakpoint for the current PC address, +* we should clear the task's RF so that the bp will be certain +* to trigger. +* +* FIXME: It's not so easy to get hold of the task's PC as a linear +* address! ptrace.c does this already... +*/ +} + +/* +* Unregister a user breakpoint structure. +*/ +void arch_unregister_user_hw_breakpoint (struct hw_breakpoint *bp, +struct thread_hw_breakpoint *thbi) +{ + thbi->tdr7 =calculate_dr7 (thbi); +} + +/* +* Register a kernel breakpoint structure. +*/ +void arch_register_kernel_hw_breakpoint (struct hw_breakpoint *bp) +{ + kdr7 =calculate_dr7 (NULL ); +} + +/* +* Unregister a kernel breakpoint structure. +*/ +void arch_unregister_kernel_hw_breakpoint (struct hw_breakpoint *bp) +{ + kdr7 =calculate_dr7 (NULL ); +} + + +/* End of arch-specific hook routines */ + + +/* +* Copy out the debug register information for a core dump. +* +* tsk must be equal to current. +*/ +void dump_thread_hw_breakpoint (struct task_struct *tsk,int u_debugreg[8 ]) +{ +struct thread_hw_breakpoint *thbi = tsk->thread.hw_breakpoint_info; +int i; + +memset (u_debugreg,0 ,sizeof u_debugreg); +if (thbi) { +for (i =0 ; i <HB_NUM ; ++i) + u_debugreg[i] = thbi->vdr_bps[i].info.address; + u_debugreg[7 ] = thbi->vdr7; + } + u_debugreg[6 ] = tsk->thread.vdr6; +} + +/* +* Handle debug exception notifications. +*/ + +int __kprobes hw_breakpoint_handler (struct die_args *args) +{ +struct cpu_hw_breakpoint *chbi; +int i; +struct hw_breakpoint *bp; +struct thread_hw_breakpoint *thbi =NULL ; + +/* The DR6 value is stored in args->err */ +#define DR6 (args->err) + +if (DR6 &DR_STEP ) +return NOTIFY_DONE ; + + chbi = &per_cpu (cpu_bp,get_cpu ()); + +/* Disable all breakpoints so that the callbacks can run without +* triggering recursive debug exceptions. +*/ +set_debugreg (0UL ,7 ); + +/* Assert that local interrupts are disabled +* Reset the DRn bits in the virtualized register value. +* The ptrace trigger routine will add in whatever is needed. +*/ + current->thread.vdr6 &= ~(DR_TRAP0 |DR_TRAP1 |DR_TRAP2 |DR_TRAP3 ); + +/* Are we a victim of lazy debug-register switching? */ +if (!chbi->bp_task) + ; +else if (chbi->bp_task != current) { + +/* No user breakpoints are valid. Perform the belated +* debug-register switch. +*/ +switch_to_none_hw_breakpoint (); + }else { + thbi = chbi->bp_task->thread.hw_breakpoint_info; + } + +/* Handle all the breakpoints that were triggered */ +for (i =0 ; i <HB_NUM ; ++i) { +if (likely (!(DR6 & (DR_TRAP0 << i)))) +continue ; + +/* Find the corresponding hw_breakpoint structure and +* invoke its triggered callback. +*/ +if (i < chbi->cur_kbpdata->num_kbps) + bp = chbi->cur_kbpdata->bps[i]; +else if (thbi) + bp = thbi->bps[i]; +else /* False alarm due to lazy DR switching */ +continue ; +if (bp) { +switch (bp->info.type) { +case HW_BREAKPOINT_WRITE : +case HW_BREAKPOINT_RW : +if (bp->triggered) + (bp->triggered)(bp, args->regs); +/* Re-enable the breakpoints */ +set_debugreg (thbi ? thbi->tkdr7 : + chbi->cur_kbpdata->mkdr7,7 ); +put_cpu_no_resched (); + +return NOTIFY_STOP ; +/* +* Presently we allow instruction breakpoints only in +* user-space when requested through ptrace. +*/ +case HW_BREAKPOINT_EXECUTE : +if (arch_check_va_in_userspace (bp->info.address, + current)) { + (bp->triggered)(bp, args->regs); +/* We'll return NOTIFY_DONE, do_debug will take care of the rest */ +return NOTIFY_DONE ; + } + } + } + } +/* Stop processing further if the exception is a stray one */ +if (!(DR6 & ~(DR_TRAP0 |DR_TRAP1 |DR_TRAP2 |DR_TRAP3 ))) +return NOTIFY_STOP ; + +return NOTIFY_DONE ; +#undef DR6 +}
arch/x86/include/asm/hw_breakpoint.h new file +132 -0
132 lines not on this page · raw message
arch/x86/kernel/Makefile +1 -1
@@ -36,7 +36,7 @@ obj-$(CONFIG_X86_64) += sys_x86_64.o x86 obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o obj-y += bootflag.o e820.o obj-y += pci-dma.o quirks.o i8237.o topology.o kdebugfs.o -obj-y += alternative.o i8253.o pci-nommu.o +obj-y += alternative.o i8253.o pci-nommu.o hw_breakpoint.o obj-y += tsc.o io_delay.o rtc.o obj-$(CONFIG_X86_TRAMPOLINE) += trampoline.o
PATCH 3/11 Modifying generic debug exception to use virtual debug registers
In reply to a message not in the archiveFrom: Alan Stern <[email protected]>
This patch modifies the breakpoint exception handler code to use the abstract register names.
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/kernel/traps.c | 73 ++++++++++++++++-------------------------------- 1 file changed, 25 insertions(+), 48 deletions(-)
arch/x86/kernel/traps.c +25 -48
@@ -530,13 +530,14 @@ asmlinkage __kprobes struct pt_regs *syn dotraplinkage void __kprobes do_debug(struct pt_regs *regs, long error_code) {struct task_struct *tsk = current; -unsigned long condition; +unsigned long dr6;int si_code; -get_debugreg (condition,6 ); +get_debugreg (dr6,6 ); +set_debugreg (0 ,6 );/* DR6 may or may not be cleared by the CPU */ /* Catch kmemcheck conditions first of all! */ -if (condition &DR_STEP &&kmemcheck_trap (regs)) +if (dr6 &DR_STEP &&kmemcheck_trap (regs))return ;/*
@@ -545,61 +546,37 @@ dotraplinkage void __kprobes do_debug(stclear_tsk_thread_flag (tsk,TIF_DEBUGCTLMSR ); tsk->thread.debugctlmsr =0 ; -if (notify_die (DIE_DEBUG ,"debug" , regs, condition, error_code, +/* Store the virtualized DR6 value */ + tsk->thread.vdr6 = dr6; + +if (notify_die (DIE_DEBUG ,"debug" , regs, dr6, error_code,SIGTRAP ) ==NOTIFY_STOP )return ;/* It's safe to allow irq's after DR6 has been saved */ preempt_conditional_sti (regs); -/* Mask out spurious debug traps due to lazy DR7 setting */ -if (condition & (DR_TRAP0 |DR_TRAP1 |DR_TRAP2 |DR_TRAP3 )) { -if (!tsk->thread.debugreg7) - goto clear_dr7; - } - -#ifdef CONFIG_X86_32 -if (regs->flags &X86_VM_MASK ) - goto debug_vm86; -#endif - -/* Save debug status register where ptrace can see it */ - tsk->thread.debugreg6 = condition; - -/* -* Single-stepping through TF: make sure we ignore any events in -* kernel space (but re-enable TF when returning to user mode). -*/ -if (condition &DR_STEP ) { -if (!user_mode (regs)) - goto clear_TF_reenable; +if (regs->flags &X86_VM_MASK ) { +handle_vm86_trap ((struct kernel_vm86_regs *) regs, + error_code,1 ); +return ; } - si_code =get_si_code (condition); -/* Ok, finally something we can handle */ -send_sigtrap (tsk, regs, error_code, si_code); -/* -* Disable additional traps. They'll be re-enabled when -* the signal is delivered. +* Single-stepping through system calls: ignore any exceptions in +* kernel space, but re-enable TF when returning to user mode. +* +* We already checked v86 mode above, so we can check for kernel mode +* by just checking the CPL of CS. */ -clear_dr7: -set_debugreg (0 ,7 ); -preempt_conditional_cli (regs); -return ; - -#ifdef CONFIG_X86_32 -debug_vm86: -/* reenable preemption: handle_vm86_trap() might sleep */ -dec_preempt_count (); -handle_vm86_trap ((struct kernel_vm86_regs *) regs, error_code,1 ); -conditional_cli (regs); -return ; -#endif - -clear_TF_reenable: -set_tsk_thread_flag (tsk,TIF_SINGLESTEP ); - regs->flags &= ~X86_EFLAGS_TF ; +if ((dr6 &DR_STEP ) && !user_mode (regs)) { + tsk->thread.vdr6 &= ~DR_STEP ; +set_tsk_thread_flag (tsk,TIF_SINGLESTEP ); + regs->flags &= ~X86_EFLAGS_TF ; + } + si_code =get_si_code (dr6); +if (tsk->thread.vdr6 & (DR_STEP |DR_TRAP0 |DR_TRAP1 |DR_TRAP2 |DR_TRAP3 )) +send_sigtrap (tsk, regs, error_code, si_code);preempt_conditional_cli (regs);return ; }
PATCH 4/11 Introduce virtual debug register in thread_struct and wrapper-routines around process related functions
In reply to a message not in the archiveFrom: Alan Stern <[email protected]>
This patch introduces virtual debug registers to used by the per-thread structure ad wrapper routines to manage debug registers by process-related functions.
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/include/asm/debugreg.h | 27 +++++++++++++++++++++++++++ arch/x86/include/asm/processor.h | 10 +++------- 2 files changed, 30 insertions(+), 7 deletions(-)
arch/x86/include/asm/debugreg.h +27 -0
@@ -49,6 +49,8 @@#define DR_LOCAL_ENABLE_SHIFT 0/* Extra shift to the local enable bit */ #define DR_GLOBAL_ENABLE_SHIFT 1/* Extra shift to the global enable bit */ +#define DR_LOCAL_ENABLE (0x1)/* Local enable for reg 0 */ +#define DR_GLOBAL_ENABLE (0x2)/* Global enable for reg 0 */ #define DR_ENABLE_SIZE 2/* 2 enable bits per register */ #define DR_LOCAL_ENABLE_MASK (0x55)/* Set local bits for all 4 regs */
@@ -67,4 +69,29 @@#define DR_LOCAL_SLOWDOWN (0x100)/* Local slow the pipeline */ #define DR_GLOBAL_SLOWDOWN (0x200)/* Global slow the pipeline */ +/* +* HW breakpoint additions +*/ +#ifdef __KERNEL__ + +/* For process management */ +void flush_thread_hw_breakpoint (struct task_struct *tsk); +int copy_thread_hw_breakpoint (struct task_struct *tsk, +struct task_struct *child,unsigned long clone_flags); +void dump_thread_hw_breakpoint (struct task_struct *tsk,int u_debugreg[8 ]); +void switch_to_thread_hw_breakpoint (struct task_struct *tsk); + +/* For CPU management */ +void load_debug_registers (void ); +static inline void disable_debug_registers (void ) +{ +set_debugreg (0UL ,7 ); +} + +/* For use by ptrace */ +unsigned long thread_get_debugreg (struct task_struct *tsk,int n); +int thread_set_debugreg (struct task_struct *tsk,int n,unsigned long val); + +#endif /* __KERNEL__ */ +#endif /* _ASM_X86_DEBUGREG_H */
arch/x86/include/asm/processor.h +3 -7
@@ -421,13 +421,9 @@ struct thread_struct {unsigned long ip;unsigned long fs;unsigned long gs; -/* Hardware debugging registers: */ -unsigned long debugreg0; -unsigned long debugreg1; -unsigned long debugreg2; -unsigned long debugreg3; -unsigned long debugreg6; -unsigned long debugreg7; +/* Hardware breakpoint info */ +unsigned long vdr6; +struct thread_hw_breakpoint *hw_breakpoint_info;/* Fault info: */ unsigned long cr2;unsigned long trap_no;
PATCH 5/11 Use wrapper routines around debug registers in processor related functions
In reply to a message not in the archiveFrom: Alan Stern <[email protected]>
This patch enables the use of wrapper routines to access the debug/breakpoint registers.
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/kernel/smpboot.c | 3 +++ arch/x86/power/cpu_32.c | 16 +++------------- arch/x86/power/cpu_64.c | 15 +++------------ 3 files changed, 9 insertions(+), 25 deletions(-)
arch/x86/power/cpu_32.c +3 -13
@@ -12,6 +12,7 @@#include <asm/mtrr.h> #include <asm/mce.h> #include <asm/xcr.h> +#include <asm/debugreg.h> static struct saved_context saved_context;
@@ -47,6 +48,7 @@ static void __save_processor_state(struc ctxt->cr2 =read_cr2 (); ctxt->cr3 =read_cr3 (); ctxt->cr4 =read_cr4_safe (); +disable_debug_registers (); }/* Needed by apm.c */
@@ -79,19 +81,7 @@ static void fix_processor_context(void)load_TR_desc ();/* This does ltr */ load_LDT (¤t->active_mm->context);/* This does lldt */ -/* -* Now maybe reload the debug registers -*/ -if (current->thread.debugreg7) { -set_debugreg (current->thread.debugreg0,0 ); -set_debugreg (current->thread.debugreg1,1 ); -set_debugreg (current->thread.debugreg2,2 ); -set_debugreg (current->thread.debugreg3,3 ); -/* no 4 and 5 */ -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } - +load_debug_registers (); }static void __restore_processor_state (struct saved_context *ctxt)
arch/x86/power/cpu_64.c +3 -12
@@ -15,6 +15,7 @@#include <asm/pgtable.h> #include <asm/mtrr.h> #include <asm/xcr.h> +#include <asm/debugreg.h> static void fix_processor_context (void );
@@ -70,6 +71,7 @@ static void __save_processor_state(struc ctxt->cr3 =read_cr3 (); ctxt->cr4 =read_cr4 (); ctxt->cr8 =read_cr8 (); +disable_debug_registers (); }void save_processor_state (void )
@@ -158,16 +160,5 @@ static void fix_processor_context(void)load_TR_desc ();/* This does ltr */ load_LDT (¤t->active_mm->context);/* This does lldt */ -/* -* Now maybe reload the debug registers -*/ -if (current->thread.debugreg7){ -loaddebug (¤t->thread,0 ); -loaddebug (¤t->thread,1 ); -loaddebug (¤t->thread,2 ); -loaddebug (¤t->thread,3 ); -/* no 4 and 5 */ -loaddebug (¤t->thread,6 ); -loaddebug (¤t->thread,7 ); - } +load_debug_registers (); }
arch/x86/kernel/smpboot.c +3 -0
@@ -63,6 +63,7 @@#include <asm/apic.h> #include <asm/setup.h> #include <asm/uv/uv.h> +#include <asm/debugreg.h> #include <linux/mc146818rtc.h> #include <asm/smpboot_hooks.h>
@@ -337,6 +338,7 @@ notrace static void __cpuinit start_secosetup_secondary_clock ();wmb (); +load_debug_registers ();cpu_idle (); }
@@ -1312,6 +1314,7 @@ void cpu_disable_common(void)remove_cpu_from_maps (cpu);unlock_vector_lock ();fixup_irqs (); +disable_debug_registers (); }int native_cpu_disable (void )
PATCH 6/11 Use virtual debug registers in process/thread handling code
In reply to a message not in the archiveFrom: Alan Stern <[email protected]>
This patch enables the use of abstract/virtual debug registers in process-handling routines.
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/kernel/process.c | 21 ++++----------------- arch/x86/kernel/process_32.c | 19 +++++++++++++++++++ arch/x86/kernel/process_64.c | 14 ++++++++++++++ 3 files changed, 37 insertions(+), 17 deletions(-)
arch/x86/kernel/process.c +4 -17
@@ -82,6 +82,8 @@ void exit_thread(void) t->io_bitmap_max =0 ;put_cpu (); } +if (unlikely (t.hw_breakpoint_info)) +flush_thread_hw_breakpoint (me);ds_exit_thread (current); }
@@ -102,14 +104,9 @@ void flush_thread(void) }#endif -clear_tsk_thread_flag (tsk,TIF_DEBUG ); +if (unlikely (tsk->thread.hw_breakpoint_info)) +flush_thread_hw_breakpoint (tsk); - tsk->thread.debugreg0 =0 ; - tsk->thread.debugreg1 =0 ; - tsk->thread.debugreg2 =0 ; - tsk->thread.debugreg3 =0 ; - tsk->thread.debugreg6 =0 ; - tsk->thread.debugreg7 =0 ;memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array));/* * Forget coprocessor state..
@@ -191,16 +188,6 @@ void __switch_to_xtra(struct task_structelse if (next->debugctlmsr != prev->debugctlmsr)update_debugctlmsr (next->debugctlmsr); -if (test_tsk_thread_flag (next_p,TIF_DEBUG )) { -set_debugreg (next->debugreg0,0 ); -set_debugreg (next->debugreg1,1 ); -set_debugreg (next->debugreg2,2 ); -set_debugreg (next->debugreg3,3 ); -/* no 4 and 5 */ -set_debugreg (next->debugreg6,6 ); -set_debugreg (next->debugreg7,7 ); - } -if (test_tsk_thread_flag (prev_p,TIF_NOTSC ) ^test_tsk_thread_flag (next_p,TIF_NOTSC )) {/* prev and next are different */
arch/x86/kernel/process_32.c +19 -0
@@ -59,6 +59,8 @@#include <asm/idle.h> #include <asm/syscalls.h> #include <asm/ds.h> +#include <asm/debugreg.h> +#include <asm/hw_breakpoint.h> asmlinkage void ret_from_fork(void) __asm__("ret_from_fork");
@@ -263,7 +265,15 @@ int copy_thread(int nr, unsigned long cltask_user_gs (p) =get_user_gs (regs); + p->thread.hw_breakpoint_info =NULL ; + p->thread.io_bitmap_ptr =NULL ; + tsk = current; + err = -ENOMEM ; +if (unlikely (tsk->thread.hw_breakpoint_info)) { +if (copy_thread_hw_breakpoint (tsk, p, clone_flags)) + goto out; + }if (unlikely (test_tsk_thread_flag (tsk,TIF_IO_BITMAP ))) { p->thread.io_bitmap_ptr =kmemdup (tsk->thread.io_bitmap_ptr,IO_BITMAP_BYTES ,GFP_KERNEL );
@@ -283,10 +293,13 @@ int copy_thread(int nr, unsigned long cl err =do_set_thread_area (p,-1 , (struct user_desc __user *)childregs->si,0 ); +out:if (err && p->thread.io_bitmap_ptr) {kfree (p->thread.io_bitmap_ptr); p->thread.io_bitmap_max =0 ; } +if (err) +flush_thread_hw_breakpoint (p);ds_copy_thread (p, current);
@@ -424,6 +437,12 @@ __switch_to(struct task_struct *prev_p,lazy_load_gs (next->gs);percpu_write (current_task, next_p); +/* +* Handle debug registers. This must be done _after_ current +* is updated. +*/ +if (unlikely (test_tsk_thread_flag (next_p,TIF_DEBUG ))) +switch_to_thread_hw_breakpoint (next_p);return prev_p; }
arch/x86/kernel/process_64.c +14 -0
@@ -55,6 +55,8 @@#include <asm/idle.h> #include <asm/syscalls.h> #include <asm/ds.h> +#include <asm/debugreg.h> +#include <asm/hw_breakpoint.h> asmlinkage extern void ret_from_fork(void);
@@ -248,6 +250,8 @@ void release_thread(struct task_struct *BUG (); } } +if (unlikely (me->thread.hw_breakpoint_info)) +flush_thread_hw_breakpoint (me); }static inline void set_32bit_tls (struct task_struct *t,int tls,u32 addr)
@@ -303,12 +307,19 @@ int copy_thread(int nr, unsigned long cl p->thread.fs = me->thread.fs; p->thread.gs = me->thread.gs; + p->thread.hw_breakpoint_info =NULL ; + p->thread.io_bitmap_ptr =NULL ;savesegment (gs, p->thread.gsindex);savesegment (fs, p->thread.fsindex);savesegment (es, p->thread.es);savesegment (ds, p->thread.ds); + err = -ENOMEM ; +if (unlikely (me->thread.hw_breakpoint_info)) { +if (copy_thread_hw_breakpoint (me, p, clone_flags)) + goto out; + }if (unlikely (test_tsk_thread_flag (me,TIF_IO_BITMAP ))) { p->thread.io_bitmap_ptr =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL );if (!p->thread.io_bitmap_ptr) {
@@ -346,6 +357,9 @@ out:kfree (p->thread.io_bitmap_ptr); p->thread.io_bitmap_max =0 ; } +if (err) +flush_thread_hw_breakpoint (p); +return err; }
PATCH 7/11 Modify signal handling code to refrain from re-enabling HW Breakpoints
In reply to a message not in the archiveFrom: Alan Stern <[email protected]>
This patch disables re-enabling of Hardware Breakpoint registers through the signal handling code. This is now done during hw_breakpoint_handler().
--- arch/x86/kernel/signal.c | 9 --------- 1 file changed, 9 deletions(-)
arch/x86/kernel/signal.c +0 -9
@@ -794,15 +794,6 @@ static void do_signal(struct pt_regs *re signr =get_signal_to_deliver (&info, &ka, regs,NULL );if (signr >0 ) { -/* -* Re-enable any watchpoints before delivering the -* signal to user space. The processor register will -* have been cleared if the watchpoint triggered -* inside the kernel. -*/ -if (current->thread.debugreg7) -set_debugreg (current->thread.debugreg7,7 ); -/* Whee! Actually deliver the signal. */ if (handle_signal (signr, &info, &ka, oldset, regs) ==0 ) {/*
PATCH 8/11 Modify Ptrace routines to access breakpoint registers
In reply to a message not in the archiveFrom: Alan Stern <[email protected]>
This patch modifies the ptrace code to use the new wrapper routines around the debug/breakpoint registers.
[K.Prasad: Adapted the ptrace routines and to changes post x86/x86_64 merger]
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/kernel/ptrace.c | 242 +++++++++++++++++++++++++++++------------------ 1 file changed, 152 insertions(+), 90 deletions(-)
arch/x86/kernel/ptrace.c +152 -90
@@ -33,6 +33,7 @@#include <asm/prctl.h> #include <asm/proto.h> #include <asm/ds.h> +#include <asm/hw_breakpoint.h> #include "tls.h"
@@ -133,11 +134,6 @@ static int set_segment_reg(struct task_s return 0; } -static unsigned long debugreg_addr_limit (struct task_struct *task) -{ -return TASK_SIZE -3 ; -} -#else /* CONFIG_X86_64 */ #define FLAG_MASK (FLAG_MASK_32 | X86_EFLAGS_NT)
@@ -262,15 +258,6 @@ static int set_segment_reg(struct task_s return 0; } -static unsigned long debugreg_addr_limit (struct task_struct *task) -{ -#ifdef CONFIG_IA32_EMULATION -if (test_tsk_thread_flag (task,TIF_IA32 )) -return IA32_PAGE_OFFSET -3 ; -#endif -return TASK_SIZE_MAX -7 ; -} -#endif /* CONFIG_X86_32 */ static unsigned long get_flags (struct task_struct *task)
@@ -461,95 +448,170 @@ static int genregs_set(struct task_struc }/* -* This function is trivial and will be inlined by the compiler. -* Having it separates the implementation details of debug -* registers from the interface details of ptrace. +* Decode the length and type bits for a particular breakpoint as +* stored in debug register 7. Return the "enabled" status. */ -static unsigned long ptrace_get_debugreg (struct task_struct *child,int n) +static int decode_dr7 (unsigned long dr7,int bpnum,unsigned *len, +unsigned *type) { -switch (n) { -case 0 :return child->thread.debugreg0; -case 1 :return child->thread.debugreg1; -case 2 :return child->thread.debugreg2; -case 3 :return child->thread.debugreg3; -case 6 :return child->thread.debugreg6; -case 7 :return child->thread.debugreg7; +int temp = dr7 >> (DR_CONTROL_SHIFT + bpnum *DR_CONTROL_SIZE ); + + *len = (temp &0xc ) |0x40 ; + *type = (temp &0x3 ) |0x80 ; +return (dr7 >> (bpnum *DR_ENABLE_SIZE )) &0x3 ; +} + +static void ptrace_triggered (struct hw_breakpoint *bp,struct pt_regs *regs) +{ +struct task_struct *tsk = current; +struct thread_hw_breakpoint *thbi = tsk->thread.hw_breakpoint_info; +int i; + +/* Store in the virtual DR6 register the fact that the breakpoint +* was hit so the thread's debugger will see it. +*/ +if (thbi) { + i = bp - thbi->vdr_bps; + tsk->thread.vdr6 |= (DR_TRAP0 << i); } -return 0 ; } -static int ptrace_set_debugreg (struct task_struct *child, -int n,unsigned long data) +/* +* Handle ptrace writes to debug register 7. +*/ +static int ptrace_write_dr7 (struct task_struct *tsk, +struct thread_hw_breakpoint *thbi,unsigned long data) { +struct hw_breakpoint *bp;int i; +int rc =0 ; +unsigned long old_dr7 = thbi->vdr7; -if (unlikely (n ==4 || n ==5 )) -return -EIO ; + data &= ~DR_CONTROL_RESERVED ; -if (n <4 &&unlikely (data >=debugreg_addr_limit (child))) -return -EIO ; +/* Loop through all the hardware breakpoints, making the +* appropriate changes to each. +*/ + restore_settings: + thbi->vdr7 = data; + bp = &thbi->vdr_bps[0 ]; +for (i =0 ; i <HB_NUM ; (++i, ++bp)) { +int enabled; +unsigned len, type; + + enabled =decode_dr7 (data, i, &len, &type); + +/* Unregister the breakpoint before trying to change it */ +if (bp->status) +__unregister_user_hw_breakpoint (tsk, bp); -switch (n) { -case 0 : child->thread.debugreg0 = data;break ; -case 1 : child->thread.debugreg1 = data;break ; -case 2 : child->thread.debugreg2 = data;break ; -case 3 : child->thread.debugreg3 = data;break ; +/* Now register the breakpoint if it should be enabled. +* New invalid entries will raise an error here. +*/ +if (enabled) { + bp->triggered = ptrace_triggered; + bp->info.len = len; + bp->info.type = type; + + bp->priority =HW_BREAKPOINT_PRIO_PTRACE ; +if (rc ==0 &&__register_user_hw_breakpoint (tsk, + bp) <0 ) +break ; + } + } -case 6 : -if ((data & ~0xffffffffUL ) !=0 ) -return -EIO ; - child->thread.debugreg6 = data; -break ; +/* If anything above failed, restore the original settings */ +if (i <HB_NUM ) { + rc = -EIO ; + data = old_dr7; + goto restore_settings; + } +return rc; +} -case 7 : -/* -* Sanity-check data. Take one half-byte at once with -* check = (val >> (16 + 4*i)) & 0xf. It contains the -* R/Wi and LENi bits; bits 0 and 1 are R/Wi, and bits -* 2 and 3 are LENi. Given a list of invalid values, -* we do mask |= 1 << invalid_value, so that -* (mask >> check) & 1 is a correct test for invalid -* values. -* -* R/Wi contains the type of the breakpoint / -* watchpoint, LENi contains the length of the watched -* data in the watchpoint case. -* -* The invalid values are: -* - LENi == 0x10 (undefined), so mask |= 0x0f00. [32-bit] -* - R/Wi == 0x10 (break on I/O reads or writes), so -* mask |= 0x4444. -* - R/Wi == 0x00 && LENi != 0x00, so we have mask |= -* 0x1110. -* -* Finally, mask = 0x0f00 | 0x4444 | 0x1110 == 0x5f54. -* -* See the Intel Manual "System Programming Guide", -* 15.2.4 -* -* Note that LENi == 0x10 is defined on x86_64 in long -* mode (i.e. even for 32-bit userspace software, but -* 64-bit kernel), so the x86_64 mask value is 0x5454. -* See the AMD manual no. 24593 (AMD64 System Programming) +/* +* Handle PTRACE_PEEKUSR calls for the debug register area. +*/ +unsigned long ptrace_get_debugreg (struct task_struct *tsk,int n) +{ +struct thread_hw_breakpoint *thbi; +unsigned long val =0 ; + +mutex_lock (&hw_breakpoint_mutex); + thbi = tsk->thread.hw_breakpoint_info; +if (n <HB_NUM ) { +if (thbi) + val = thbi->vdr_bps[n].info.address; + }else if (n ==6 ) { + val = tsk->thread.vdr6; + }else if (n ==7 ) { +if (thbi) + val = thbi->vdr7; + } +mutex_unlock (&hw_breakpoint_mutex); +return val; +} + +/* +* Handle PTRACE_POKEUSR calls for the debug register area. +*/ +int ptrace_set_debugreg (struct task_struct *tsk,int n,unsigned long val) +{ +struct thread_hw_breakpoint *thbi; +int rc = -EIO ; + +/* We have to hold this lock the entire time, to prevent thbi +* from being deallocated out from under us. +*/ +mutex_lock (&hw_breakpoint_mutex); + +/* There are no DR4 or DR5 registers */ +if (n ==4 || n ==5 ) + ; + +/* Writes to DR6 modify the virtualized value */ +else if (n ==6 ) { + tsk->thread.vdr6 = val; + rc =0 ; + } + +else if (!tsk->thread.hw_breakpoint_info && val ==0 ) + rc =0 ;/* Minor optimization */ + +else if ((thbi =alloc_thread_hw_breakpoint (tsk)) ==NULL ) + rc = -ENOMEM ; + +/* Writes to DR0 - DR3 change a breakpoint address */ +else if (n <HB_NUM ) { +struct hw_breakpoint *bp = &thbi->vdr_bps[n]; + +/* If the breakpoint is registered then unregister it, +* change it, and re-register it. Revert to the original +* address if an error occurs. */ -#ifdef CONFIG_X86_32 -#define DR7_MASK 0x5f54 -#else -#define DR7_MASK 0x5554 -#endif - data &= ~DR_CONTROL_RESERVED ; -for (i =0 ; i <4 ; i++) -if ((DR7_MASK >> ((data >> (16 +4 *i)) &0xf )) &1 ) -return -EIO ; - child->thread.debugreg7 = data; -if (data) -set_tsk_thread_flag (child,TIF_DEBUG ); -else -clear_tsk_thread_flag (child,TIF_DEBUG ); -break ; +if (bp->status) { +unsigned long old_addr = bp->info.address; + +__unregister_user_hw_breakpoint (tsk, bp); + + bp->info.address = val; + rc =__register_user_hw_breakpoint (tsk, bp); +if (rc <0 ) { + bp->info.address = old_addr; +__register_user_hw_breakpoint (tsk, bp); + } + }else { + bp->info.address = val; + rc =0 ; + } } -return 0 ; +/* All that's left is DR7 */ +else + rc =ptrace_write_dr7 (tsk, thbi, val); + +mutex_unlock (&hw_breakpoint_mutex); +return rc; }/*
@@ -871,7 +933,7 @@ long arch_ptrace(struct task_struct *chi else if (addr >= offsetof(struct user, u_debugreg[0]) && addr <= offsetof(struct user, u_debugreg[7])) { addr -= offsetof(struct user, u_debugreg[0]); - tmp =ptrace_get_debugreg (child, addr /sizeof (data)); + tmp =ptrace_get_debugreg (child, addr/sizeof (data)); } ret =put_user (tmp, datap);break ;
@@ -889,7 +951,7 @@ long arch_ptrace(struct task_struct *chi addr <= offsetof(struct user, u_debugreg[7])) { addr -= offsetof(struct user, u_debugreg[0]); ret =ptrace_set_debugreg (child, - addr /sizeof (data), data); + addr/sizeof (data), data); }break ;
PATCH 9/11 Cleanup HW Breakpoint registers before kexec
In reply to a message not in the archiveFrom: Alan Stern <[email protected]>
This patch disables Hardware breakpoints before doing a 'kexec' on the machine.
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/kernel/machine_kexec_32.c | 2 ++ arch/x86/kernel/machine_kexec_64.c | 2 ++ 2 files changed, 4 insertions(+)
arch/x86/kernel/machine_kexec_32.c +2 -0
@@ -25,6 +25,7 @@#include <asm/desc.h> #include <asm/system.h> #include <asm/cacheflush.h> +#include <asm/debugreg.h> static void set_idt (void *newidt,__u16 limit) {
@@ -202,6 +203,7 @@ void machine_kexec(struct kimage *image)/* Interrupts aren't acceptable while we reboot */ local_irq_disable (); +disable_debug_registers ();if (image->preserve_context) {#ifdef CONFIG_X86_IO_APIC
arch/x86/kernel/machine_kexec_64.c +2 -0
@@ -17,6 +17,7 @@#include <asm/tlbflush.h> #include <asm/mmu_context.h> #include <asm/io.h> +#include <asm/debugreg.h> static void init_level2_page (pmd_t *level2p,unsigned long addr) {
@@ -234,6 +235,7 @@ void machine_kexec(struct kimage *image)/* Interrupts aren't acceptable while we reboot */ local_irq_disable (); +disable_debug_registers (); control_page =page_address (image->control_code_page) +PAGE_SIZE ;memcpy (control_page, relocate_kernel,PAGE_SIZE );
PATCH 10/11 Sample HW breakpoint over kernel data address
In reply to a message not in the archiveThis patch introduces a sample kernel module to demonstrate the use of Hardware Breakpoint feature. It places a breakpoint over the kernel variable 'pid_max' to monitor all write operations and emits a function-backtrace when done.
--- samples/Kconfig | 6 ++ samples/Makefile | 3 - samples/hw_breakpoint/Makefile | 1 samples/hw_breakpoint/data_breakpoint.c | 80 ++++++++++++++++++++++++++++++++ 4 files changed, 89 insertions(+), 1 deletion(-)
samples/Kconfig +6 -0
@@ -39,5 +39,11 @@ config SAMPLE_KRETPROBES default m depends on SAMPLE_KPROBES && KRETPROBES +config SAMPLE_HW_BREAKPOINT + tristate "Build kernel hardware breakpoint examples -- loadable modules only" + depends on m + help + This builds kernel hardware breakpoint example modules. + endif # SAMPLES
samples/Makefile +2 -1
@@ -1,3 +1,4 @@ # Makefile for Linux samples code -obj-$(CONFIG_SAMPLES) += markers/ kobject/ kprobes/ tracepoints/ +obj-$(CONFIG_SAMPLES) += markers/ kobject/ kprobes/ tracepoints/ \ + hw_breakpoint/
samples/hw_breakpoint/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-$(CONFIG_SAMPLE_HW_BREAKPOINT) += data_breakpoint.o
samples/hw_breakpoint/data_breakpoint.c new file +80 -0
80 lines not on this page · raw message
PATCH 11/11 ftrace plugin for kernel symbol tracing using HW Breakpoint interfaces - v2
In reply to a message not in the archiveThis patch adds an ftrace plugin to detect and profile memory access over kernel variables. It uses HW Breakpoint interfaces to 'watch memory addresses.
--- kernel/trace/Kconfig | 6 kernel/trace/Makefile | 1 kernel/trace/trace.h | 16 + kernel/trace/trace_ksym.c | 448 ++++++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_selftest.c | 36 +++ 5 files changed, 507 insertions(+)
kernel/trace/Kconfig +6 -0
@@ -249,6 +249,12 @@ config POWER_TRACER power management decisions, specifically the C-state and P-state behavior. +config KSYM_TRACER + bool "Trace read and write access on kernel memory locations" + select TRACING + help + This tracer helps find read and write operations on any given kernel + symbol i.e. /proc/kallsyms. config STACK_TRACER bool "Trace max stack"
kernel/trace/Makefile +1 -0
@@ -41,5 +41,6 @@ obj-$(CONFIG_WORKQUEUE_TRACER) += trace_ obj-$(CONFIG_BLK_DEV_IO_TRACE) += blktrace.o obj-$(CONFIG_EVENT_TRACER) += trace_events.o obj-$(CONFIG_EVENT_TRACER) += events.o +obj-$(CONFIG_KSYM_TRACER) += trace_ksym.o libftrace-y := ftrace.o
kernel/trace/trace.h +16 -0
@@ -12,6 +12,8 @@#include <trace/kmemtrace.h> #include <trace/power.h> +#include <asm/hw_breakpoint.h> +enum trace_type { __TRACE_FIRST_TYPE =0 ,
@@ -34,6 +36,7 @@ enum trace_type {TRACE_KMEM_FREE ,TRACE_POWER ,TRACE_BLK , +TRACE_KSYM , __TRACE_LAST_TYPE, };
@@ -191,6 +194,16 @@ struct kmemtrace_free_entry {const void *ptr; }; +struct trace_ksym { +struct trace_entry ent; +struct hw_breakpoint *ksym_hbkpt; +unsigned long ksym_addr; +unsigned long ip; +struct hlist_node ksym_hlist; +char ksym_name[KSYM_NAME_LEN ]; +char p_name[TASK_COMM_LEN ]; +}; +/* * trace_flag_type is an enumeration that holds different * states when a trace occurs. These are:
@@ -303,6 +316,7 @@ extern void __ftrace_bad_type(void); TRACE_KMEM_ALLOC); \ IF_ASSIGN(var, ent, struct kmemtrace_free_entry, \TRACE_KMEM_FREE ); \ + IF_ASSIGN(var, ent, struct trace_ksym, TRACE_KSYM); \__ftrace_bad_type (); \ }while (0 )
@@ -564,6 +578,8 @@ extern int trace_selftest_startup_syspro struct trace_array *tr);extern int trace_selftest_startup_branch (struct tracer *trace,struct trace_array *tr); +extern int trace_selftest_startup_ksym (struct tracer *trace, +struct trace_array *tr);#endif /* CONFIG_FTRACE_STARTUP_TEST */ extern void *head_page (struct trace_array_cpu *data);
kernel/trace/trace_ksym.c new file +448 -0
448 lines not on this page · raw message
kernel/trace/trace_selftest.c +36 -0
@@ -15,6 +15,7 @@ static inline int trace_valid_entry(strucase TRACE_BRANCH :case TRACE_GRAPH_ENT :case TRACE_GRAPH_RET : +case TRACE_KSYM :return 1 ; }return 0 ;
@@ -693,3 +694,38 @@ trace_selftest_startup_branch(struct tra return ret; }#endif /* CONFIG_BRANCH_TRACER */ + +#ifdef CONFIG_KSYM_TRACER +int +trace_selftest_startup_ksym (struct tracer *trace,struct trace_array *tr) +{ +unsigned long count; +int ret; + +/* start the tracing */ + ret =tracer_init (trace, tr); +if (ret) { +warn_failed_init_tracer (trace, ret); +return ret; + } + +/* Sleep for a 1/10 of a second */ +msleep (100 ); +/* stop the tracing. */ +tracing_stop (); +/* check the trace buffer */ + ret =trace_test_buffer (tr, &count); + trace->reset (tr); +tracing_start (); + +/* read & write operations - one each is performed on the dummy variable +* triggering two entries in the trace buffer +*/ +if (!ret && count !=2 ) { +printk (KERN_CONT "Ksym tracer startup test failed" ); + ret =-1 ; + } + +return ret; +} +#endif /* CONFIG_KSYM_TRACER */
Re: [Patch 11/11] ftrace plugin for kernel symbol tracing using HW Breakpoint interfaces - v2
Hi
Quoted 4 lines
This patch adds an ftrace plugin to detect and profile memory access over kernel variables. It uses HW Breakpoint interfaces to 'watch memory addresses.
Quoted 8 lines
Signed-off-by K.Prasad <[email protected]>--- kernel/trace/Kconfig | 6 kernel/trace/Makefile | 1 kernel/trace/trace.h | 16 + kernel/trace/trace_ksym.c | 448 ++++++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_selftest.c | 36 +++ 5 files changed, 507 insertions(+)
Could you please update Documentation/ftrace.txt?
I guess many user interesting this patch. :)
Re: [Patch 11/11] ftrace plugin for kernel symbol tracing using HW Breakpoint interfaces - v2
Quoted 15 lines · KOSAKI Motohiro
Hi
Quoted 4 lines
This patch adds an ftrace plugin to detect and profile memory access over kernel variables. It uses HW Breakpoint interfaces to 'watch memory addresses.
Quoted 8 lines
Signed-off-by K.Prasad <[email protected]>--- kernel/trace/Kconfig | 6 kernel/trace/Makefile | 1 kernel/trace/trace.h | 16 + kernel/trace/trace_ksym.c | 448 ++++++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_selftest.c | 36 +++ 5 files changed, 507 insertions(+)
Quoted 3 lines
Could you please update
Documentation/ftrace.txt?
I guess many user interesting this patch. :)
Sure. We have plans for a documentation file for the hardware breakpoint interfaces (something like Documentation/hw_breakpoint.txt). I shall update the Documentation/ftrace.txt with usage details for the kernel symbol tracing plugin along with them.
Thanks,
K.Prasad
Re: [Patch 11/11] ftrace plugin for kernel symbol tracing using HW Breakpoint interfaces - v2
In reply to KOSAKI MotohiroQuoted 15 lines · KOSAKI Motohiro
Hi
Quoted 4 lines
This patch adds an ftrace plugin to detect and profile memory access over kernel variables. It uses HW Breakpoint interfaces to 'watch memory addresses.
Quoted 8 lines
Signed-off-by K.Prasad <[email protected]>--- kernel/trace/Kconfig | 6 kernel/trace/Makefile | 1 kernel/trace/trace.h | 16 + kernel/trace/trace_ksym.c | 448 ++++++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_selftest.c | 36 +++ 5 files changed, 507 insertions(+)
Quoted 2 lines
Could you please update
Documentation/ftrace.txt?
I guess many user interesting this patch. :)
Yeah, it has become a really nice feature this way. As i told it to K.Prasad before: we need this tracer because the data tracer will likely become the most common usecase of this facility. We will get the hw breakpoints facility tested and used.
And in fact we can go one step further: it would also be nice to wire it up with the ftrace histogram code: so that we can get usage histograms of kernel symbol read/write activities without the overhead of tracing. (The branch tracer already has this.)
Especially frequently used variables generate a _lot_ of events.
Ingo
Re: [Patch 11/11] ftrace plugin for kernel symbol tracing using HW Breakpoint interfaces - v2
Quoted 26 lines · Ingo Molnar
Quoted 15 lines · KOSAKI Motohiro
Hi
Quoted 4 lines
This patch adds an ftrace plugin to detect and profile memory access over kernel variables. It uses HW Breakpoint interfaces to 'watch memory addresses.
Quoted 8 lines
Signed-off-by K.Prasad <[email protected]>--- kernel/trace/Kconfig | 6 kernel/trace/Makefile | 1 kernel/trace/trace.h | 16 + kernel/trace/trace_ksym.c | 448 ++++++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_selftest.c | 36 +++ 5 files changed, 507 insertions(+)Quoted 2 lines
Could you please update
Documentation/ftrace.txt?
I guess many user interesting this patch. :)Yeah, it has become a really nice feature this way. As i told it to K.Prasad before: we need this tracer because the data tracer will likely become the most common usecase of this facility. We will get the hw breakpoints facility tested and used.
Quoted 8 lines
And in fact we can go one step further: it would also be nice to wire it up with the ftrace histogram code: so that we can get usage histograms of kernel symbol read/write activities without the overhead of tracing. (The branch tracer already has this.)
Especially frequently used variables generate a _lot_ of events.
Ingo
Right, it will even be an occasion to improve and further test the histogram tracing.
K. Prasad if you need some help on how to use it, don't hesitate to tell.
Frederic.
Re: [Patch 11/11] ftrace plugin for kernel symbol tracing using HW Breakpoint interfaces - v2
Quoted 34 lines · Frederic Weisbecker
Quoted 33 lines · Ingo Molnar
Quoted 15 lines · KOSAKI Motohiro
Hi
Quoted 4 lines
This patch adds an ftrace plugin to detect and profile memory access over kernel variables. It uses HW Breakpoint interfaces to 'watch memory addresses.
Quoted 8 lines
Signed-off-by K.Prasad <[email protected]>--- kernel/trace/Kconfig | 6 kernel/trace/Makefile | 1 kernel/trace/trace.h | 16 + kernel/trace/trace_ksym.c | 448 ++++++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_selftest.c | 36 +++ 5 files changed, 507 insertions(+)Quoted 2 lines
Could you please update
Documentation/ftrace.txt?
I guess many user interesting this patch. :)Yeah, it has become a really nice feature this way. As i told it to K.Prasad before: we need this tracer because the data tracer will likely become the most common usecase of this facility. We will get the hw breakpoints facility tested and used.
And in fact we can go one step further: it would also be nice to wire it up with the ftrace histogram code: so that we can get usage histograms of kernel symbol read/write activities without the overhead of tracing. (The branch tracer already has this.)
Especially frequently used variables generate a _lot_ of events.
Quoted 8 lines
Quoted 1 line
Ingo
Right, it will even be an occasion to improve and further test the histogram tracing.
K. Prasad if you need some help on how to use it, don't hesitate to tell.Frederic.
Hi Frederic,
Thanks for the offer of help.
As I try to get ksym tracer generate histogram information, I see a few challenges and would like to know your thoughts about them.
- Unlike branch tracer which stores the branch stats in statically declared data structures ('struct ftrace_branch_data'), ksym generates data at runtime (during every memory access of interest on
the target variable) and would require dynamic memory allocation. Let's consider the case where we want the information shown below in the histogram:
Access_Type Symbol_name Function Counter
------------ ----------- -------- -------
W Sym_A Fn_A 10
W Sym_A Fn_B 15
RW Sym_C Fn_C 20We need a data structure to store the above information and a new instance of it for every new Fn_X that accesses Sym_X, while all this information captured in the context of the hardware breakpoint exception. I am not sure if dynamically allocating GFP_ATOMIC memory for such a huge requirement is a good idea.
- Alternatively if we choose to statically declare a data section (as done by the branch tracer). It should be accompanied by code to check
if we reached the end of section and wrap the pointer around. In effect, it would become a ring-buffer containing only statistics about the 'snapshot' in the buffer, and not historically aggregated data.
- Removal of the 'Function' column to display only aggregate 'hit' statistics would help reduce the complexity to a large extent, as the
counter can be embedded in the data structures containing 'ksym_trace_filter' information. But indeed, we are trading useful information for simplicity.
- Perhaps your proposed enhancements to the 'trace_stat' infrastructure, say - a generic buffering mechanism to store histogram related
information (or a framework to read through data in the ring-buffer) would help solve many of the issues. Or is 'histogram' best done in user-space?
Thanks,
K.PrasadP.S.: You can refer me as 'Prasad' although I sign as above, which is a patronymic nomenclature
(http://en.wikipedia.org/wiki/Patronymic_name#Indian_subcontinent).
Here's an illustration from another IBMer:
http://www.almaden.ibm.com/u/mohan/#name :-)
Re: [Patch 11/11] ftrace plugin for kernel symbol tracing using HW Breakpoint interfaces - v2
Quoted 63 lines · K.Prasad
Quoted 34 lines · Frederic Weisbecker
Quoted 33 lines · Ingo Molnar
Quoted 15 lines · KOSAKI Motohiro
Hi
Quoted 4 lines
This patch adds an ftrace plugin to detect and profile memory access over kernel variables. It uses HW Breakpoint interfaces to 'watch memory addresses.
Quoted 8 lines
Signed-off-by K.Prasad <[email protected]>--- kernel/trace/Kconfig | 6 kernel/trace/Makefile | 1 kernel/trace/trace.h | 16 + kernel/trace/trace_ksym.c | 448 ++++++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_selftest.c | 36 +++ 5 files changed, 507 insertions(+)Quoted 2 lines
Could you please update
Documentation/ftrace.txt?
I guess many user interesting this patch. :)Yeah, it has become a really nice feature this way. As i told it to K.Prasad before: we need this tracer because the data tracer will likely become the most common usecase of this facility. We will get the hw breakpoints facility tested and used.
And in fact we can go one step further: it would also be nice to wire it up with the ftrace histogram code: so that we can get usage histograms of kernel symbol read/write activities without the overhead of tracing. (The branch tracer already has this.)
Especially frequently used variables generate a _lot_ of events.
Quoted 8 lines
Quoted 1 line
Ingo
Right, it will even be an occasion to improve and further test the histogram tracing.
K. Prasad if you need some help on how to use it, don't hesitate to tell.Frederic.
Hi Frederic,
Thanks for the offer of help.As I try to get ksym tracer generate histogram information, I see a few challenges and would like to know your thoughts about them.
- Unlike branch tracer which stores the branch stats in statically declared data structures ('struct
ftrace_branch_data'), ksym generates data at runtime (during every memory access of interest on
the target variable) and would require dynamic memory allocation. Let's consider the case where we want the information shown below in the histogram:Access_Type Symbol_name Function Counter ------------ ----------- -------- ------- W Sym_A Fn_A 10 W Sym_A Fn_B 15 RW Sym_C Fn_C 20
Quoted 5 lines
We need a data structure to store the above information and a new instance of it for every new
Fn_Xthat accessesSym_X, while all this information captured in the context of the hardware breakpoint exception. I am not sure if dynamically allocatingGFP_ATOMICmemory for such a huge requirement is a good idea.
Ah, I see... And the number of functions that will dereference it is not predictable...
Quoted 5 lines
- Alternatively if we choose to statically declare a data section (as done by the branch tracer). It should be accompanied by code to check
if we reached the end of section and wrap the pointer around. In effect, it would become a ring-buffer containing only statistics about the 'snapshot' in the buffer, and not historically aggregated data.
That's possible, but we could loose interesting cases of function that didn't dereference a variable for a while but did it often, because they could be overriden by another. Anyway, it seems a good idea.
Quoted 5 lines
- Removal of the 'Function' column to display only aggregate 'hit' statistics would help reduce the complexity to a large extent, as the
counter can be embedded in the data structures containing 'ksym_trace_filter' information. But indeed, we are trading useful information for simplicity.
As you said, it would be too much a loss of useful informations.
Quoted 5 lines
- Perhaps your proposed enhancements to the '
trace_stat' infrastructure, say - a generic buffering mechanism to store histogram related
information (or a framework to read through data in the ring-buffer) would help solve many of the issues. Or is 'histogram' best done in user-space?
For now it's just something that provides some abstracts over the seq file and sorting facilities.
And if it can be enhanced in any way that can help, it would be great. But I don't know how I could write something generic enough to support any kind of problem that matches yours in its pattern.
In such case, the stat tracing looks a bit like regular events tracing: we don't want to allocate memory for all entries because of the different context sources and because of the allocation overhead, but we still want to store all of the events.
Such thing could rely on the ring-buffer that we are using, but we would need to create a sort of private instance of the ring buffer because we will store our hits and then we will read all that from the ring buffer directly to get the stats sum. I'm not sure we can create such private instances yet.
For now I would suggest to pre-allocate a set of entries for each breakpoints, using a predefined number (I call it n here) of entries and count the hits for the n first functions that were trapped on the breakpoint.
And if you missed some functions because n is too small, then increment an overrun variable for the current breakpoint that you can display with the stats. So that the user will know that he missed some things.
I guess it could be accompagnied by a file to change the value of n.
It's just an opinion anyway.
Quoted 7 lines
Thanks,
K.PrasadP.S.: You can refer me as 'Prasad' although I sign as above, which is a patronymic nomenclature
(http://en.wikipedia.org/wiki/Patronymic_name#Indian_subcontinent).
Here's an illustration from another IBMer:
http://www.almaden.ibm.com/u/mohan/#name :-)
Ok :-)
Thanks.
Re: [Patch 11/11] ftrace plugin for kernel symbol tracing using HW Breakpoint interfaces - v2
In reply to Ingo MolnarQuoted 26 lines · Ingo Molnar
Quoted 15 lines · KOSAKI Motohiro
Hi
Quoted 4 lines
This patch adds an ftrace plugin to detect and profile memory access over kernel variables. It uses HW Breakpoint interfaces to 'watch memory addresses.
Quoted 8 lines
Signed-off-by K.Prasad <[email protected]>--- kernel/trace/Kconfig | 6 kernel/trace/Makefile | 1 kernel/trace/trace.h | 16 + kernel/trace/trace_ksym.c | 448 ++++++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_selftest.c | 36 +++ 5 files changed, 507 insertions(+)Quoted 2 lines
Could you please update
Documentation/ftrace.txt?
I guess many user interesting this patch. :)Yeah, it has become a really nice feature this way. As i told it to K.Prasad before: we need this tracer because the data tracer will likely become the most common usecase of this facility. We will get the hw breakpoints facility tested and used.
Quoted 8 lines
And in fact we can go one step further: it would also be nice to wire it up with the ftrace histogram code: so that we can get usage histograms of kernel symbol read/write activities without the overhead of tracing. (The branch tracer already has this.)
Especially frequently used variables generate a _lot_ of events.
Ingo
I am looking at the histogram infrastructure, its implementation in branch tracer for getting the same in ksym tracer. Meanwhile I'm thinking if the patchset can be included in -tip tree for some good amount of testing and feedback from the community before they make their way into mainline (2.6.30?).
I will be glad if you can pick the patches to be a part of -tip tree.
Thanks,
K.Prasad