Hardware Breakpoint interfaces
Subsystems (4)
cover Hardware Breakpoint interfaces
[Re-posting the patches with LKML included as a recipient]
Hi Ingo,
Please find the patches that introduce interfaces to access
Hardware Breakpoint (or watchpoint) registers and an arch-specific implementation for x86/x86_64 architecture. The patches have been re-based to -tip tree (commit 4e3bae758b552d6bd094fb36e78a4d738cb42257)
A short description of the framework and interfaces is provided below. Some of the details will be specific to x86/x86_64 architecture and may vary depending upon the support provided by the target processor.
Kindly accept these patches to become a part of -tip tree.
Description
-------------
The Hardware Breakpoint registers can be used for tracing changes to a variable or data location (even I/O ports in x86/x86_64) and will be extremely helpful in debugging problems such as memory corruption. While these registers have been used by user-space debuggers for long (through 'ptrace' syscalls), Kgdb exploited them for kernel-space addresses.
The proposed framework, introduces interfaces to use them directly on both user- and kernel-space addresses apart from arbitrating requests from various such users for the limited number of registers.
The interfaces are:
int register_kernel_hw_breakpoint(struct hw_breakpoint *); void unregister_kernel_hw_breakpoint(struct hw_breakpoint *bp);
int register_user_hw_breakpoint(struct task_struct *tsk,
struct hw_breakpoint *bp);
void unregister_user_hw_breakpoint(struct task_struct *tsk,
struct hw_breakpoint *bp);The 'struct hw_breakpoint' will be the anchor data-structure containing all the necessary information such as name or address, type, length, priority and pointers to handler functions (some of which are arch-specific). More information about the role of each field, the handler
functions and their return values can be found in the descriptive comments
preceding these functions and in "include/asm-generic/hw_breakpoint.h".
While (un)register_user_hw_breakpoint() isn't exported yet, its worker-routine __register_user-hw_breakpoint() is used by ptrace syscall for all breakpoint register requirements. For the kernel-space, a simple use case to trace 'write' operations on a kernel variable can be found in samples/hw_breakpoint/data_breakpoint.c.
In the current patchset, support is provided only for read and read-write breakpoints on data locations, which can be later expand to include instruction and I/O breakpoints for x86/x86_64.
There is pending integration with 'KGDB' without which mutual exclusion between them (KGDB and HW breakpoint use through above interfaces) needs to be observed.
Thanks,
K.Prasad
PATCH 1/10 Introducing generic hardware breakpoint handler interfaces
Introducing generic hardware breakpoint handler interfaces
From: Alan Stern <[email protected]>
This patch introduces two new files hw_breakpoint.[ch] which defines the generic interfaces to use hardware breakpoint infrastructure of the system.
[K.Prasad: Re-based the original patch to newer kernel base, modified the
register_<kernel/user>_hw_breakpoint() interfaces and split the monolithic patch into smaller ones. Split-out from the bigger patch and minor changes following re-basing]
--- include/asm-generic/hw_breakpoint.h | 243 +++++++++++ kernel/Makefile | 2 kernel/hw_breakpoint.c | 772 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1016 insertions(+), 1 deletion(-)
kernel/hw_breakpoint.c new file +772 -0
@@ -0,0 +1,772 @@ +/* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +* +* Copyright (C) 2007 Alan Stern +* Copyright (C) IBM Corporation, 2009 +*/ + +/* +* HW_breakpoint: a unified kernel/user-space hardware breakpoint facility, +* using the CPU's debug registers. +* +* This file contains the arch-independent routines. It is not meant +* to be compiled as a standalone source file; rather it should be +* #include'd by the arch-specific implementation. +*/ + +#include <linux/init.h> +#include <linux/irqflags.h> +#include <linux/kdebug.h> +#include <linux/kernel.h> +#include <linux/kprobes.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/notifier.h> +#include <linux/rculist.h> +#include <linux/sched.h> +#include <linux/smp.h> +#include <linux/percpu.h> +#include <linux/kallsyms.h> + +#include <asm/debugreg.h> +#include <asm/hw_breakpoint.h> +#include <asm/processor.h> + +/* Global info */ +struct kernel_bp_data kbpdata[2 ];/* Old and new settings */ +int cur_kbpindex;/* Alternates 0, 1, ... */ +struct kernel_bp_data *cur_kbpdata = &kbpdata[0 ]; +/* Always equal to &kbpdata[cur_kbpindex] */ + +static u8 tprio[HB_NUM ];/* Thread bp max priorities */ +LIST_HEAD (kernel_bps);/* Kernel breakpoint list */ +static LIST_HEAD (thread_list);/* thread_hw_breakpoint list */ +DEFINE_PER_CPU(struct cpu_hw_breakpoint, cpu_bp); + +/* +* Install the debug register values for a new thread. +*/ +void switch_to_thread_hw_breakpoint (struct task_struct *tsk) +{ +struct thread_hw_breakpoint *thbi = tsk->thread.hw_breakpoint_info; +struct cpu_hw_breakpoint *chbi; +struct kernel_bp_data *thr_kbpdata; + +/* This routine is on the hot path; it gets called for every +* context switch into a task with active breakpoints. We +* must make sure that the common case executes as quickly as +* possible. +*/ + chbi = &per_cpu (cpu_bp,get_cpu ()); + chbi->bp_task = tsk; + +/* Use RCU to synchronize with external updates */ +rcu_read_lock (); + +/* Other CPUs might be making updates to the list of kernel +* breakpoints at this time. If they are, they will modify +* the other entry in kbpdata[] -- the one not pointed to +* by chbi->cur_kbpdata. So the update itself won't affect +* us directly. +* +* However when the update is finished, an IPI will arrive +* telling this CPU to change chbi->cur_kbpdata. We need +* to use a single consistent kbpdata[] entry, the present one. +* So we'll copy the pointer to a local variable, thr_kbpdata, +* and we must prevent the compiler from aliasing the two +* pointers. Only a compiler barrier is required, not a full +* memory barrier, because everything takes place on a single CPU. +*/ + restart: + thr_kbpdata =ACCESS_ONCE (chbi->cur_kbpdata); + +/* Normally we can keep the same debug register settings as the +* last time this task ran. But if the kernel breakpoints have +* changed or any user breakpoints have been registered or +* unregistered, we need to handle the updates and possibly +* send out some notifications. +*/ +if (unlikely (thbi->gennum != thr_kbpdata->gennum)) { +struct hw_breakpoint *bp; +int i; +int num; + + thbi->gennum = thr_kbpdata->gennum; +arch_update_thbi (thbi, thr_kbpdata); + num = thr_kbpdata->num_kbps; + +/* This code can be invoked while a debugger is actively +* updating the thread's breakpoint list. We use RCU to +* protect our access to the list pointers. */ + thbi->num_installed =0 ; + i =HB_NUM ; +list_for_each_entry_rcu (bp, &thbi->thread_bps, node) { + +/* If this register is allocated for kernel bps, +* don't install. Otherwise do. */ +if (--i < num) { +if (bp->status ==HW_BREAKPOINT_INSTALLED ) { +if (bp->uninstalled) + (bp->uninstalled)(bp); + bp->status =HW_BREAKPOINT_REGISTERED ; + } + }else { + ++thbi->num_installed; +if (bp->status !=HW_BREAKPOINT_INSTALLED ) { + bp->status =HW_BREAKPOINT_INSTALLED ; +if (bp->installed) + (bp->installed)(bp); + } + } + } + } + +/* Set the debug register */ +arch_install_thbi (thbi); + +/* Were there any kernel breakpoint changes while we were running? */ +if (unlikely (chbi->cur_kbpdata != thr_kbpdata)) { + +/* Some debug registers now be assigned to kernel bps and +* we might have messed them up. Reload all the kernel bps +* and then reload the thread bps. +*/ +arch_install_chbi (chbi); + goto restart; + } + +rcu_read_unlock (); +put_cpu_no_resched (); +} + +/* +* Install the debug register values for just the kernel, no thread. +*/ +void switch_to_none_hw_breakpoint (void ) +{ +struct cpu_hw_breakpoint *chbi; + + chbi = &per_cpu (cpu_bp,get_cpu ()); + chbi->bp_task =NULL ; + +/* This routine gets called from only two places. In one +* the caller holds the hw_breakpoint_mutex; in the other +* interrupts are disabled. In either case, no kernel +* breakpoint updates can arrive while the routine runs. +* So we don't need to use RCU. +*/ +arch_install_none (chbi); +put_cpu_no_resched (); +} + +/* +* Update the debug registers on this CPU. +*/ +static void update_this_cpu (void *unused) +{ +struct cpu_hw_breakpoint *chbi; +struct task_struct *tsk = current; + + chbi = &per_cpu (cpu_bp,get_cpu ()); + +rcu_read_lock (); +/* Install both the kernel and the user breakpoints */ +arch_install_chbi (chbi); +if (test_tsk_thread_flag (tsk,TIF_DEBUG )) +switch_to_thread_hw_breakpoint (tsk); + +rcu_read_unlock (); +put_cpu_no_resched (); +} + +/* +* Tell all CPUs to update their debug registers. +* +* The caller must hold hw_breakpoint_mutex. +*/ +static void update_all_cpus (void ) +{ +/* We don't need to use any sort of memory barrier. The IPI +* carried out by on_each_cpu() includes its own barriers. +*/ +on_each_cpu (update_this_cpu,NULL ,0 ); +synchronize_rcu (); +} + +/* +* Load the debug registers during startup of a CPU. +*/ +void load_debug_registers (void ) +{ +unsigned long flags; + +/* Prevent IPIs for new kernel breakpoint updates */ +local_irq_save (flags); +update_this_cpu (NULL ); +local_irq_restore (flags); +} + +/* +* Take the 4 highest-priority breakpoints in a thread and accumulate +* their priorities in tprio. Highest-priority entry is in tprio[3]. +*/ +static void accum_thread_tprio (struct thread_hw_breakpoint *thbi) +{ +int i; + +for (i =HB_NUM -1 ; i >=0 && thbi->bps[i]; --i) + tprio[i] =max (tprio[i], thbi->bps[i]->priority); +} + +/* +* Recalculate the value of the tprio array, the maximum priority levels +* requested by user breakpoints in all threads. +* +* Each thread has a list of registered breakpoints, kept in order of +* decreasing priority. We'll set tprio[0] to the maximum priority of +* the first entries in all the lists, tprio[1] to the maximum priority +* of the second entries in all the lists, etc. In the end, we'll know +* that no thread requires breakpoints with priorities higher than the +* values in tprio. +* +* The caller must hold hw_breakpoint_mutex. +*/ +static void recalc_tprio (void ) +{ +struct thread_hw_breakpoint *thbi; + +memset (tprio,0 ,sizeof tprio); + +/* Loop through all threads having registered breakpoints +* and accumulate the maximum priority levels in tprio. +*/ +list_for_each_entry (thbi, &thread_list, node) +accum_thread_tprio (thbi); +} + +/* +* Decide how many debug registers will be allocated to kernel breakpoints +* and consequently, how many remain available for user breakpoints. +* +* The priorities of the entries in the list of registered kernel bps +* are compared against the priorities stored in tprio[]. The 4 highest +* winners overall get to be installed in a debug register; num_kpbs +* keeps track of how many of those winners come from the kernel list. +* +* If num_kbps changes, or if a kernel bp changes its installation status, +* then call update_all_cpus() so that the debug registers will be set +* correctly on every CPU. If neither condition holds then the set of +* kernel bps hasn't changed, and nothing more needs to be done. +* +* The caller must hold hw_breakpoint_mutex. +*/ +static void balance_kernel_vs_user (void ) +{ +int k, u; +int changed =0 ; +struct hw_breakpoint *bp; +struct kernel_bp_data *new_kbpdata; + +/* Determine how many debug registers are available for kernel +* breakpoints as opposed to user breakpoints, based on the +* priorities. Ties are resolved in favor of user bps. +*/ + k =0 ;/* Next kernel bp to allocate */ + u =HB_NUM -1 ;/* Next user bp to allocate */ + + bp = list_entry(kernel_bps.next, struct hw_breakpoint, node); +while (k <= u) { +if (&bp->node == &kernel_bps || tprio[u] >= bp->priority) + --u;/* User bps win a slot */ +else { + ++k;/* Kernel bp wins a slot */ +if (bp->status !=HW_BREAKPOINT_INSTALLED ) + changed =1 ; + bp = list_entry(bp->node.next, struct hw_breakpoint, + node); + } + } +if (k != cur_kbpdata->num_kbps) + changed =1 ; + +/* Notify the remaining kernel breakpoints that they are about +* to be uninstalled. +*/ +list_for_each_entry_from (bp, &kernel_bps, node) { +if (bp->status ==HW_BREAKPOINT_INSTALLED ) { +if (bp->uninstalled) + (bp->uninstalled)(bp); + bp->status =HW_BREAKPOINT_REGISTERED ; + changed =1 ; + } + } + +if (changed) { + cur_kbpindex ^=1 ; + new_kbpdata = &kbpdata[cur_kbpindex]; + new_kbpdata->gennum = cur_kbpdata->gennum +1 ; + new_kbpdata->num_kbps = k; +arch_new_kbpdata (new_kbpdata); + u =0 ; +list_for_each_entry (bp, &kernel_bps, node) { +if (u >= k) +break ; + new_kbpdata->bps[u] = bp; + ++u; + } +rcu_assign_pointer (cur_kbpdata, new_kbpdata); + +/* Tell all the CPUs to update their debug registers */ +update_all_cpus (); + +/* Notify the breakpoints that just got installed */ +for (u =0 ; u < k; ++u) { + bp = new_kbpdata->bps[u]; +if (bp->status !=HW_BREAKPOINT_INSTALLED ) { + bp->status =HW_BREAKPOINT_INSTALLED ; +if (bp->installed) + (bp->installed)(bp); + } + } + } +} + +/* +* Return the pointer to a thread's hw_breakpoint info area, +* and try to allocate one if it doesn't exist. +* +* The caller must hold hw_breakpoint_mutex. +*/ +struct thread_hw_breakpoint *alloc_thread_hw_breakpoint ( +struct task_struct *tsk) +{ +if (!tsk->thread.hw_breakpoint_info && !(tsk->flags &PF_EXITING )) { +struct thread_hw_breakpoint *thbi; + + thbi =kzalloc (sizeof (struct thread_hw_breakpoint ), +GFP_KERNEL ); +if (thbi) { +INIT_LIST_HEAD (&thbi->node); +INIT_LIST_HEAD (&thbi->thread_bps); + +/* Force an update the next time tsk runs */ + thbi->gennum = cur_kbpdata->gennum -2 ; + tsk->thread.hw_breakpoint_info = thbi; + } + } +return tsk->thread.hw_breakpoint_info; +} + +/* +* Erase all the hardware breakpoint info associated with a thread. +* +* If tsk != current then tsk must not be usable (for example, a +* child being cleaned up from a failed fork). +*/ +void flush_thread_hw_breakpoint (struct task_struct *tsk) +{ +struct thread_hw_breakpoint *thbi = tsk->thread.hw_breakpoint_info; +struct hw_breakpoint *bp; + +if (!thbi) +return ; +mutex_lock (&hw_breakpoint_mutex); + +/* Let the breakpoints know they are being uninstalled */ +list_for_each_entry (bp, &thbi->thread_bps, node) { +if (bp->status ==HW_BREAKPOINT_INSTALLED && bp->uninstalled) + (bp->uninstalled)(bp); + bp->status =0 ; + } + +/* Remove tsk from the list of all threads with registered bps */ +list_del (&thbi->node); + +/* The thread no longer has any breakpoints associated with it */ +clear_tsk_thread_flag (tsk,TIF_DEBUG ); + tsk->thread.hw_breakpoint_info =NULL ; +kfree (thbi); + +/* Recalculate and rebalance the kernel-vs-user priorities */ +recalc_tprio (); +balance_kernel_vs_user (); + +/* Actually uninstall the breakpoints if necessary */ +if (tsk == current) +switch_to_none_hw_breakpoint (); +mutex_unlock (&hw_breakpoint_mutex); +} + +/* +* Copy the hardware breakpoint info from a thread to its cloned child. +*/ +int copy_thread_hw_breakpoint (struct task_struct *tsk, +struct task_struct *child,unsigned long clone_flags) +{ +/* We will assume that breakpoint settings are not inherited +* and the child starts out with no debug registers set. +* But what about CLONE_PTRACE? +*/ +clear_tsk_thread_flag (child,TIF_DEBUG ); +return 0 ; +} + +/* +* Store the highest-priority thread breakpoint entries in an array. +*/ +static void store_thread_bp_array (struct thread_hw_breakpoint *thbi) +{ +struct hw_breakpoint *bp; +int i; + + i =HB_NUM -1 ; +list_for_each_entry (bp, &thbi->thread_bps, node) { + thbi->bps[i] = bp; +arch_store_thread_bp_array (thbi, bp, i); +if (--i <0 ) +break ; + } +while (i >=0 ) + thbi->bps[i--] =NULL ; + +/* Force an update the next time this task runs */ + thbi->gennum = cur_kbpdata->gennum -2 ; +} + +/* +* Insert a new breakpoint in a priority-sorted list. +* Return the bp's index in the list. +* +* Thread invariants: +* tsk_thread_flag(tsk, TIF_DEBUG) set implies +* tsk->thread.hw_breakpoint_info is not NULL. +* tsk_thread_flag(tsk, TIF_DEBUG) set iff thbi->thread_bps is non-empty +* iff thbi->node is on thread_list. +*/ +static int insert_bp_in_list (struct hw_breakpoint *bp, +struct thread_hw_breakpoint *thbi,struct task_struct *tsk) +{ +struct list_head *head; +int pos; +struct hw_breakpoint *temp_bp; + +/* tsk and thbi are NULL for kernel bps, non-NULL for user bps */ +if (tsk) + head = &thbi->thread_bps; +else + head = &kernel_bps; + +/* Equal-priority breakpoints get listed first-come-first-served */ + pos =0 ; +list_for_each_entry (temp_bp, head, node) { +if (bp->priority > temp_bp->priority) +break ; + ++pos; + } + bp->status =HW_BREAKPOINT_REGISTERED ; +list_add_tail (&bp->node, &temp_bp->node); + +if (tsk) { +store_thread_bp_array (thbi); + +/* Is this the thread's first registered breakpoint? */ +if (list_empty (&thbi->node)) { +set_tsk_thread_flag (tsk,TIF_DEBUG ); +list_add (&thbi->node, &thread_list); + } + } +return pos; +} + +/* +* Remove a breakpoint from its priority-sorted list. +* +* See the invariants mentioned above. +*/ +static void remove_bp_from_list (struct hw_breakpoint *bp, +struct thread_hw_breakpoint *thbi,struct task_struct *tsk) +{ +/* Remove bp from the thread's/kernel's list. If the list is now +* empty we must clear the TIF_DEBUG flag. But keep the +* thread_hw_breakpoint structure, so that the virtualized debug +* register values will remain valid. +*/ +list_del (&bp->node); +if (tsk) { +store_thread_bp_array (thbi); + +if (list_empty (&thbi->thread_bps)) { +list_del_init (&thbi->node); +clear_tsk_thread_flag (tsk,TIF_DEBUG ); + } + } + +/* Tell the breakpoint it is being uninstalled */ +if (bp->status ==HW_BREAKPOINT_INSTALLED && bp->uninstalled) + (bp->uninstalled)(bp); + bp->status =0 ; +} + +/* +* Validate the settings in a hw_breakpoint structure. +*/ +static int validate_settings (struct hw_breakpoint *bp,struct task_struct *tsk) +{ +int ret; +unsigned int align; + + ret =arch_validate_hwbkpt_settings (bp, &align, tsk); +if (ret <0 ) + goto err; + +/* Check that the low-order bits of the address are appropriate +* for the alignment implied by len. +*/ +if (bp->info.address & align) +return -EINVAL ; + +/* Check that the virtual address is in the proper range */ +if (tsk) { +if (!arch_check_va_in_userspace (bp->info.address, tsk)) +return -EFAULT ; + }else { +if (!arch_check_va_in_kernelspace (bp->info.address)) +return -EFAULT ; + } + err: +return ret; +} + +/* +* Actual implementation of register_user_hw_breakpoint. +*/ +int __register_user_hw_breakpoint (struct task_struct *tsk, +struct hw_breakpoint *bp) +{ +int rc; +struct thread_hw_breakpoint *thbi; +int pos; + + bp->status =0 ; + rc =validate_settings (bp, tsk); +if (rc) +return rc; + + thbi =alloc_thread_hw_breakpoint (tsk); +if (!thbi) +return -ENOMEM ; + +/* Insert bp in the thread's list */ + pos =insert_bp_in_list (bp, thbi, tsk); +arch_register_user_hw_breakpoint (bp, thbi); + +/* Update and rebalance the priorities. We don't need to go through +* the list of all threads; adding a breakpoint can only cause the +* priorities for this thread to increase. +*/ +accum_thread_tprio (thbi); +balance_kernel_vs_user (); + +/* Did bp get allocated to a debug register? We can tell from its +* position in the list. The number of registers allocated to +* kernel breakpoints is num_kbps; all the others are available for +* user breakpoints. If bp's position in the priority-ordered list +* is low enough, it will get a register. +*/ +if (pos <HB_NUM - cur_kbpdata->num_kbps) { + rc =1 ; + +/* Does it need to be installed right now? */ +if (tsk == current) +switch_to_thread_hw_breakpoint (tsk); +/* Otherwise it will get installed the next time tsk runs */ + } + +return rc; +} + +/** +* register_user_hw_breakpoint - register a hardware breakpoint for user space +* @tsk: the task in whose memory space the breakpoint will be set +* @bp: the breakpoint structure to register +* @address: location (virtual address) of the breakpoint +* @len: encoded extent of the breakpoint address (1, 2, 4, or 8 bytes) +* @type: breakpoint type (read-only, write-only, read-write, or execute) +* +* This routine registers a breakpoint to be associated with @tsk's +* memory space and active only while @tsk is running. It does not +* guarantee that the breakpoint will be allocated to a debug register +* immediately; there may be other higher-priority breakpoints registered +* which require the use of all the debug registers. +* +* @tsk will normally be a process being debugged by the current process, +* but it may also be the current process. +* +* @bp->address, @bp->len, @bp->type, @bp->triggered and @bp->priority must be +* set properly before invocation +* +* Returns 1 if @bp is allocated to a debug register, 0 if @bp is +* registered but not allowed to be installed, otherwise a negative error +* code. +*/ +int register_user_hw_breakpoint (struct task_struct *tsk, +struct hw_breakpoint *bp) +{ +int rc; + +mutex_lock (&hw_breakpoint_mutex); + rc =__register_user_hw_breakpoint (tsk, bp); +mutex_unlock (&hw_breakpoint_mutex); +return rc; +} + +/* +* Actual implementation of unregister_user_hw_breakpoint. +*/ +void __unregister_user_hw_breakpoint (struct task_struct *tsk, +struct hw_breakpoint *bp) +{ +struct thread_hw_breakpoint *thbi = tsk->thread.hw_breakpoint_info; + +if (!bp->status) +return ;/* Not registered */ + +/* Remove bp from the thread's list */ +remove_bp_from_list (bp, thbi, tsk); +arch_unregister_user_hw_breakpoint (bp, thbi); + +/* Recalculate and rebalance the kernel-vs-user priorities, +* and actually uninstall bp if necessary. +*/ +recalc_tprio (); +balance_kernel_vs_user (); +if (tsk == current) +switch_to_thread_hw_breakpoint (tsk); +} + +/** +* unregister_user_hw_breakpoint - unregister a hardware breakpoint for user space +* @tsk: the task in whose memory space the breakpoint is registered +* @bp: the breakpoint structure to unregister +* +* Uninstalls and unregisters @bp. +*/ +void unregister_user_hw_breakpoint (struct task_struct *tsk, +struct hw_breakpoint *bp) +{ +mutex_lock (&hw_breakpoint_mutex); +__unregister_user_hw_breakpoint (tsk, bp); +mutex_unlock (&hw_breakpoint_mutex); +} + +/** +* register_kernel_hw_breakpoint - register a hardware breakpoint for kernel space +* @bp: the breakpoint structure to register +* +* This routine registers a breakpoint to be active at all times. It +* does not guarantee that the breakpoint will be allocated to a debug +* register immediately; there may be other higher-priority breakpoints +* registered which require the use of all the debug registers. +* +* @bp.info->name or @bp.info->address, @bp.info->len, @bp.info->type, +* @bp->triggered and @bp->priority must be set properly before invocation +* +* Returns 1 if @bp is allocated to a debug register, 0 if @bp is +* registered but not allowed to be installed, otherwise a negative error +* code. +*/ +int register_kernel_hw_breakpoint (struct hw_breakpoint *bp) +{ +int rc; +int pos; + + bp->status =0 ; + rc =validate_settings (bp,NULL ); +if (rc) +return rc; + +mutex_lock (&hw_breakpoint_mutex); + +/* Insert bp in the kernel's list */ + pos =insert_bp_in_list (bp,NULL ,NULL ); +arch_register_kernel_hw_breakpoint (bp); + +/* Rebalance the priorities. This will install bp if it +* was allocated a debug register. +*/ +balance_kernel_vs_user (); + +/* Did bp get allocated to a debug register? We can tell from its +* position in the list. The number of registers allocated to +* kernel breakpoints is num_kbps; all the others are available for +* user breakpoints. If bp's position in the priority-ordered list +* is low enough, it will get a register. +*/ +if (pos < cur_kbpdata->num_kbps) + rc =1 ; + +mutex_unlock (&hw_breakpoint_mutex); +return rc; +} +EXPORT_SYMBOL_GPL (register_kernel_hw_breakpoint); + +/** +* unregister_kernel_hw_breakpoint - unregister a hardware breakpoint for kernel space +* @bp: the breakpoint structure to unregister +* +* Uninstalls and unregisters @bp. +*/ +void unregister_kernel_hw_breakpoint (struct hw_breakpoint *bp) +{ +if (!bp->status) +return ;/* Not registered */ +mutex_lock (&hw_breakpoint_mutex); + +/* Remove bp from the kernel's list */ +remove_bp_from_list (bp,NULL ,NULL ); +arch_unregister_kernel_hw_breakpoint (bp); + +/* Rebalance the priorities. This will uninstall bp if it +* was allocated a debug register. +*/ +balance_kernel_vs_user (); + +mutex_unlock (&hw_breakpoint_mutex); +} +EXPORT_SYMBOL_GPL (unregister_kernel_hw_breakpoint); + +/* +* Handle debug exception notifications. +*/ +static int __kprobes hw_breakpoint_exceptions_notify ( +struct notifier_block *unused,unsigned long val,void *data) +{ +if (val !=DIE_DEBUG ) +return NOTIFY_DONE ; +return hw_breakpoint_handler (data); +} + +static struct notifier_block hw_breakpoint_exceptions_nb = { + .notifier_call = hw_breakpoint_exceptions_notify, + .priority =0x7fffffff /* we need to be notified first */ +}; + +static int __init init_hw_breakpoint (void ) +{ +load_debug_registers (); +return register_die_notifier (&hw_breakpoint_exceptions_nb); +} + +core_initcall (init_hw_breakpoint);
kernel/Makefile +1 -1
@@ -10,7 +10,7 @@ obj-y = sched.o fork.o exec_domain.o kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o mutex.o \ hrtimer.o rwsem.o nsproxy.o srcu.o semaphore.o \ notifier.o ksysfs.o pm_qos_params.o sched_clock.o cred.o \ - async.o + async.o hw_breakpoint.o ifdef CONFIG_FUNCTION_TRACER # Do not trace debug files and internal ftrace files
include/asm-generic/hw_breakpoint.h new file +243 -0
243 lines not on this page · raw message
PATCH 2/10 x86 architecture implementation of Hardware Breakpoint interfaces
In reply to K.Prasadx86 architecture implementation of Hardware Breakpoint interfaces
From: Alan Stern <[email protected]>
This patch introduces two new files named hw_breakpoint.[ch] inside x86 specific directories. They contain functions which help validate and serve requests for using Hardware Breakpoint registers on x86 processors.
[K.Prasad: More declarations in hw_breakpoint.h to independently compile each
hw_breakpoint.c files. Split-out from the bigger patch and minor changes following re-basing]
--- arch/x86/include/asm/hw_breakpoint.h | 132 ++++++++++ arch/x86/kernel/Makefile | 2 arch/x86/kernel/hw_breakpoint.c | 437 +++++++++++++++++++++++++++++++++++ 3 files changed, 570 insertions(+), 1 deletion(-)
arch/x86/kernel/hw_breakpoint.c new file +437 -0
@@ -0,0 +1,437 @@ +/* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +* +* Copyright (C) 2007 Alan Stern +* Copyright (C) 2009 IBM Corporation +*/ + +/* +* HW_breakpoint: a unified kernel/user-space hardware breakpoint facility, +* using the CPU's debug registers. +*/ + +#include <linux/init.h> +#include <linux/irqflags.h> +#include <linux/kdebug.h> +#include <linux/kernel.h> +#include <linux/kprobes.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/notifier.h> +#include <linux/rculist.h> +#include <linux/sched.h> +#include <linux/smp.h> +#include <linux/percpu.h> +#include <linux/kallsyms.h> + +#include <asm/debugreg.h> +#include <asm/hw_breakpoint.h> +#include <asm/processor.h> + +static unsigned long kdr7;/* Unmasked kernel DR7 value */ + +/* Masks for the bits in DR7 related to kernel breakpoints, for various +* values of num_kbps. Entry n is the mask for when there are n kernel +* breakpoints, in debug registers 0 - (n-1). The DR_GLOBAL_SLOWDOWN bit +* (GE) is handled specially. +*/ +static const unsigned long kdr7_masks[HB_NUM +1 ] = { +0x00000000 , +0x000f0003 ,/* LEN0, R/W0, G0, L0 */ +0x00ff000f ,/* Same for 0,1 */ +0x0fff003f ,/* Same for 0,1,2 */ +0xffff00ff /* Same for 0,1,2,3 */ +}; + +/* +* Install the kernel breakpoints in their debug registers. +*/ +void arch_install_chbi (struct cpu_hw_breakpoint *chbi) +{ +struct hw_breakpoint **bps; + +/* Don't allow debug exceptions while we update the registers */ +set_debugreg (0UL ,7 ); + chbi->cur_kbpdata =rcu_dereference (cur_kbpdata); + +/* Kernel breakpoints are stored starting in DR0 and going up */ + bps = chbi->cur_kbpdata->bps; +switch (chbi->cur_kbpdata->num_kbps) { +case 4 : +set_debugreg (bps[3 ]->info.address,3 ); +case 3 : +set_debugreg (bps[2 ]->info.address,2 ); +case 2 : +set_debugreg (bps[1 ]->info.address,1 ); +case 1 : +set_debugreg (bps[0 ]->info.address,0 ); + } +/* No need to set DR6 */ +set_debugreg (chbi->cur_kbpdata->mkdr7,7 ); +} + +/* +* Update an out-of-date thread hw_breakpoint info structure. +*/ +void arch_update_thbi (struct thread_hw_breakpoint *thbi, +struct kernel_bp_data *thr_kbpdata) +{ +int num = thr_kbpdata->num_kbps; + + thbi->tkdr7 = thr_kbpdata->mkdr7 | (thbi->tdr7 & ~kdr7_masks[num]); +} + +/* +* Install the thread breakpoints in their debug registers. +*/ +void arch_install_thbi (struct thread_hw_breakpoint *thbi) +{ +/* Install the user breakpoints. Kernel breakpoints are stored +* starting in DR0 and going up; there are num_kbps of them. +* User breakpoints are stored starting in DR3 and going down, +* as many as we have room for. +*/ +switch (thbi->num_installed) { +case 4 : +set_debugreg (thbi->tdr[0 ],0 ); +case 3 : +set_debugreg (thbi->tdr[1 ],1 ); +case 2 : +set_debugreg (thbi->tdr[2 ],2 ); +case 1 : +set_debugreg (thbi->tdr[3 ],3 ); + } +/* No need to set DR6 */ +set_debugreg (thbi->tkdr7,7 ); +} + +/* +* Install the debug register values for just the kernel, no thread. +*/ +void arch_install_none (struct cpu_hw_breakpoint *chbi) +{ +set_debugreg (chbi->cur_kbpdata->mkdr7,7 ); +} + +/* +* Create a new kbpdata entry. +*/ +void arch_new_kbpdata (struct kernel_bp_data *new_kbpdata) +{ +int num = new_kbpdata->num_kbps; + + new_kbpdata->mkdr7 = kdr7 & (kdr7_masks[num] |DR_GLOBAL_SLOWDOWN ); +} + +/* +* Store a thread breakpoint array entry's address +*/ +void arch_store_thread_bp_array (struct thread_hw_breakpoint *thbi, +struct hw_breakpoint *bp,int i) +{ + thbi->tdr[i] = bp->info.address; +} + +/* +* Check for virtual address in user space. +*/ +int arch_check_va_in_userspace (unsigned long va,struct task_struct *tsk) +{ +return (va <TASK_SIZE ); +} + +/* +* Check for virtual address in kernel space. +*/ +int arch_check_va_in_kernelspace (unsigned long va) +{ +return (va >=TASK_SIZE ); +} + +/* +* Store a breakpoint's encoded address, length, and type. +*/ +void arch_store_info (struct hw_breakpoint *bp) +{ +/* +* User-space requests will always have the address field populated +* For kernel-addresses, either the address or symbol name can be +* specified. +*/ +if (bp->info.address) +return ; + bp->info.address = (unsigned long )kallsyms_lookup_name (bp->info.name); +} + +/* +* Validate the arch-specific HW Breakpoint register settings +*/ +int arch_validate_hwbkpt_settings (struct hw_breakpoint *bp, +unsigned int *align,struct task_struct *tsk) +{ +int ret = -EINVAL ; + +switch (bp->info.type) { + +/* Ptrace-refactoring code +* For now, we'll allow instruction breakpoint only for user-space +* addresses +*/ +case HW_BREAKPOINT_EXECUTE : +if ((!arch_check_va_in_userspace (bp->info.address, tsk)) && + bp->info.len !=HW_BREAKPOINT_LEN_EXECUTE ) +return ret; +break ; +case HW_BREAKPOINT_WRITE : +break ; +case HW_BREAKPOINT_RW : +break ; +default : +return ret; + } + +switch (bp->info.len) { +case HW_BREAKPOINT_LEN_1 : + *align =0 ; +break ; +case HW_BREAKPOINT_LEN_2 : + *align =1 ; +break ; +case HW_BREAKPOINT_LEN_4 : + *align =3 ; +break ; +default : +return ret; + } + +if (bp->triggered) { + ret =0 ; +arch_store_info (bp); + } +return ret; +} + +/* +* Encode the length, type, Exact, and Enable bits for a particular breakpoint +* as stored in debug register 7. +*/ +static unsigned long encode_dr7 (int drnum,unsigned len,unsigned type) +{ +unsigned long temp; + + temp = (len | type) &0xf ; + temp <<= (DR_CONTROL_SHIFT + drnum *DR_CONTROL_SIZE ); + temp |= (DR_GLOBAL_ENABLE << (drnum *DR_ENABLE_SIZE )) | +DR_GLOBAL_SLOWDOWN ; +return temp; +} + +/* +* Calculate the DR7 value for a list of kernel or user breakpoints. +*/ +static unsigned long calculate_dr7 (struct thread_hw_breakpoint *thbi) +{ +int is_user; +struct list_head *bp_list; +struct hw_breakpoint *bp; +int i; +int drnum; +unsigned long dr7; + +if (thbi) { + is_user =1 ; + bp_list = &thbi->thread_bps; + drnum =HB_NUM -1 ; + }else { + is_user =0 ; + bp_list = &kernel_bps; + drnum =0 ; + } + +/* Kernel bps are assigned from DR0 on up, and user bps are assigned +* from DR3 on down. Accumulate all 4 bps; the kernel DR7 mask will +* select the appropriate bits later. +*/ + dr7 =0 ; + i =0 ; +list_for_each_entry (bp, bp_list, node) { + +/* Get the debug register number and accumulate the bits */ + dr7 |=encode_dr7 (drnum, bp->info.len, bp->info.type); +if (++i >=HB_NUM ) +break ; +if (is_user) + --drnum; +else + ++drnum; + } +return dr7; +} + +/* +* Register a new user breakpoint structure. +*/ +void arch_register_user_hw_breakpoint (struct hw_breakpoint *bp, +struct thread_hw_breakpoint *thbi) +{ + thbi->tdr7 =calculate_dr7 (thbi); + +/* If this is an execution breakpoint for the current PC address, +* we should clear the task's RF so that the bp will be certain +* to trigger. +* +* FIXME: It's not so easy to get hold of the task's PC as a linear +* address! ptrace.c does this already... +*/ +} + +/* +* Unregister a user breakpoint structure. +*/ +void arch_unregister_user_hw_breakpoint (struct hw_breakpoint *bp, +struct thread_hw_breakpoint *thbi) +{ + thbi->tdr7 =calculate_dr7 (thbi); +} + +/* +* Register a kernel breakpoint structure. +*/ +void arch_register_kernel_hw_breakpoint (struct hw_breakpoint *bp) +{ + kdr7 =calculate_dr7 (NULL ); +} + +/* +* Unregister a kernel breakpoint structure. +*/ +void arch_unregister_kernel_hw_breakpoint (struct hw_breakpoint *bp) +{ + kdr7 =calculate_dr7 (NULL ); +} + + +/* End of arch-specific hook routines */ + + +/* +* Copy out the debug register information for a core dump. +* +* tsk must be equal to current. +*/ +void dump_thread_hw_breakpoint (struct task_struct *tsk,int u_debugreg[8 ]) +{ +struct thread_hw_breakpoint *thbi = tsk->thread.hw_breakpoint_info; +int i; + +memset (u_debugreg,0 ,sizeof u_debugreg); +if (thbi) { +for (i =0 ; i <HB_NUM ; ++i) + u_debugreg[i] = thbi->vdr_bps[i].info.address; + u_debugreg[7 ] = thbi->vdr7; + } + u_debugreg[6 ] = tsk->thread.vdr6; +} + +/* +* Handle debug exception notifications. +*/ + +int __kprobes hw_breakpoint_handler (struct die_args *args) +{ +struct cpu_hw_breakpoint *chbi; +int i; +struct hw_breakpoint *bp; +struct thread_hw_breakpoint *thbi =NULL ; + +/* The DR6 value is stored in args->err */ +#define DR6 (args->err) + +if (DR6 &DR_STEP ) +return NOTIFY_DONE ; + + chbi = &per_cpu (cpu_bp,get_cpu ()); + +/* Disable all breakpoints so that the callbacks can run without +* triggering recursive debug exceptions. +*/ +set_debugreg (0UL ,7 ); + +/* Assert that local interrupts are disabled +* Reset the DRn bits in the virtualized register value. +* The ptrace trigger routine will add in whatever is needed. +*/ + current->thread.vdr6 &= ~(DR_TRAP0 |DR_TRAP1 |DR_TRAP2 |DR_TRAP3 ); + +/* Are we a victim of lazy debug-register switching? */ +if (!chbi->bp_task) + ; +else if (chbi->bp_task != current) { + +/* No user breakpoints are valid. Perform the belated +* debug-register switch. +*/ +switch_to_none_hw_breakpoint (); + }else { + thbi = chbi->bp_task->thread.hw_breakpoint_info; + } + +/* Handle all the breakpoints that were triggered */ +for (i =0 ; i <HB_NUM ; ++i) { +if (likely (!(DR6 & (DR_TRAP0 << i)))) +continue ; + +/* Find the corresponding hw_breakpoint structure and +* invoke its triggered callback. +*/ +if (i < chbi->cur_kbpdata->num_kbps) + bp = chbi->cur_kbpdata->bps[i]; +else if (thbi) + bp = thbi->bps[i]; +else /* False alarm due to lazy DR switching */ +continue ; +if (bp) { +switch (bp->info.type) { +case HW_BREAKPOINT_WRITE : +case HW_BREAKPOINT_RW : +if (bp->triggered) + (bp->triggered)(bp, args->regs); +/* Re-enable the breakpoints */ +set_debugreg (thbi ? thbi->tkdr7 : + chbi->cur_kbpdata->mkdr7,7 ); +put_cpu_no_resched (); + +return NOTIFY_STOP ; +/* +* Presently we allow instruction breakpoints only in +* user-space when requested through ptrace. +*/ +case HW_BREAKPOINT_EXECUTE : +if (arch_check_va_in_userspace (bp->info.address, + current)) { + (bp->triggered)(bp, args->regs); +/* We'll return NOTIFY_DONE, do_debug will take care of the rest */ +return NOTIFY_DONE ; + } + } + } + } +/* Stop processing further if the exception is a stray one */ +if (!(DR6 & ~(DR_TRAP0 |DR_TRAP1 |DR_TRAP2 |DR_TRAP3 ))) +return NOTIFY_STOP ; + +return NOTIFY_DONE ; +#undef DR6 +}
arch/x86/include/asm/hw_breakpoint.h new file +132 -0
132 lines not on this page · raw message
arch/x86/kernel/Makefile +1 -1
@@ -36,7 +36,7 @@ obj-$(CONFIG_X86_64) += sys_x86_64.o x86 obj-$(CONFIG_X86_64) += syscall_64.o vsyscall_64.o obj-y += bootflag.o e820.o obj-y += pci-dma.o quirks.o i8237.o topology.o kdebugfs.o -obj-y += alternative.o i8253.o pci-nommu.o +obj-y += alternative.o i8253.o pci-nommu.o hw_breakpoint.o obj-y += tsc.o io_delay.o rtc.o obj-$(CONFIG_X86_TRAMPOLINE) += trampoline.o
PATCH 3/10 Modifying generic debug exception to use virtual debug registers
In reply to K.PrasadModifying generic debug exception to use virtual debug registers
From: Alan Stern <[email protected]>
This patch modifies the breakpoint exception handler code to use the abstract register names.
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/kernel/traps.c | 66 +++++++++++++++++------------------------------- 1 file changed, 24 insertions(+), 42 deletions(-)
arch/x86/kernel/traps.c +24 -42
@@ -570,10 +570,11 @@ asmlinkage __kprobes struct pt_regs *syn dotraplinkage void __kprobes do_debug(struct pt_regs *regs, long error_code) {struct task_struct *tsk = current; -unsigned long condition; +unsigned long dr6;int si_code; -get_debugreg (condition,6 ); +get_debugreg (dr6,6 ); +set_debugreg (0 ,6 );/* DR6 may or may not be cleared by the CPU */ /* Catch kmemcheck conditions first of all! */ if (condition &DR_STEP &&kmemcheck_trap (regs))
@@ -585,61 +586,42 @@ dotraplinkage void __kprobes do_debug(stclear_tsk_thread_flag (tsk,TIF_DEBUGCTLMSR ); tsk->thread.debugctlmsr =0 ; -if (notify_die (DIE_DEBUG ,"debug" , regs, condition, error_code, +/* Store the virtualized DR6 value */ + tsk->thread.vdr6 = dr6; + +if (notify_die (DIE_DEBUG ,"debug" , regs, dr6, error_code,SIGTRAP ) ==NOTIFY_STOP )return ;/* It's safe to allow irq's after DR6 has been saved */ preempt_conditional_sti (regs); -/* Mask out spurious debug traps due to lazy DR7 setting */ -if (condition & (DR_TRAP0 |DR_TRAP1 |DR_TRAP2 |DR_TRAP3 )) { -if (!tsk->thread.debugreg7) - goto clear_dr7; +if (regs->flags &X86_VM_MASK ) { +handle_vm86_trap ((struct kernel_vm86_regs *) regs, + error_code,1 ); +return ; }#ifdef CONFIG_X86_32 -if (regs->flags &X86_VM_MASK ) - goto debug_vm86;#endif -/* Save debug status register where ptrace can see it */ - tsk->thread.debugreg6 = condition; - -/* -* Single-stepping through TF: make sure we ignore any events in -* kernel space (but re-enable TF when returning to user mode). -*/ -if (condition &DR_STEP ) { -if (!user_mode (regs)) - goto clear_TF_reenable; - } - - si_code =get_si_code (condition); -/* Ok, finally something we can handle */ -send_sigtrap (tsk, regs, error_code, si_code); -/* -* Disable additional traps. They'll be re-enabled when -* the signal is delivered. +* Single-stepping through system calls: ignore any exceptions in +* kernel space, but re-enable TF when returning to user mode. +* +* We already checked v86 mode above, so we can check for kernel mode +* by just checking the CPL of CS. */ -clear_dr7: -set_debugreg (0 ,7 ); -preempt_conditional_cli (regs); -return ; - -#ifdef CONFIG_X86_32 -debug_vm86: -handle_vm86_trap ((struct kernel_vm86_regs *) regs, error_code,1 ); -preempt_conditional_cli (regs); -return ; -#endif +if ((dr6 &DR_STEP ) && !user_mode (regs)) { + tsk->thread.vdr6 &= ~DR_STEP ; +set_tsk_thread_flag (tsk,TIF_SINGLESTEP ); + regs->flags &= ~X86_EFLAGS_TF ; + } + si_code =get_si_code (dr6); +if (tsk->thread.vdr6 & (DR_STEP |DR_TRAP0 |DR_TRAP1 |DR_TRAP2 |DR_TRAP3 )) +send_sigtrap (tsk, regs, error_code, si_code); -clear_TF_reenable: -set_tsk_thread_flag (tsk,TIF_SINGLESTEP ); - regs->flags &= ~X86_EFLAGS_TF ;preempt_conditional_cli (regs); -return ; }#ifdef CONFIG_X86_64
PATCH 4/10 Introduce virtual debug register in thread_struct and wrapper-routines around process related functions
In reply to K.PrasadIntroduce virtual debug register in thread_struct and wrapper-routines around process related functions
From: Alan Stern <[email protected]>
This patch introduces virtual debug registers to used by the per-thread structure ad wrapper routines to manage debug registers by process-related functions.
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/include/asm/debugreg.h | 27 +++++++++++++++++++++++++++ arch/x86/include/asm/processor.h | 10 +++------- 2 files changed, 30 insertions(+), 7 deletions(-)
arch/x86/include/asm/debugreg.h +27 -0
@@ -49,6 +49,8 @@#define DR_LOCAL_ENABLE_SHIFT 0/* Extra shift to the local enable bit */ #define DR_GLOBAL_ENABLE_SHIFT 1/* Extra shift to the global enable bit */ +#define DR_LOCAL_ENABLE (0x1)/* Local enable for reg 0 */ +#define DR_GLOBAL_ENABLE (0x2)/* Global enable for reg 0 */ #define DR_ENABLE_SIZE 2/* 2 enable bits per register */ #define DR_LOCAL_ENABLE_MASK (0x55)/* Set local bits for all 4 regs */
@@ -67,4 +69,29 @@#define DR_LOCAL_SLOWDOWN (0x100)/* Local slow the pipeline */ #define DR_GLOBAL_SLOWDOWN (0x200)/* Global slow the pipeline */ +/* +* HW breakpoint additions +*/ +#ifdef __KERNEL__ + +/* For process management */ +void flush_thread_hw_breakpoint (struct task_struct *tsk); +int copy_thread_hw_breakpoint (struct task_struct *tsk, +struct task_struct *child,unsigned long clone_flags); +void dump_thread_hw_breakpoint (struct task_struct *tsk,int u_debugreg[8 ]); +void switch_to_thread_hw_breakpoint (struct task_struct *tsk); + +/* For CPU management */ +void load_debug_registers (void ); +static inline void disable_debug_registers (void ) +{ +set_debugreg (0UL ,7 ); +} + +/* For use by ptrace */ +unsigned long thread_get_debugreg (struct task_struct *tsk,int n); +int thread_set_debugreg (struct task_struct *tsk,int n,unsigned long val); + +#endif /* __KERNEL__ */ +#endif /* _ASM_X86_DEBUGREG_H */
arch/x86/include/asm/processor.h +3 -7
@@ -427,13 +427,9 @@ struct thread_struct {unsigned long ip;unsigned long fs;unsigned long gs; -/* Hardware debugging registers: */ -unsigned long debugreg0; -unsigned long debugreg1; -unsigned long debugreg2; -unsigned long debugreg3; -unsigned long debugreg6; -unsigned long debugreg7; +/* Hardware breakpoint info */ +unsigned long vdr6; +struct thread_hw_breakpoint *hw_breakpoint_info;/* Fault info: */ unsigned long cr2;unsigned long trap_no;
PATCH 5/10 Use wrapper routines around debug registers in processor related functions
In reply to K.PrasadUse wrapper routines around debug registers in processor related functions
From: Alan Stern <[email protected]>
This patch enables the use of wrapper routines to access the debug/breakpoint registers.
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/kernel/smpboot.c | 3 +++ arch/x86/power/cpu_32.c | 16 +++------------- arch/x86/power/cpu_64.c | 15 +++------------ 3 files changed, 9 insertions(+), 25 deletions(-)
arch/x86/power/cpu_32.c +3 -13
@@ -12,6 +12,7 @@#include <asm/mtrr.h> #include <asm/mce.h> #include <asm/xcr.h> +#include <asm/debugreg.h> static struct saved_context saved_context;
@@ -47,6 +48,7 @@ static void __save_processor_state(struc ctxt->cr2 =read_cr2 (); ctxt->cr3 =read_cr3 (); ctxt->cr4 =read_cr4_safe (); +disable_debug_registers (); }/* Needed by apm.c */
@@ -79,19 +81,7 @@ static void fix_processor_context(void)load_TR_desc ();/* This does ltr */ load_LDT (¤t->active_mm->context);/* This does lldt */ -/* -* Now maybe reload the debug registers -*/ -if (current->thread.debugreg7) { -set_debugreg (current->thread.debugreg0,0 ); -set_debugreg (current->thread.debugreg1,1 ); -set_debugreg (current->thread.debugreg2,2 ); -set_debugreg (current->thread.debugreg3,3 ); -/* no 4 and 5 */ -set_debugreg (current->thread.debugreg6,6 ); -set_debugreg (current->thread.debugreg7,7 ); - } - +load_debug_registers (); }static void __restore_processor_state (struct saved_context *ctxt)
arch/x86/power/cpu_64.c +3 -12
@@ -15,6 +15,7 @@#include <asm/pgtable.h> #include <asm/mtrr.h> #include <asm/xcr.h> +#include <asm/debugreg.h> static void fix_processor_context (void );
@@ -70,6 +71,7 @@ static void __save_processor_state(struc ctxt->cr3 =read_cr3 (); ctxt->cr4 =read_cr4 (); ctxt->cr8 =read_cr8 (); +disable_debug_registers (); }void save_processor_state (void )
@@ -158,16 +160,5 @@ static void fix_processor_context(void)load_TR_desc ();/* This does ltr */ load_LDT (¤t->active_mm->context);/* This does lldt */ -/* -* Now maybe reload the debug registers -*/ -if (current->thread.debugreg7){ -loaddebug (¤t->thread,0 ); -loaddebug (¤t->thread,1 ); -loaddebug (¤t->thread,2 ); -loaddebug (¤t->thread,3 ); -/* no 4 and 5 */ -loaddebug (¤t->thread,6 ); -loaddebug (¤t->thread,7 ); - } +load_debug_registers (); }
arch/x86/kernel/smpboot.c +3 -0
@@ -63,6 +63,7 @@#include <asm/genapic.h> #include <asm/setup.h> #include <asm/uv/uv.h> +#include <asm/debugreg.h> #include <linux/mc146818rtc.h> #include <asm/genapic.h>
@@ -338,6 +339,7 @@ notrace static void __cpuinit start_secosetup_secondary_clock ();wmb (); +load_debug_registers ();cpu_idle (); }
@@ -1318,6 +1320,7 @@ void cpu_disable_common(void)remove_cpu_from_maps (cpu);unlock_vector_lock ();fixup_irqs (); +disable_debug_registers (); }int native_cpu_disable (void )
PATCH 6/10 Use virtual debug registers in process/thread handling code
In reply to K.PrasadUse virtual debug registers in process/thread handling code
From: Alan Stern <[email protected]>
This patch enables the use of abstract/virtual debug registers in process-handling routines.
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/kernel/process_32.c | 43 +++++++++++++++++++++++++------------------ arch/x86/kernel/process_64.c | 41 ++++++++++++++++++++++++----------------- 2 files changed, 49 insertions(+), 35 deletions(-)
arch/x86/kernel/process_32.c +25 -18
@@ -61,6 +61,8 @@#include <asm/idle.h> #include <asm/syscalls.h> #include <asm/ds.h> +#include <asm/debugreg.h> +#include <asm/hw_breakpoint.h> asmlinkage void ret_from_fork(void) __asm__("ret_from_fork");
@@ -238,6 +240,8 @@ EXPORT_SYMBOL(kernel_thread);*/ void exit_thread (void ) { +struct task_struct *tsk = current; +/* The process may have allocated an io port bitmap... nuke it. */ if (unlikely (test_thread_flag (TIF_IO_BITMAP ))) {struct task_struct *tsk = current;
@@ -258,6 +262,8 @@ void exit_thread(void) tss->x86_tss.io_bitmap_base =INVALID_IO_BITMAP_OFFSET ;put_cpu (); } +if (unlikely (tsk->thread.hw_breakpoint_info)) +flush_thread_hw_breakpoint (tsk);ds_exit_thread (current); }
@@ -266,14 +272,9 @@ void flush_thread(void) {struct task_struct *tsk = current; - tsk->thread.debugreg0 =0 ; - tsk->thread.debugreg1 =0 ; - tsk->thread.debugreg2 =0 ; - tsk->thread.debugreg3 =0 ; - tsk->thread.debugreg6 =0 ; - tsk->thread.debugreg7 =0 ;memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array)); -clear_tsk_thread_flag (tsk,TIF_DEBUG ); +if (unlikely (tsk->thread.hw_breakpoint_info)) +flush_thread_hw_breakpoint (tsk);/* * Forget coprocessor state.. */
@@ -317,7 +318,15 @@ int copy_thread(int nr, unsigned long cltask_user_gs (p) =get_user_gs (regs); + p->thread.hw_breakpoint_info =NULL ; + p->thread.io_bitmap_ptr =NULL ; + tsk = current; + err = -ENOMEM ; +if (unlikely (tsk->thread.hw_breakpoint_info)) { +if (copy_thread_hw_breakpoint (tsk, p, clone_flags)) + goto out; + }if (unlikely (test_tsk_thread_flag (tsk,TIF_IO_BITMAP ))) { p->thread.io_bitmap_ptr =kmemdup (tsk->thread.io_bitmap_ptr,IO_BITMAP_BYTES ,GFP_KERNEL );
@@ -336,11 +345,13 @@ int copy_thread(int nr, unsigned long clif (clone_flags &CLONE_SETTLS ) err =do_set_thread_area (p,-1 , (struct user_desc __user *)childregs->si,0 ); - + out:if (err && p->thread.io_bitmap_ptr) {kfree (p->thread.io_bitmap_ptr); p->thread.io_bitmap_max =0 ; } +if (err) +flush_thread_hw_breakpoint (p);ds_copy_thread (p, current);
@@ -442,16 +453,6 @@ __switch_to_xtra(struct task_struct *preelse if (next->debugctlmsr != prev->debugctlmsr)update_debugctlmsr (next->debugctlmsr); -if (test_tsk_thread_flag (next_p,TIF_DEBUG )) { -set_debugreg (next->debugreg0,0 ); -set_debugreg (next->debugreg1,1 ); -set_debugreg (next->debugreg2,2 ); -set_debugreg (next->debugreg3,3 ); -/* no 4 and 5 */ -set_debugreg (next->debugreg6,6 ); -set_debugreg (next->debugreg7,7 ); - } -if (test_tsk_thread_flag (prev_p,TIF_NOTSC ) ^test_tsk_thread_flag (next_p,TIF_NOTSC )) {/* prev and next are different */
@@ -600,6 +601,12 @@ __switch_to(struct task_struct *prev_p,percpu_write (current_task, next_p); +/* +* Handle debug registers. This must be done _after_ current +* is updated. +*/ +if (unlikely (test_tsk_thread_flag (next_p,TIF_DEBUG ))) +switch_to_thread_hw_breakpoint (next_p);return prev_p; }
arch/x86/kernel/process_64.c +24 -17
90 lines not on this page · raw message
PATCH 7/10 Modify signal handling code to refrain from re-enabling HW Breakpoints
In reply to K.PrasadModify signal handling code to refrain from re-enabling HW Breakpoints
From: Alan Stern <[email protected]>
This patch disables re-enabling of Hardware Breakpoint registers through the signal handling code. This is now done during hw_breakpoint_handler().
--- arch/x86/kernel/signal.c | 9 --------- 1 file changed, 9 deletions(-)
arch/x86/kernel/signal.c +0 -9
@@ -803,15 +803,6 @@ static void do_signal(struct pt_regs *re signr =get_signal_to_deliver (&info, &ka, regs,NULL );if (signr >0 ) { -/* -* Re-enable any watchpoints before delivering the -* signal to user space. The processor register will -* have been cleared if the watchpoint triggered -* inside the kernel. -*/ -if (current->thread.debugreg7) -set_debugreg (current->thread.debugreg7,7 ); -/* Whee! Actually deliver the signal. */ if (handle_signal (signr, &info, &ka, oldset, regs) ==0 ) {/*
PATCH 8/10 Modify Ptrace routines to access breakpoint registers
In reply to K.PrasadModify Ptrace routines to access breakpoint registers
From: Alan Stern <[email protected]>
This patch modifies the ptrace code to use the new wrapper routines around the debug/breakpoint registers.
[K.Prasad: Adapted the ptrace routines and to changes post x86/x86_64 merger]
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/kernel/ptrace.c | 242 +++++++++++++++++++++++++++++------------------ 1 file changed, 152 insertions(+), 90 deletions(-)
arch/x86/kernel/ptrace.c +152 -90
@@ -33,6 +33,7 @@#include <asm/prctl.h> #include <asm/proto.h> #include <asm/ds.h> +#include <asm/hw_breakpoint.h> #include "tls.h"
@@ -133,11 +134,6 @@ static int set_segment_reg(struct task_s return 0; } -static unsigned long debugreg_addr_limit (struct task_struct *task) -{ -return TASK_SIZE -3 ; -} -#else /* CONFIG_X86_64 */ #define FLAG_MASK (FLAG_MASK_32 | X86_EFLAGS_NT)
@@ -262,15 +258,6 @@ static int set_segment_reg(struct task_s return 0; } -static unsigned long debugreg_addr_limit (struct task_struct *task) -{ -#ifdef CONFIG_IA32_EMULATION -if (test_tsk_thread_flag (task,TIF_IA32 )) -return IA32_PAGE_OFFSET -3 ; -#endif -return TASK_SIZE64 -7 ; -} -#endif /* CONFIG_X86_32 */ static unsigned long get_flags (struct task_struct *task)
@@ -461,95 +448,170 @@ static int genregs_set(struct task_struc }/* -* This function is trivial and will be inlined by the compiler. -* Having it separates the implementation details of debug -* registers from the interface details of ptrace. +* Decode the length and type bits for a particular breakpoint as +* stored in debug register 7. Return the "enabled" status. */ -static unsigned long ptrace_get_debugreg (struct task_struct *child,int n) +static int decode_dr7 (unsigned long dr7,int bpnum,unsigned *len, +unsigned *type) { -switch (n) { -case 0 :return child->thread.debugreg0; -case 1 :return child->thread.debugreg1; -case 2 :return child->thread.debugreg2; -case 3 :return child->thread.debugreg3; -case 6 :return child->thread.debugreg6; -case 7 :return child->thread.debugreg7; +int temp = dr7 >> (DR_CONTROL_SHIFT + bpnum *DR_CONTROL_SIZE ); + + *len = (temp &0xc ) |0x40 ; + *type = (temp &0x3 ) |0x80 ; +return (dr7 >> (bpnum *DR_ENABLE_SIZE )) &0x3 ; +} + +static void ptrace_triggered (struct hw_breakpoint *bp,struct pt_regs *regs) +{ +struct task_struct *tsk = current; +struct thread_hw_breakpoint *thbi = tsk->thread.hw_breakpoint_info; +int i; + +/* Store in the virtual DR6 register the fact that the breakpoint +* was hit so the thread's debugger will see it. +*/ +if (thbi) { + i = bp - thbi->vdr_bps; + tsk->thread.vdr6 |= (DR_TRAP0 << i); } -return 0 ; } -static int ptrace_set_debugreg (struct task_struct *child, -int n,unsigned long data) +/* +* Handle ptrace writes to debug register 7. +*/ +static int ptrace_write_dr7 (struct task_struct *tsk, +struct thread_hw_breakpoint *thbi,unsigned long data) { +struct hw_breakpoint *bp;int i; +int rc =0 ; +unsigned long old_dr7 = thbi->vdr7; -if (unlikely (n ==4 || n ==5 )) -return -EIO ; + data &= ~DR_CONTROL_RESERVED ; -if (n <4 &&unlikely (data >=debugreg_addr_limit (child))) -return -EIO ; +/* Loop through all the hardware breakpoints, making the +* appropriate changes to each. +*/ + restore_settings: + thbi->vdr7 = data; + bp = &thbi->vdr_bps[0 ]; +for (i =0 ; i <HB_NUM ; (++i, ++bp)) { +int enabled; +unsigned len, type; + + enabled =decode_dr7 (data, i, &len, &type); + +/* Unregister the breakpoint before trying to change it */ +if (bp->status) +__unregister_user_hw_breakpoint (tsk, bp); -switch (n) { -case 0 : child->thread.debugreg0 = data;break ; -case 1 : child->thread.debugreg1 = data;break ; -case 2 : child->thread.debugreg2 = data;break ; -case 3 : child->thread.debugreg3 = data;break ; +/* Now register the breakpoint if it should be enabled. +* New invalid entries will raise an error here. +*/ +if (enabled) { + bp->triggered = ptrace_triggered; + bp->info.len = len; + bp->info.type = type; + + bp->priority =HW_BREAKPOINT_PRIO_PTRACE ; +if (rc ==0 &&__register_user_hw_breakpoint (tsk, + bp) <0 ) +break ; + } + } -case 6 : -if ((data & ~0xffffffffUL ) !=0 ) -return -EIO ; - child->thread.debugreg6 = data; -break ; +/* If anything above failed, restore the original settings */ +if (i <HB_NUM ) { + rc = -EIO ; + data = old_dr7; + goto restore_settings; + } +return rc; +} -case 7 : -/* -* Sanity-check data. Take one half-byte at once with -* check = (val >> (16 + 4*i)) & 0xf. It contains the -* R/Wi and LENi bits; bits 0 and 1 are R/Wi, and bits -* 2 and 3 are LENi. Given a list of invalid values, -* we do mask |= 1 << invalid_value, so that -* (mask >> check) & 1 is a correct test for invalid -* values. -* -* R/Wi contains the type of the breakpoint / -* watchpoint, LENi contains the length of the watched -* data in the watchpoint case. -* -* The invalid values are: -* - LENi == 0x10 (undefined), so mask |= 0x0f00. [32-bit] -* - R/Wi == 0x10 (break on I/O reads or writes), so -* mask |= 0x4444. -* - R/Wi == 0x00 && LENi != 0x00, so we have mask |= -* 0x1110. -* -* Finally, mask = 0x0f00 | 0x4444 | 0x1110 == 0x5f54. -* -* See the Intel Manual "System Programming Guide", -* 15.2.4 -* -* Note that LENi == 0x10 is defined on x86_64 in long -* mode (i.e. even for 32-bit userspace software, but -* 64-bit kernel), so the x86_64 mask value is 0x5454. -* See the AMD manual no. 24593 (AMD64 System Programming) +/* +* Handle PTRACE_PEEKUSR calls for the debug register area. +*/ +unsigned long ptrace_get_debugreg (struct task_struct *tsk,int n) +{ +struct thread_hw_breakpoint *thbi; +unsigned long val =0 ; + +mutex_lock (&hw_breakpoint_mutex); + thbi = tsk->thread.hw_breakpoint_info; +if (n <HB_NUM ) { +if (thbi) + val = thbi->vdr_bps[n].info.address; + }else if (n ==6 ) { + val = tsk->thread.vdr6; + }else if (n ==7 ) { +if (thbi) + val = thbi->vdr7; + } +mutex_unlock (&hw_breakpoint_mutex); +return val; +} + +/* +* Handle PTRACE_POKEUSR calls for the debug register area. +*/ +int ptrace_set_debugreg (struct task_struct *tsk,int n,unsigned long val) +{ +struct thread_hw_breakpoint *thbi; +int rc = -EIO ; + +/* We have to hold this lock the entire time, to prevent thbi +* from being deallocated out from under us. +*/ +mutex_lock (&hw_breakpoint_mutex); + +/* There are no DR4 or DR5 registers */ +if (n ==4 || n ==5 ) + ; + +/* Writes to DR6 modify the virtualized value */ +else if (n ==6 ) { + tsk->thread.vdr6 = val; + rc =0 ; + } + +else if (!tsk->thread.hw_breakpoint_info && val ==0 ) + rc =0 ;/* Minor optimization */ + +else if ((thbi =alloc_thread_hw_breakpoint (tsk)) ==NULL ) + rc = -ENOMEM ; + +/* Writes to DR0 - DR3 change a breakpoint address */ +else if (n <HB_NUM ) { +struct hw_breakpoint *bp = &thbi->vdr_bps[n]; + +/* If the breakpoint is registered then unregister it, +* change it, and re-register it. Revert to the original +* address if an error occurs. */ -#ifdef CONFIG_X86_32 -#define DR7_MASK 0x5f54 -#else -#define DR7_MASK 0x5554 -#endif - data &= ~DR_CONTROL_RESERVED ; -for (i =0 ; i <4 ; i++) -if ((DR7_MASK >> ((data >> (16 +4 *i)) &0xf )) &1 ) -return -EIO ; - child->thread.debugreg7 = data; -if (data) -set_tsk_thread_flag (child,TIF_DEBUG ); -else -clear_tsk_thread_flag (child,TIF_DEBUG ); -break ; +if (bp->status) { +unsigned long old_addr = bp->info.address; + +__unregister_user_hw_breakpoint (tsk, bp); + + bp->info.address = val; + rc =__register_user_hw_breakpoint (tsk, bp); +if (rc <0 ) { + bp->info.address = old_addr; +__register_user_hw_breakpoint (tsk, bp); + } + }else { + bp->info.address = val; + rc =0 ; + } } -return 0 ; +/* All that's left is DR7 */ +else + rc =ptrace_write_dr7 (tsk, thbi, val); + +mutex_unlock (&hw_breakpoint_mutex); +return rc; }/*
@@ -871,7 +933,7 @@ long arch_ptrace(struct task_struct *chi else if (addr >= offsetof(struct user, u_debugreg[0]) && addr <= offsetof(struct user, u_debugreg[7])) { addr -= offsetof(struct user, u_debugreg[0]); - tmp =ptrace_get_debugreg (child, addr /sizeof (data)); + tmp =ptrace_get_debugreg (child, addr/sizeof (data)); } ret =put_user (tmp, datap);break ;
@@ -889,7 +951,7 @@ long arch_ptrace(struct task_struct *chi addr <= offsetof(struct user, u_debugreg[7])) { addr -= offsetof(struct user, u_debugreg[0]); ret =ptrace_set_debugreg (child, - addr /sizeof (data), data); + addr/sizeof (data), data); }break ;
PATCH 9/10 Cleanup HW Breakpoint registers before kexec
In reply to K.PrasadCleanup HW Breakpoint registers before kexec
From: Alan Stern <[email protected]>
This patch disables Hardware breakpoints before doing a 'kexec' on the machine.
[K.Prasad: Split-out from the bigger patch and minor changes following
re-basing]
--- arch/x86/kernel/machine_kexec_32.c | 2 ++ arch/x86/kernel/machine_kexec_64.c | 2 ++ 2 files changed, 4 insertions(+)
arch/x86/kernel/machine_kexec_32.c +2 -0
@@ -25,6 +25,7 @@#include <asm/desc.h> #include <asm/system.h> #include <asm/cacheflush.h> +#include <asm/debugreg.h> static void set_idt (void *newidt,__u16 limit) {
@@ -202,6 +203,7 @@ void machine_kexec(struct kimage *image)/* Interrupts aren't acceptable while we reboot */ local_irq_disable (); +disable_debug_registers ();if (image->preserve_context) {#ifdef CONFIG_X86_IO_APIC
arch/x86/kernel/machine_kexec_64.c +2 -0
@@ -17,6 +17,7 @@#include <asm/tlbflush.h> #include <asm/mmu_context.h> #include <asm/io.h> +#include <asm/debugreg.h> static void init_level2_page (pmd_t *level2p,unsigned long addr) {
@@ -234,6 +235,7 @@ void machine_kexec(struct kimage *image)/* Interrupts aren't acceptable while we reboot */ local_irq_disable (); +disable_debug_registers (); control_page =page_address (image->control_code_page) +PAGE_SIZE ;memcpy (control_page, relocate_kernel,PAGE_SIZE );
PATCH 10/10 Sample HW breakpoint over kernel data address
In reply to K.PrasadSample HW breakpoint over kernel data address
This patch introduces a sample kernel module to demonstrate the use of Hardware Breakpoint feature. It places a breakpoint over the kernel variable 'pid_max' to monitor all write operations and emits a function-backtrace when done.
--- samples/Kconfig | 6 ++ samples/Makefile | 3 - samples/hw_breakpoint/Makefile | 1 samples/hw_breakpoint/data_breakpoint.c | 80 ++++++++++++++++++++++++++++++++ 4 files changed, 89 insertions(+), 1 deletion(-)
samples/Kconfig +6 -0
@@ -39,5 +39,11 @@ config SAMPLE_KRETPROBES default m depends on SAMPLE_KPROBES && KRETPROBES +config SAMPLE_HW_BREAKPOINT + tristate "Build kernel hardware breakpoint examples -- loadable modules only" + depends on m + help + This builds kernel hardware breakpoint example modules. + endif # SAMPLES
samples/Makefile +2 -1
@@ -1,3 +1,4 @@ # Makefile for Linux samples code -obj-$(CONFIG_SAMPLES) += markers/ kobject/ kprobes/ tracepoints/ +obj-$(CONFIG_SAMPLES) += markers/ kobject/ kprobes/ tracepoints/ \ + hw_breakpoint/
samples/hw_breakpoint/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-$(CONFIG_SAMPLE_HW_BREAKPOINT) += data_breakpoint.o
samples/hw_breakpoint/data_breakpoint.c new file +80 -0
80 lines not on this page · raw message