Fix 2.6.33 x86 regression to kgdb hw breakpoints - due to perf API changes
RFC 1/1 Fix 2.6.33 x86 regression to kgdb hw breakpoints - due to perf API changes
I'll lead with a question. Are the people making the Perf API changes booting allyesconfig kernels?
The regression tests built into the kernel for kgdb fail as a result of the perf API changes and can result in a hard kernel hang. My hope would have been that someone would have reported the problem, created a patch to disable the test that hangs the kernel, or to fix kgdb such that it works with the API changes. Likewise, if there are tests I should run to regression test the changes to the perf API, I would like to know since we all want to make use of the same hw resource.
A patch has been included in this mail which allows kgdb to pass the internal regression tests for hw breakpoints. I would like to get some comments or start a discussion as to how to solve this problem in the 2.6.33 cycle, as it is a regression in functionality.
I did not address this in the patch, but I found that the hw_breakpoint.c API is lacking a function to query if there is a breakpoint slot available on every processor, while atomic. Having that would allow an end user to see an error generated up to gdb that there are not enough breakpoints available, as opposed to finding out about it via a printk message after you resume the system.
Thanks,
Jason.
From: Jason Wessel <[email protected]>
Subject: [PATCH] x86,hw_breakpoint,kgdb: kgdb to use hw_breakpoint API
In the 2.6.33 kernel, the hw_breakpoint API is now used for the performance event counters. The hw_breakpoint_handler() now consumes the hw breakpoints that were previously set by kgdb arch specific code. In order for kgdb to work in conjunction with this core API change, kgdb must use some of the low level functions of the hw_breakpoint API to install, uninstall, and receive call backs for hw breakpoints.
The kgdb core needs to call kgdb_disable_hw_debug anytime a slave cpu enters kgdb_wait() in order to keep all the hw breakpoints in sync as well as to prevent hitting a hw breakpoint while kgdb is active.
During the architecture specific initialization of kgdb, it will pre-allocate 4 disabled (struct perf event **) structures. Kgdb will use these to manage the capabilities for the 4 hw breakpoint registers. Right now the hw_breakpoint API does not have a way to ask how many breakpoints are available, on each CPU so it is possible that the install of a breakpoint might fail when kgdb restores the system to the run state. The intent of this patch is to first get the basic functionality of hw breakpoints working and leave it to the person debugging the kernel to understand what hw breakpoints are in use and what restrictions have been imposed as a result.
While atomic, the x86 specific kgdb code will call arch_uninstall_hw_breakpoint() and arch_install_hw_breakpoint() to manage the cpu specific hw breakpoints.
The arch specific hw_breakpoint_handler() was changed to restore the cpu specific dr7 instead of the dr7 that was locally saved, because the dr7 can be modified while in a call back to kgdb.
The net result of these changes allow kgdb to use the same pool of hw_breakpoints that are used by the perf event API, but neither knows about future reservations for the available hw breakpoint slots.
--- arch/x86/kernel/hw_breakpoint.c | 5 - arch/x86/kernel/kgdb.c | 179 +++++++++++++++++++++++++++------------- kernel/kgdb.c | 3 3 files changed, 127 insertions(+), 60 deletions(-)
arch/x86/kernel/kgdb.c +122 -57
@@ -42,6 +42,7 @@#include <linux/init.h> #include <linux/smp.h> #include <linux/nmi.h> +#include <linux/hw_breakpoint.h> #include <asm/debugreg.h> #include <asm/apicdef.h>
@@ -204,40 +205,27 @@ void gdb_regs_to_pt_regs(unsigned long *static struct hw_breakpoint {unsigned enabled; -unsigned type; -unsigned len;unsigned long addr; +struct perf_event **pev; } breakinfo[4 ];static void kgdb_correct_hw_break (void ) { -unsigned long dr7; -int correctit =0 ; -int breakbit;int breakno; -get_debugreg (dr7,7 );for (breakno =0 ; breakno <4 ; breakno++) { - breakbit =2 << (breakno <<1 ); -if (!(dr7 & breakbit) && breakinfo[breakno].enabled) { - correctit =1 ; - dr7 |= breakbit; - dr7 &= ~(0xf0000 << (breakno <<2 )); - dr7 |= ((breakinfo[breakno].len <<2 ) | - breakinfo[breakno].type) << - ((breakno <<2 ) +16 ); -set_debugreg (breakinfo[breakno].addr, breakno); - - }else { -if ((dr7 & breakbit) && !breakinfo[breakno].enabled) { - correctit =1 ; - dr7 &= ~breakbit; - dr7 &= ~(0xf0000 << (breakno <<2 )); - } - } +struct perf_event *bp; +int val; +int cpu =raw_smp_processor_id (); +if (!breakinfo[breakno].enabled) +continue ; + bp = *per_cpu_ptr (breakinfo[breakno].pev, cpu); +if (bp->attr.disabled !=1 ) +continue ; + val =arch_install_hw_breakpoint (bp); +if (!val) + bp->attr.disabled =0 ; } -if (correctit) -set_debugreg (dr7,7 ); } static int
@@ -259,46 +247,74 @@ kgdb_remove_hw_break(unsigned long addr,static void kgdb_remove_all_hw_break (void ) {int i; +int cpu =raw_smp_processor_id (); +struct perf_event *bp; -for (i =0 ; i <4 ; i++) -memset (&breakinfo[i],0 ,sizeof (struct hw_breakpoint )); +for (i =0 ; i <4 ; i++) { +if (!breakinfo[i].enabled) +continue ; + bp = *per_cpu_ptr (breakinfo[i].pev, cpu); +if (bp->attr.disabled ==1 ) +continue ; +arch_uninstall_hw_breakpoint (bp); + bp->attr.disabled =1 ; + } }static int kgdb_set_hw_break (unsigned long addr,int len,enum kgdb_bptype bptype) { -unsigned type;int i; +struct perf_event *bp; +struct arch_hw_breakpoint *info;for (i =0 ; i <4 ; i++)if (!breakinfo[i].enabled)break ;if (i ==4 )return -1 ; - + bp = *per_cpu_ptr (breakinfo[i].pev,raw_smp_processor_id ()); + info =counter_arch_bp (bp);switch (bptype) {case BP_HARDWARE_BREAKPOINT : - type =0 ; - len =1 ; + len =1 ; + info->type =X86_BREAKPOINT_EXECUTE ;break ;case BP_WRITE_WATCHPOINT : - type =1 ; + info->type =X86_BREAKPOINT_WRITE ;break ;case BP_ACCESS_WATCHPOINT : - type =3 ; + info->type =X86_BREAKPOINT_RW ;break ;default :return -1 ; } -if (len ==1 || len ==2 || len ==4 ) - breakinfo[i].len = len -1 ; -else +switch (len) { +case 1 : + info->len =X86_BREAKPOINT_LEN_1 ; +break ; +case 2 : + info->len =X86_BREAKPOINT_LEN_2 ; +break ; +case 4 : + info->len =X86_BREAKPOINT_LEN_4 ; +break ; +#ifdef CONFIG_X86_64 +case 8 : + info->len =X86_BREAKPOINT_LEN_8 ; +break ; +#endif +default :return -1 ; + } - breakinfo[i].enabled =1 ; breakinfo[i].addr = addr; - breakinfo[i].type = type; + info->address = addr; + bp->attr.bp_addr = info->address; + bp->attr.bp_len = info->len; + bp->attr.bp_type = info->type; + breakinfo[i].enabled =1 ;return 0 ; }
@@ -313,8 +329,21 @@ kgdb_set_hw_break(unsigned long addr, in*/ void kgdb_disable_hw_debug (struct pt_regs *regs) { +int i; +int cpu =raw_smp_processor_id (); +struct perf_event *bp; +/* Disable hardware debugging while we are in kgdb: */ set_debugreg (0UL ,7 ); +for (i =0 ; i <4 ; i++) { +if (!breakinfo[i].enabled) +continue ; + bp = *per_cpu_ptr (breakinfo[i].pev, cpu); +if (bp->attr.disabled ==1 ) +continue ; +arch_uninstall_hw_breakpoint (bp); + bp->attr.disabled =1 ; + } }/**
@@ -378,7 +407,6 @@ int kgdb_arch_handle_exception(int e_vec struct pt_regs *linux_regs) {unsigned long addr; -unsigned long dr6;char *ptr;int newPC;
@@ -404,20 +432,6 @@ int kgdb_arch_handle_exception(int e_vec raw_smp_processor_id()); } -get_debugreg (dr6,6 ); -if (!(dr6 &0x4000 )) { -int breakno; - -for (breakno =0 ; breakno <4 ; breakno++) { -if (dr6 & (1 << breakno) && - breakinfo[breakno].type ==0 ) { -/* Set restore flag: */ - linux_regs->flags |=X86_EFLAGS_RF ; -break ; - } - } - } -set_debugreg (0UL ,6 );kgdb_correct_hw_break ();return 0 ;
@@ -448,6 +462,7 @@ single_step_cont(struct pt_regs *regs, s }static int was_in_debug_nmi[NR_CPUS ]; +static int recieved_hw_brk[NR_CPUS ];static int __kgdb_notify (struct die_args *args,unsigned long cmd) {
@@ -485,16 +500,19 @@ static int __kgdb_notify(struct die_argsbreak ;case DIE_DEBUG : -if (atomic_read (&kgdb_cpu_doing_single_step) == -raw_smp_processor_id ()) { +if (atomic_read (&kgdb_cpu_doing_single_step) !=-1 ) {if (user_mode (regs))return single_step_cont (regs, args);break ; - }else if (test_thread_flag (TIF_SINGLESTEP )) + }else if (test_thread_flag (TIF_SINGLESTEP )) {/* This means a user thread is single stepping * a system call which should be ignored */ return NOTIFY_DONE ; + }else if (recieved_hw_brk[raw_smp_processor_id ()] ==1 ) { + recieved_hw_brk[raw_smp_processor_id ()] =0 ; +return NOTIFY_STOP ; + }/* fall through */ default :if (user_mode (regs))
@@ -531,6 +549,21 @@ static struct notifier_block kgdb_notifi .priority = -INT_MAX, }; +static void kgdb_hw_bp (struct perf_event *bp,void *data) +{ +struct die_args args; + + args.trapnr =0 ; + args.signr =5 ; + args.err =0 ; + args.regs = data; + args.str ="debug" ; +if (__kgdb_notify (&args,DIE_DEBUG ) ==NOTIFY_STOP ) + recieved_hw_brk[raw_smp_processor_id ()] =1 ; +else + recieved_hw_brk[raw_smp_processor_id ()] =0 ; +} +/** * kgdb_arch_init - Perform any architecture specific initalization. *
@@ -539,7 +572,32 @@ static struct notifier_block kgdb_notifi*/ int kgdb_arch_init (void ) { -return register_die_notifier (&kgdb_notifier); +int i; +int ret; +DEFINE_BREAKPOINT_ATTR (attr); + + ret =register_die_notifier (&kgdb_notifier); +if (ret !=0 ) +return ret; +/* Pre-allocate the hw breakpoint structions in the non-atomic +* portion of kgdb because this operation requires mutexs to +* complete. */ + attr.bp_addr = (unsigned long )kgdb_arch_init; + attr.type =1 ; + attr.bp_len =HW_BREAKPOINT_LEN_1 ; + attr.bp_type =HW_BREAKPOINT_X ; + attr.disabled =1 ; +for (i =0 ; i <4 ; i++) { + breakinfo[i].pev =register_wide_hw_breakpoint (&attr, + kgdb_hw_bp); +if (IS_ERR (breakinfo[i].pev)) { +printk (KERN_ERR "kgdb: Could not allocate hw breakpoints\n" ); + breakinfo[i].pev =NULL ; +kgdb_arch_exit (); +return -1 ; + } + } +return ret; }/**
@@ -550,6 +608,13 @@ int kgdb_arch_init(void)*/ void kgdb_arch_exit (void ) { +int i; +for (i =0 ; i <4 ; i++) { +if (breakinfo[i].pev) { +unregister_wide_hw_breakpoint (breakinfo[i].pev); + breakinfo[i].pev =NULL ; + } + }unregister_die_notifier (&kgdb_notifier); }
arch/x86/kernel/hw_breakpoint.c +2 -3
@@ -467,7 +467,7 @@ static int __kprobes hw_breakpoint_handl {int i, cpu, rc =NOTIFY_STOP ;struct perf_event *bp; -unsigned long dr7, dr6; +unsigned long dr6;unsigned long *dr6_p;/* The DR6 value is pointed by args->err */
@@ -478,7 +478,6 @@ static int __kprobes hw_breakpoint_handlif ((dr6 &DR_TRAP_BITS ) ==0 )return NOTIFY_DONE ; -get_debugreg (dr7,7 );/* Disable breakpoints during exception handling */ set_debugreg (0UL ,7 );/*
@@ -526,7 +525,7 @@ static int __kprobes hw_breakpoint_handlif (dr6 & (~DR_TRAP_BITS )) rc =NOTIFY_DONE ; -set_debugreg (dr7,7 ); +set_debugreg (__get_cpu_var (cpu_dr7),7 );put_cpu ();return rc;
kernel/kgdb.c +3 -0
@@ -583,6 +583,9 @@ static void kgdb_wait(struct pt_regs *resmp_wmb ();atomic_set (&cpu_in_kgdb[cpu],1 ); +/* Disable any cpu specific hw breakpoints */ +kgdb_disable_hw_debug (regs); +/* Wait till primary CPU is done with debugging */ while (atomic_read (&passive_cpu_wait[cpu]))cpu_relax ();
Quoted 13 lines · Jason Wessel
I'll lead with a question. Are the people making the Perf API changes booting allyesconfig kernels?
The regression tests built into the kernel for kgdb fail as a result of the perf API changes and can result in a hard kernel hang.
My hope would have been that someone would have reported the problem, created a patch to disable the test that hangs the kernel, or to fix kgdb such that it works with the API changes. Likewise, if there are tests I should run to regression test the changes to the perf API, I would like to know since we all want to make use of the same hw resource.
Quoted 4 lines
A patch has been included in this mail which allows kgdb to pass the internal regression tests for hw breakpoints. I would like to get some comments or start a discussion as to how to solve this problem in the 2.6.33 cycle, as it is a regression in functionality.
Hm, the kgdb hw-breakpoint changes freshly put into v2.6.33 look pretty
broken: they access the raw hw registers and ignore the higher level abstraction. Your patch still uses way too low level primitives. Please use the highlevel abstraction: register/unregister_wide_hw_breakpoint() should do the trick. (if there's any changes/extensions needed to it then please let us know about it.)
If that's too much for v2.6.33 then i guess we need to revert or disable the kgdb hw-breakpoint changes for now.
Thanks,
Ingo
Quoted 2 lines · Ingo Molnar
Hm, the kgdb hw-breakpoint changes freshly put into v2.6.33 look pretty broken: [...]
Sorry - i thought for a moment that it was introduced by this recent change:
7f8b7ed: kgdb: Always process the whole breakpoint list on activate or deactivate
but it's deeper than that indeed.
Basically we have two options:
A- change kgdb to use the hw-breakpoints highlevel APIs (i'd prefer
that)
B- or keep what we had so far: kgdb overrides existing GDB (and now
perf) breakpoints
I havent noticed that hw-breakpoints lock up under kgdb.
Ingo
Quoted 7 lines · Ingo Molnar
Quoted 3 lines · Ingo Molnar
Hm, the kgdb hw-breakpoint changes freshly put into v2.6.33 look pretty broken: [...]
Quoted 8 lines
Sorry - i thought for a moment that it was introduced by this recent change:
7f8b7ed: kgdb: Always process the whole breakpoint list on activate or deactivate
but it's deeper than that indeed.
Yes, the broken parts came from the perf merge.
Quoted 6 lines
Basically we have two options:
A- change kgdb to use the hw-breakpoints highlevel APIs (i'd prefer
that)
Right now we can't because the high level code uses all sorts of mutexes and sync points to get the hw breakpoints installs on the various processors. After I re-spun my RFC patch, I found another problem. I do use the high level code to create a block of 4 (struct perf_event **) structures, but doing so ultimately calls the reserve hw breakpoint even though they are marked as disabled when created.
Should I, or can I change that behavior?
The aim of the RFC patch was to allow kgdb to using the arch_install* break point functions directly because they can be used atomically, and the perf code at the high level will still not get a breakpoint if kgdb is already using it that way, and kgdb won't squash a breakpoint that the perf code is using.
In the ideal world the management routines could be the same, but the problem is in the way we want to deal with the break points in kgdb while the master and all slave cpus are effectively stopped.
Quoted 3 lines
B- or keep what we had so far: kgdb overrides existing GDB (and now
perf) breakpoints
This is broken today, because the perf installs itself as the lowest priority.
If kgdb were to get a lower priority there is a way to make it work again using the existing code.
The RFC patch I made aligns things a little better in that at least perf, user space, and kgdb are all pulling from the same low level management code. It has the desirable side effect of making user space hw breakpoints work in conjunction with kernel space hw breakpoints which is something that never worked before.
Quoted 2 lines
I havent noticed that hw-breakpoints lock up under kgdb.
I don't see the lockups all the time, it happens when using the kgdb test suite from boot with the allyesconfig. Or if you boot the kernel with kgdbts=V1F100.
I think for now we might just consider turning off the hw breakpoint code in kgdb so we don't hang kernels, until this is sorted out. Patch provided.
Jason.
From: Jason Wessel <[email protected]>
Subject: [PATCH] x86,kgdb: disable kgdb hw breakpoints until they are fixed
Using hw breakpoints can hang the kgdb test suite or yeild unpredictable results while debugging due to the recent hw_breakpoint changes in 2.6.33. Until this is fixed, disable kgdb hw breakpoint support.
--- arch/x86/kernel/kgdb.c | 1 - 1 file changed, 1 deletion(-)
arch/x86/kernel/kgdb.c +0 -1
@@ -576,7 +576,6 @@ unsigned long kgdb_arch_pc(int exceptionstruct kgdb_arch arch_kgdb_ops = {/* Breakpoint instruction: */ .gdb_bpt_instr = {0xcc }, - .flags =KGDB_HW_BREAKPOINT , .set_hw_breakpoint = kgdb_set_hw_break, .remove_hw_breakpoint = kgdb_remove_hw_break, .remove_all_hw_break = kgdb_remove_all_hw_break,
Quoted 8 lines · Jason Wessel
Quoted 6 lines · Ingo Molnar
Basically we have two options:
A- change kgdb to use the hw-breakpoints highlevel APIs (i'd prefer
that)
Quoted 8 lines
Right now we can't because the high level code uses all sorts of mutexes and sync points to get the hw breakpoints installs on the various processors. After I re-spun my RFC patch, I found another problem. I do use the high level code to create a block of 4 (struct
perf_event**) structures, but doing so ultimately calls the reserve hw breakpoint even though they are marked as disabled when created.Should I, or can I change that behavior?
We could probably have a helper that allocates a disabled breakpoint without reserving it. But the problem remains: you'll need to take locks when you eventually reserve it and when you activate it.
The fact that it can happen from nmi is really a problem.
Is there any possibility that we know the user has started a kgdb session, and then reserve as much hardware breakpoints as we can in kgdb at this time?
Thanks.
Re: [RFC] Fix 2.6.33 x86 regression to kgdb hw breakpoints - dueto perf API changes
Quoted 18 lines · Frederic Weisbecker
Quoted 16 lines · Jason Wessel
Quoted 7 lines · Ingo Molnar
Basically we have two options:
A- change kgdb to use the hw-breakpoints highlevel APIs (i'd prefer
that)Right now we can't because the high level code uses all sorts of mutexes and sync points to get the hw breakpoints installs on the various processors. After I re-spun my RFC patch, I found another problem. I do use the high level code to create a block of 4 (struct
perf_event**) structures, but doing so ultimately calls the reserve hw breakpoint even though they are marked as disabled when created.
Quoted 7 lines
Quoted 2 lines
Should I, or can I change that behavior?
We could probably have a helper that allocates a disabled breakpoint without reserving it.
I worked around that restriction for now, in the current version of the kgdb patches. When kgdb registers with the die notifier in its init phase, it allocates the perf structures via the perf API and subsequently disables the breakpoints with the low level API.
Quoted 6 lines
But the problem remains: you'll need to take
locks when you eventually reserve it and when you activate it.The fact that it can happen from nmi is really a problem.
I talked with Jan a bit with respect to this problem. He recommended to possibly allow kgdb to obtain hw breakpoints locklessly and to break reservations that exist with the low level API. The current patch in the kgdb series does not break reservations, it only uses a slot that is not already in use. Let us call the scenarios A and B.
A) allow kgdb to break existing reservations
B) kgdb can use what is not reserved, without locks
What is missing right now is a notification mechanism and a separate
count for the debugger as to what is in use. I tend to think that B is
the right default approach, but Jan was leaning towards scenario A.
Quoted 5 lines
Is there any possibility that we know the user has started a kgdb session, and then reserve as much hardware breakpoints as we can in kgdb at this time?
That is the way I implemented it the first time. Reserve all the slots, and then nothing else could use them. That didn't work out too well because then the user space could not make use of hw breakpoints, granted this never worked before with user space + kernel space sharing between ptrace and kgdb.
Jason.
Re: [RFC] Fix 2.6.33 x86 regression to kgdb hw breakpoints - dueto perf API changes
Quoted 8 lines · Jason Wessel
Quoted 2 lines · Frederic Weisbecker
We could probably have a helper that allocates a disabled breakpoint without reserving it.
I worked around that restriction for now, in the current version of the kgdb patches. When kgdb registers with the die notifier in its init phase, it allocates the perf structures via the perf API and subsequently disables the breakpoints with the low level API.
It disables it but then the breakpoint still has a reserved slot, that looks too much for an opt-in config that may or
may not be used. One slot among four would be irremediably unavailable from userspace once we set CONFIG_KGDB, if I understand well... Is it doing so in the beginning of a debugging session or
at boot time?
Quoted 13 lines
Quoted 6 lines
But the problem remains: you'll need to take
locks when you eventually reserve it and when you activate it.The fact that it can happen from nmi is really a problem.
I talked with Jan a bit with respect to this problem. He recommended to possibly allow kgdb to obtain hw breakpoints locklessly and to break reservations that exist with the low level API. The current patch in the kgdb series does not break reservations, it only uses a slot that is not already in use. Let us call the scenarios A and B.
Quoted 6 lines
A) allow kgdb to break existing reservations
B) kgdb can use what is not reserved, without locksWhat is missing right now is a notification mechanism and a separate
count for the debugger as to what is in use. I tend to think that B is
the right default approach, but Jan was leaning towards scenario A.
A looks dangerous, in that an overflow of the possible number of breakpoints would make them fighting for the debug registers. Only 4 of them will make it :) (in x86)
I guess that in overflow case, the plan is to kick out one of the running breakpoints.
I see several problems in this preempt scheme:
- which breakpoint should we preempt?
- that will bring some complexities in the current code. You'll need to keep track of the preempted breakpoint, deactivate, reactivate it etc... Moreover the activation things require to take some locks.
I don't think A is good idea.
B looks feasible, but only at the cost of using a best effort try (in case of NMI).
You'll need to check if the current lock that protects the reservation datas is already taken. If so you have to give up.
That said this should be fine as the breakpoint reservation is a rare path, and I doubt one would try to set up a breakpoint in kgdb in the same time perf does, or any other users.
That said, either A or B, you'll need to take locks to activate/deactivate the breakpoints.
Quoted 11 lines
Quoted 5 lines
Is there any possibility that we know the user has started a kgdb session, and then reserve as much hardware breakpoints as we can in kgdb at this time?
That is the way I implemented it the first time. Reserve all the slots, and then nothing else could use them. That didn't work out too well because then the user space could not make use of hw breakpoints, granted this never worked before with user space + kernel space sharing between ptrace and kgdb.
Say one opens a kgdb session. A very cool thing would be to reserve as much breakpoints as we can (without prempting existing ones) and release them once the session is closed. This is not a problem that userspace can't reserve new ones during this session, we are debugging the kernel at this time. I doubt we need userspace breakpoints at the same time. Do we?
That said I really lack some kgdb background. In which context the user is connecting to kgdb? The above would only work in a non-NMI context.
Re: [RFC] Fix 2.6.33 x86 regression to kgdb hw breakpoints - duetoperf API changes
Quoted 15 lines · Frederic Weisbecker
Quoted 10 lines · Jason Wessel
Quoted 3 lines · Frederic Weisbecker
We could probably have a helper that allocates a disabled breakpoint without reserving it.
I worked around that restriction for now, in the current version of the kgdb patches. When kgdb registers with the die notifier in its init phase, it allocates the perf structures via the perf API and subsequently disables the breakpoints with the low level API.
Quoted 9 lines
It disables it but then the breakpoint still has a reserved slot, that looks too much for an opt-in config that may or
may not be used. One slot among four would be irremediably unavailable from userspace once we setCONFIG_KGDB, if I understand well... Is it doing so in the beginning of a debugging session or
at boot time?
In my latest patch, a single HW breakpoint slot goes away only while running through the arch specific kgdb init, it is immediately given back after the kgdb arch init completes. The arch init either occurs at boot via kernel cmd line, or is requested via "echo > /sys/module/kgdboc/...".
Ideally, I would like to get this fixed or at least back to the state of working where kgdb can make use of HW breakpoints in 2.6.33.
To address your questions.
I used the principle that kgdb can have what ever is "left over" in terms of available breakpoints that are not presently in use by the perf code. In order to configure kgdb there must be at least one HW breakpoint available or configuration fails.
We assume there is a person running the kernel debugger and has some knowledge about what all is going on and will make a decision about how he or she wants to allocate HW breakpoint resources.
Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
Is there any possibility that we know the user has started a kgdb session, and then reserve as much hardware breakpoints as we can in kgdb at this time?
Quoted 6 lines
That is the way I implemented it the first time. Reserve all the slots, and then nothing else could use them. That didn't work out too well because then the user space could not make use of hw breakpoints, granted this never worked before with user space + kernel space sharing between ptrace and kgdb.
Quoted 8 lines
Say one opens a kgdb session. A very cool thing would be to reserve as much breakpoints as we can (without prempting existing ones) and release them once the session is closed. This is not a problem that userspace can't reserve new ones during this session, we are debugging the kernel at this time. I doubt we need userspace breakpoints at the same time. Do we?
You can end up using user space HW breakpoints if you happen to be running an application that uses ptrace to set them. The current kgdb has never worked with user and system breaks.
Quoted 5 lines
That said I really lack some kgdb background. In which context the user is connecting to kgdb? The above would only work in a non-NMI context.
As I understand it, this is mainly about how the context switching occurs. There is some scheduling that sets and unsets the HW breakpoints which are managed through the perf API. You can have a perf system level HW break, a user space HW break (gdb debugging for instance), or kgdb setting a HW break. And as the system changes contexts, the kgdb breaks would in theory get restored by virtue of use of the low level API.
Do you have a test case for some typical use of perf + a HW breakpoint? Perhaps we can narrow down the problem set to the typical cases so as to move on.
Here is a simple case for instance:
* User sets perf HW breakpoint to get some data (perhaps you have an
example?)
* User makes use of KGDB to set HW breakpoint at do_fork()
Simple case 2:
* User uses a HW breakpoint in a looped app and does a "c 1000000" in gdb
* User uses KGDB to set a HW breakpoint in do_fork()
I figure you iterate a few times through to see that it is working. It
is possible to make an in kernel test case as well, but I figure it would be best to describe some typical, plausible case first.
Thanks,
Jason.
--
The patch is not inlined here but we are talking about:
http://git.kernel.org/?p=linux/kernel/git/jwessel/linux-2.6-kgdb.git;a=commitdiff;h=15bc6ce269f1d1f46751236bf5f099e051aa65ee