introduce NMI_AUTO as nmi_watchdog option
Hi Ingo,
To dig up an old thread last November:
======
Quoted 10 lines · Aristeu Rozanski
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
NMI_AUTOis a newnmi_watchdogoption that makes LAPIC be tried first
and if the CPU isn't supported, IOAPIC will be used. It's useful in
cases where NMI watchdog is enabled by default in a kernel built for
different machines. It can be configured by default or selected with
nmi_watchdog=3 or nmi_watchdog=auto parameters.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
What i'd like to see for the NMI watchdog is much more ambitious than
this: the use of perf events to run a periodic NMI callback.The NMI watchdog would cause the creation of a per-cpu
perf_eventstructure (in-kernel). All x86 CPUs that have perf event support (the
> > > > majority of them) will thus be able to have an NMI
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
watchdog using a
nice, generic piece of code and we'd be able to phase out the open-coded
NMI watchdog code.The user would not notice much from this: we'd still have the
/proc/sys/kernel/nmi_watchdogtoggle to turn it on/off, and we'd still
have the nmi_watchog= boot parameter as well. But the underlying implementation would be far more generic and far more usable than the
current code.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Would you be interested in moving the NMI watchdog code in this direction? Most of the perf events changes (callbacks, helpers for
> > > > in-kernel event allocations, etc.) are in latest
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
-tip already, so you
could use that as a base.but that would work only for LAPIC. You're suggesting killing IOAPIC mode too?
Quoted 8 lines
Quoted 4 lines
Would it be a big loss, with all modern systems expected to have a working lapic based NMI source? I wrote the IOAPIC mode originally but i
dont feel too attached to it ;-)ok, fair enough. but since it'll be another implementation, do you mind applying the patches I submitted so they can be used until the new implementation is in place?
For that i need to see at least an RFC v1 version series of the new implementation - otherwise we might end up sitting on this interim version with no-one doing the better variant.
========
I was going to jump in and try to do this work. I wanted to make sure what you were looking for here. When you say convert nmi watchdog to perf events, I assume you mean merging over the bits of perfctr-watchdog.c to perf_events.c, modify nmi.c to just register as a normal perf event and probably cleanup the oprofile stuff to match, correct?
Cheers,
Don
Quoted 2 lines · Don Zickus
Hi Ingo,
...
Quoted 9 lines
I was going to jump in and try to do this work. I wanted to make sure what you were looking for here. When you say convert nmi watchdog to perf events, I assume you mean merging over the bits of
perfctr-watchdog.ctoperf_events.c, modifynmi.cto just register as a normal perf event and probably cleanup the oprofile stuff to match, correct?Cheers,
Don
As far as I know -- converting perfctr-watchdog.c to into perfevents style would be quite a desirable feature. But I still didn't manage to find time for this task :( If you're interested to start this work -- that would be just great!
-- Cyrill
Quoted 10 lines · Cyrill Gorcunov
Quoted 2 lines · Don Zickus
Hi Ingo,
...
Quoted 6 lines
I was going to jump in and try to do this work. I wanted to make sure what you were looking for here. When you say convert nmi watchdog to perf events, I assume you mean merging over the bits of
perfctr-watchdog.ctoperf_events.c, modifynmi.cto just register as a normal perf event and probably cleanup the oprofile stuff to match, correct?
Quoted 8 lines
Quoted 3 lines
Cheers,
DonAs far as I know -- converting
perfctr-watchdog.cto into perfevents style would be quite a desirable feature. But I still didn't manage to find time for this task :( If you're interested to start this work -- that would be just great!
After looking through the code I just had some questions, perhaps you have thought about this longer than me, what to do with the reservation code (just remove it I assume and let perf_events _be_ the only code that
handles perf events) and what to do with some of the cpu quirks as noted
in perfctr-watchdog.c (notable some of the Intel errata for the Core chipsets).
Cheers,
Don
Quoted 27 lines · Don Zickus
Quoted 10 lines · Cyrill Gorcunov
Quoted 2 lines · Don Zickus
Hi Ingo,
...
Quoted 6 lines
I was going to jump in and try to do this work. I wanted to make sure what you were looking for here. When you say convert nmi watchdog to perf events, I assume you mean merging over the bits of
perfctr-watchdog.ctoperf_events.c, modifynmi.cto just register as a normal perf event and probably cleanup the oprofile stuff to match, correct?Quoted 8 lines
Quoted 3 lines
Cheers,
DonAs far as I know -- converting
perfctr-watchdog.cto into perfevents style would be quite a desirable feature. But I still didn't manage to find time for this task :( If you're interested to start this work -- that would be just great!After looking through the code I just had some questions, perhaps you have thought about this longer than me, what to do with the reservation code (just remove it I assume and let
perf_events_be_ the only code that
handles perf events) and what to do with some of the cpu quirks as noted
inperfctr-watchdog.c(notable some of the Intel errata for the Core chipsets).
Quoted 3 lines
Cheers,
Don
Hi Don,
well I must admit I didn't look too close to this code (if I had I would have sent some patch for review at least :). But I was suggested to take a look on hw_breakpoint.c (Frederic worked on it iirc, CC'ed) as an example of perfevent'ed code. So converting to perf-event is not trivial task and I fear I can't give any useful advice at moment since as I said I didn't manage to find time for this task and as result didn't read code byte-to-byte, sorry. But if I get some idea -- will share!
-- Cyrill
Quoted 20 lines · Don Zickus
Quoted 10 lines · Cyrill Gorcunov
Quoted 2 lines · Don Zickus
Hi Ingo,
...
Quoted 6 lines
I was going to jump in and try to do this work. I wanted to make sure what you were looking for here. When you say convert nmi watchdog to perf events, I assume you mean merging over the bits of
perfctr-watchdog.ctoperf_events.c, modifynmi.cto just register as a normal perf event and probably cleanup the oprofile stuff to match, correct?Quoted 8 lines
Quoted 3 lines
Cheers,
DonAs far as I know -- converting
perfctr-watchdog.cto into perfevents style would be quite a desirable feature. But I still didn't manage to find time for this task :( If you're interested to start this work -- that would be just great!
Quoted 5 lines
After looking through the code I just had some questions, perhaps you have thought about this longer than me, what to do with the reservation code (just remove it I assume and let
perf_events_be_ the only code that
handles perf events) and what to do with some of the cpu quirks as noted inperfctr-watchdog.c(notable some of the Intel errata for the Core chipsets).
Given the amount of quirks in the perctr code it might make sense to shape this as a new feature initially: introduce a new NMI watchdog that is perf based and has a different codebase.
Then, once it's capable enough and has been in circulation long enough we can simply drop the old NMI watchdog. (without users noticing anything [modulo bugs])
v1 should concentrate on x86 CPUs that are supported by perf currently. Note, it _might_ make sense to do it via a new kernel/nmi_watchdog.c file - other architectures have NMI concepts as well, such as Sparc64. A further idea would be to maybe even merge it with the softlockup code in kernel/softlockup.c - so that we dont have two sets of apis like touch_nmi_watchdog and touch_softlockup_watchdog.
So there's a wide spectrum of possibilities - the important thing is to start small :-)
Ingo
Quoted 1 line · Ingo Molnar
other architectures have NMI concepts as well, such as Sparc64.
I think both sparc64 and ppc64 fake NMIs by playing games with hw IRQ priorities and partial masks. But yes.
One interesting 'feature' for the perf-nmi interaction is creating an idle scheduling class for counters, because as long as there is a counter present you can use his NMIs to drive the watchdog, but as soon as there are non left, you need to install one.
Quoted 10 lines · Peter Zijlstra
Quoted 1 line · Ingo Molnar
other architectures have NMI concepts as well, such as Sparc64.
I think both sparc64 and ppc64 fake NMIs by playing games with hw IRQ priorities and partial masks. But yes.
One interesting 'feature' for the perf-nmi interaction is creating an idle scheduling class for counters, because as long as there is a counter present you can use his NMIs to drive the watchdog, but as soon as there are non left, you need to install one.
Interesting idea. How can I guarantee the frequency of the NMI I want to piggyback off of? A breakpoint that takes an hour to trigger may not be the best NMI to use? Then again I am still trying to understand the perf event code a little better.
Cheers,
Don
Quoted 12 lines · Don Zickus
Quoted 10 lines · Peter Zijlstra
Quoted 1 line · Ingo Molnar
other architectures have NMI concepts as well, such as Sparc64.
I think both sparc64 and ppc64 fake NMIs by playing games with hw IRQ priorities and partial masks. But yes.
One interesting 'feature' for the perf-nmi interaction is creating an idle scheduling class for counters, because as long as there is a counter present you can use his NMIs to drive the watchdog, but as soon as there are non left, you need to install one.
Quoted 4 lines
Interesting idea. How can I guarantee the frequency of the NMI I want to piggyback off of? A breakpoint that takes an hour to trigger may not be the best NMI to use? Then again I am still trying to understand the perf event code a little better.
You could play games with the period, we can handle getting more NMIs than are needed. This is how we implement a period larger than the physical counter for example.
But yeah, its a tricky game since a tight loop might never generate the event we're counting.. we could limit this to things like cycles/ins/bus-cycles etc.. those will always tick.
Anyway, its all an optimization, the simple/first implementation would simply install a kernel cpu perf counter and hook the overflow handler.
Quoted 10 lines · Peter Zijlstra
Quoted 1 line · Ingo Molnar
other architectures have NMI concepts as well, such as Sparc64.
I think both sparc64 and ppc64 fake NMIs by playing games with hw IRQ priorities and partial masks. But yes.
One interesting 'feature' for the perf-nmi interaction is creating an idle scheduling class for counters, because as long as there is a counter present you can use his NMIs to drive the watchdog, but as soon as there are non left, you need to install one.
Yeah. I'd suggest to not complicate things with that initially - but to simply create a standalone event for it and 'waste' a counter on NMI generation.
Later on it can indeed be a good feature to make the NMI watchdog 'seemless' in the sense of it not causing any wasted hw resources - it can piggyback on any existing NMI event. (as long as that event is at least ~1 HZ strong or so)
Ingo
Quoted 14 lines · Ingo Molnar
Quoted 5 lines
After looking through the code I just had some questions, perhaps you have thought about this longer than me, what to do with the reservation code (just remove it I assume and let
perf_events_be_ the only code that
handles perf events) and what to do with some of the cpu quirks as noted inperfctr-watchdog.c(notable some of the Intel errata for the Core chipsets).Given the amount of quirks in the perctr code it might make sense to shape this as a new feature initially: introduce a new NMI watchdog that is perf based and has a different codebase.
Then, once it's capable enough and has been in circulation long enough we can simply drop the old NMI watchdog. (without users noticing anything [modulo bugs])
Quoted 6 lines
v1 should concentrate on x86 CPUs that are supported by perf currently. Note, it _might_ make sense to do it via a new
kernel/nmi_watchdog.cfile - other architectures have NMI concepts as well, such as Sparc64. A further idea would be to maybe even merge it with the softlockup code inkernel/softlockup.c- so that we dont have two sets of apis liketouch_nmi_watchdogandtouch_softlockup_watchdog.
Ok, interesting. Right now I am working on making sure I know how to register something with the perf event framework (from kernel space). Once I can do that, I'll expand it outward and see where it goes. :-)
Quoted 3 lines
So there's a wide spectrum of possibilities - the important thing is to start small :-)
I see. Thanks.
Cheers,
Don