NMI watchdog: when booting with reset_devices, clear the performance counters
PATCH 1/1 NMI watchdog: when booting with reset_devices, clear the performance counters
P4s have a quirk that makes necessary to clear P4_CCCR_OVF bit on the CCCR everytime the PMI is triggered. When booting the kernel with reset_devices (more specific kdump case), the counters reach zero and the PMI will be generated. This is not a problem on other processors but on P4s, it'll continue to generate NMIs until that bit is cleared. Since there may be other users of the performance counters, clear and disable all of them when booting with reset_devices option.
We have a P4 box here that crashes because of this problem. Since the kdump kernel usually boots with only one processor active, the second logical unit won't be set up, therefore, MSR_P4_IQ_CCCR1 (and other performance counter registers) won't be cleared and P4_CCCR_OVF may be still set because the previous kernel was using this register. An NMI is triggered because of the MSR_P4_IQ_CCCR1 right after the NMI delivery is enabled, triggering the race fixed on my previous email.
--- arch/x86/kernel/cpu/perfctr-watchdog.c | 40 +++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)
arch/x86/kernel/cpu/perfctr-watchdog.c +41 -0
@@ -444,6 +444,27 @@ static const struct wd_ops p6_wd_ops = {#define P4_CCCR_ENABLE (1 << 12)#define P4_CCCR_OVF (1 << 31) +#define P4_CONTROLS 18 +static unsigned int p4_controls[18 ] = { +MSR_P4_BPU_CCCR0 , +MSR_P4_BPU_CCCR1 , +MSR_P4_BPU_CCCR2 , +MSR_P4_BPU_CCCR3 , +MSR_P4_MS_CCCR0 , +MSR_P4_MS_CCCR1 , +MSR_P4_MS_CCCR2 , +MSR_P4_MS_CCCR3 , +MSR_P4_FLAME_CCCR0 , +MSR_P4_FLAME_CCCR1 , +MSR_P4_FLAME_CCCR2 , +MSR_P4_FLAME_CCCR3 , +MSR_P4_IQ_CCCR0 , +MSR_P4_IQ_CCCR1 , +MSR_P4_IQ_CCCR2 , +MSR_P4_IQ_CCCR3 , +MSR_P4_IQ_CCCR4 , +MSR_P4_IQ_CCCR5 , +};/* * Set up IQ_COUNTER0 to behave like a clock, by having IQ_CCCR0 filter * CRU_ESCR0 (with any non-null event selector) through a complemented
@@ -485,6 +506,26 @@ static int setup_p4_watchdog(unsigned nm evntsel_msr =MSR_P4_CRU_ESCR0 ; cccr_msr =MSR_P4_IQ_CCCR0 ; cccr_val =P4_CCCR_OVF_PMI0 |P4_CCCR_ESCR_SELECT (4 ); + +/* +* If we're on the kdump kernel or other situation, we may +* still have other performance counter registers set to +* interrupt and they'll keep interrupting forever because +* of the P4_CCCR_OVF quirk. So we need to ACK all the +* pending interrupts and disable all the registers here, +* before reenabling the NMI delivery. Refer to p4_rearm() +* about the P4_CCCR_OVF quirk. +*/ +if (reset_devices) { +unsigned int low, high; +int i; + +for (i =0 ; i <P4_CONTROLS ; i++) { +rdmsr (p4_controls[i], low, high); + low &= ~(P4_CCCR_ENABLE |P4_CCCR_OVF ); +wrmsr (p4_controls[i], low, high); + } + } }else {/* logical cpu 1 */ perfctr_msr =MSR_P4_IQ_PERFCTR1 ;
Ingo Molnar
AppliedQuoted 16 lines · Aristeu Rozanski
P4s have a quirk that makes necessary to clear
P4_CCCR_OVFbit on the CCCR everytime the PMI is triggered. When booting the kernel withreset_devices(more specific kdump case), the counters reach zero and the PMI will be generated. This is not a problem on other processors but on P4s, it'll continue to generate NMIs until that bit is cleared. Since there may be other users of the performance counters, clear and disable all of them when booting withreset_devicesoption.We have a P4 box here that crashes because of this problem. Since the kdump kernel usually boots with only one processor active, the second logical unit won't be set up, therefore,
MSR_P4_IQ_CCCR1(and other performance counter registers) won't be cleared andP4_CCCR_OVFmay be still set because the previous kernel was using this register. An NMI is triggered because of theMSR_P4_IQ_CCCR1right after the NMI delivery is enabled, triggering the race fixed on my previous email.
Quoted 4 lines
Signed-off-by Aristeu Rozanski <[email protected]>Acked-by Don Zickus <[email protected]>Acked-by Prarit Bhargava <[email protected]>Acked-by Vivek Goyal <[email protected]>
applied to tip/x86/nmi-watchdog, thanks.
i'm wondering, is this fix a v2.6.27 candidate?
Ingo
Quoted 16 lines
Quoted 16 lines
P4s have a quirk that makes necessary to clear
P4_CCCR_OVFbit on the CCCR everytime the PMI is triggered. When booting the kernel withreset_devices(more specific kdump case), the counters reach zero and the PMI will be generated. This is not a problem on other processors but on P4s, it'll continue to generate NMIs until that bit is cleared. Since there may be other users of the performance counters, clear and disable all of them when booting withreset_devicesoption.We have a P4 box here that crashes because of this problem. Since the kdump kernel usually boots with only one processor active, the second logical unit won't be set up, therefore,
MSR_P4_IQ_CCCR1(and other performance counter registers) won't be cleared andP4_CCCR_OVFmay be still set because the previous kernel was using this register. An NMI is triggered because of theMSR_P4_IQ_CCCR1right after the NMI delivery is enabled, triggering the race fixed on my previous email.
Quoted 8 lines
Quoted 4 lines
Signed-off-by Aristeu Rozanski <[email protected]>Acked-by Don Zickus <[email protected]>Acked-by Prarit Bhargava <[email protected]>Acked-by Vivek Goyal <[email protected]>applied to tip/x86/nmi-watchdog, thanks.
i'm wondering, is this fix a v2.6.27 candidate?
I believe so. Even being a small affected group (P4 based boxes using NMI watchdog and kdump), it's fully reproducible.
Signature · 1 line
Aristeu
Quoted 21 lines · Aristeu Rozanski
Quoted 21 lines
Quoted 20 lines
P4s have a quirk that makes necessary to clear
P4_CCCR_OVFbit on the CCCR everytime the PMI is triggered. When booting the kernel withreset_devices(more specific kdump case), the counters reach zero and the PMI will be generated. This is not a problem on other processors but on P4s, it'll continue to generate NMIs until that bit is cleared. Since there may be other users of the performance counters, clear and disable all of them when booting withreset_devicesoption.We have a P4 box here that crashes because of this problem. Since the kdump kernel usually boots with only one processor active, the second logical unit won't be set up, therefore,
MSR_P4_IQ_CCCR1(and other performance counter registers) won't be cleared andP4_CCCR_OVFmay be still set because the previous kernel was using this register. An NMI is triggered because of theMSR_P4_IQ_CCCR1right after the NMI delivery is enabled, triggering the race fixed on my previous email.Signed-off-by Aristeu Rozanski <[email protected]>Acked-by Don Zickus <[email protected]>Acked-by Prarit Bhargava <[email protected]>Acked-by Vivek Goyal <[email protected]>
Quoted 5 lines
Quoted 3 lines
applied to tip/x86/nmi-watchdog, thanks.
i'm wondering, is this fix a v2.6.27 candidate?
I believe so. Even being a small affected group (P4 based boxes using NMI watchdog and kdump), it's fully reproducible.
is there a bugzilla.kernel.org entry for it? If not, could you please file it? Thanks,
Ingo
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
applied to tip/x86/nmi-watchdog, thanks.
i'm wondering, is this fix a v2.6.27 candidate?
I believe so. Even being a small affected group (P4 based boxes using NMI watchdog and kdump), it's fully reproducible.
is there a bugzilla.kernel.org entry for it? If not, could you please file it? Thanks,
#11621 created
Signature · 1 line
Aristeu