perf: Clear MSRs on kexec
Subsystems (4)
RFC 1/1 perf: Clear MSRs on kexec
hi,
I'm getting following message on the kdump kernel start
Broken BIOS detected, complain to your hardware vendor.\ [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is b0)
it seems to be caused by NMI watchdog being configured and fixed counter values stays in MSRs, which triggers warning in check_hw_exists and disables perf support in kdump kernel.. which probably does not hurt ;-)
zeroing MSRs during kdump shutdown seems to work (attached) but I'm not sure thats correct place for kdump perf callback
thanks,
jirka
--- arch/x86/include/asm/perf_event.h | 2 ++ arch/x86/kernel/cpu/perf_event.c | 23 +++++++++++++++++++++++ arch/x86/kernel/cpu/perf_event.h | 1 + arch/x86/kernel/crash.c | 3 +++ 4 files changed, 29 insertions(+)
arch/x86/include/asm/perf_event.h +2 -0
@@ -217,6 +217,7 @@ static inline u32 get_ibs_caps(void) { return 0; }#ifdef CONFIG_PERF_EVENTS extern void perf_events_lapic_init (void ); +extern void perf_clear_msrs (void );/* * Abuse bits {3,5} of the cpu eflags register. These flags are otherwise
@@ -275,6 +276,7 @@ static inline void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap)static inline void perf_events_lapic_init (void ) { }static inline void perf_check_microcode (void ) { } +static inline void perf_clear_msrs (void ) { }#endif #if defined(CONFIG_PERF_EVENTS ) && defined(CONFIG_CPU_SUP_AMD )
arch/x86/kernel/cpu/perf_event.c +23 -0
@@ -269,6 +269,27 @@ msr_fail:return false; } +void perf_clear_msrs (void ) +{ +int i, reg, ret; + +if (!x86_pmu.enabled) +return ; + +for (i =0 ; i < x86_pmu.num_counters; i++) { + reg =x86_pmu_config_addr (i); + ret =wrmsrl_safe (reg,0 ); +if (WARN_ONCE (ret,"failed to zero perf counter msr, reg %x\n" , reg)) +break ; + } + +if (x86_pmu.num_counters_fixed) { + reg =MSR_ARCH_PERFMON_FIXED_CTR_CTRL ; + ret =wrmsrl_safe (reg,0 ); +WARN_ONCE (ret,"failed to zero perf fixed counters msr\n" ); + } +} +static void hw_perf_event_destroy (struct perf_event *event) {x86_release_hardware ();
@@ -1689,6 +1710,8 @@ static int __init init_hw_perf_events(void)if (!check_hw_exists ())return 0 ; + x86_pmu.enabled = true; +pr_cont ("%s PMU driver.\n" , x86_pmu.name); x86_pmu.attr_rdpmc =1 ;/* enable userspace RDPMC usage by default */
arch/x86/kernel/cpu/perf_event.h +1 -0
@@ -508,6 +508,7 @@ struct x86_pmu {*/ const char *name;int version; +bool enabled;int (*handle_irq)(struct pt_regs *);void (*disable_all)(void );void (*enable_all)(int added);
arch/x86/kernel/crash.c +3 -0
@@ -35,6 +35,7 @@#include <asm/cpu.h> #include <asm/reboot.h> #include <asm/virtext.h> +#include <asm/perf_event.h> /* Alignment required for elf header segment */ #define ELF_CORE_HEADER_ALIGN 4096
@@ -128,6 +129,7 @@ static void kdump_nmi_callback(int cpu, struct pt_regs *regs)cpu_emergency_svm_disable ();disable_local_APIC (); +perf_clear_msrs (); }static void kdump_nmi_shootdown_cpus (void )
@@ -182,6 +184,7 @@ void native_machine_crash_shutdown(struct pt_regs *regs)hpet_disable ();#endif crash_save_cpu (regs,safe_smp_processor_id ()); +perf_clear_msrs (); }#ifdef CONFIG_KEXEC_FILE
Signature · 1 line
2.4.3
Peter Zijlstra
maintainerQuoted 6 lines · Jiri Olsa
hi,
I'm getting following message on the kdump kernel startBroken BIOS detected, complain to your hardware vendor.\ [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is b0)
Quoted 7 lines
it seems to be caused by NMI watchdog being configured and fixed counter values stays in MSRs, which triggers warning in
check_hw_existsand disables perf support in kdump kernel.. which probably does not hurt ;-)zeroing MSRs during kdump shutdown seems to work (attached) but I'm not sure thats correct place for kdump perf callback
Right, but why bother? All that kernel needs to do is write a memory dump to someplace and reboot, right? The less you do, the less can go wrong.
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 12 lines · Peter Zijlstra
Quoted 11 lines · Jiri Olsa
hi,
I'm getting following message on the kdump kernel startBroken BIOS detected, complain to your hardware vendor.\ [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is b0)it seems to be caused by NMI watchdog being configured and fixed counter values stays in MSRs, which triggers warning in
check_hw_existsand disables perf support in kdump kernel.. which probably does not hurt ;-)
Quoted 6 lines
Quoted 2 lines
zeroing MSRs during kdump shutdown seems to work (attached) but I'm not sure thats correct place for kdump perf callback
Right, but why bother? All that kernel needs to do is write a memory dump to someplace and reboot, right? The less you do, the less can go wrong.
well, I was hunting that 'Broken BIOS..' message which is wrong
I wouldn't think anyone wants to use perf under kdump kernel, but you never know ;-)
jirka
Peter Zijlstra
maintainerQuoted 13 lines · Jiri Olsa
Quoted 12 lines · Peter Zijlstra
Quoted 11 lines · Jiri Olsa
hi,
I'm getting following message on the kdump kernel startBroken BIOS detected, complain to your hardware vendor.\ [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is b0)it seems to be caused by NMI watchdog being configured and fixed counter values stays in MSRs, which triggers warning in
check_hw_existsand disables perf support in kdump kernel.. which probably does not hurt ;-)
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
zeroing MSRs during kdump shutdown seems to work (attached) but I'm not sure thats correct place for kdump perf callback
Right, but why bother? All that kernel needs to do is write a memory dump to someplace and reboot, right? The less you do, the less can go wrong.
well, I was hunting that 'Broken BIOS..' message which is wrong
Not really. The previous kernel being the BIOS in this case did leave the counters in a funky state.
Quoted 2 lines
I wouldn't think anyone wants to use perf under kdump kernel, but you never know ;-)
Yeah, I think we knew about this back then (might've been 2010) and chose to not 'fix' it.
http://lkml.iu.edu/hypermail/linux/kernel/1012.1/00380.html
Is what google finds me.
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 13 lines · Jiri Olsa
Quoted 12 lines · Peter Zijlstra
Quoted 11 lines · Jiri Olsa
hi,
I'm getting following message on the kdump kernel startBroken BIOS detected, complain to your hardware vendor.\ [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is b0)it seems to be caused by NMI watchdog being configured and fixed counter values stays in MSRs, which triggers warning in
check_hw_existsand disables perf support in kdump kernel.. which probably does not hurt ;-)
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
zeroing MSRs during kdump shutdown seems to work (attached) but I'm not sure thats correct place for kdump perf callback
Right, but why bother? All that kernel needs to do is write a memory dump to someplace and reboot, right? The less you do, the less can go wrong.
well, I was hunting that 'Broken BIOS..' message which is wrong
It was the limitation we understood, when we implemented the check_hw_exists() years ago.
Quoted 3 lines
I wouldn't think anyone wants to use perf under kdump kernel, but you never know ;-)
The amount of warning messages in the kdump kernel is significant, this message is just another message to add to the noise. Heck we have to poll irqs because if we enable them, drivers would crash left and right due to left over pending irqs from the previous kernel.
As Peter said, how much code are we going to put in the crash path to make kdump less noisy?
Eric B. usually frowns pretty heavily on this.
Cheers,
Don