perf, x86: only do lbr init if bts is available
Subsystems (2)
KVM recently added support for a version 2 PMU. When passing -cpu host as the CPU model for the guest we get an abnormal configuration from perf's perspective in that the guest identifies the processor as a Westmere or Nehalem (etc):
[ 0.013998] Performance Events: Westmere events, Intel PMU driver.
but yet the processor does not have the debug store mechanisms (X86_FEATURE_DTES64 is not set) meaning there is no PEBS or BTS.
Right now the LBR init functions are run based on processor model which leads to attempts to write to LBR MSRs generating messages like:
[ 1170.125605] kvm: 6873: cpu0 unhandled rdmsr: 0x345 [ 1170.135354] kvm_set_msr_common: 54 callbacks suppressed [ 1170.135612] kvm: 6873: cpu0 unhandled wrmsr: 0x680 data 0 [ 1170.135900] kvm: 6873: cpu0 unhandled wrmsr: 0x6c0 data 0 [ 1170.136163] kvm: 6873: cpu0 unhandled wrmsr: 0x681 data 0 [ 1170.136437] kvm: 6873: cpu0 unhandled wrmsr: 0x6c1 data 0 [ 1170.136703] kvm: 6873: cpu0 unhandled wrmsr: 0x682 data 0 [ 1170.136968] kvm: 6873: cpu0 unhandled wrmsr: 0x6c2 data 0 [ 1170.137227] kvm: 6873: cpu0 unhandled wrmsr: 0x683 data 0 [ 1170.137502] kvm: 6873: cpu0 unhandled wrmsr: 0x6c3 data 0 [ 1170.137797] kvm: 6873: cpu0 unhandled wrmsr: 0x684 data 0 [ 1170.138061] kvm: 6873: cpu0 unhandled wrmsr: 0x6c4 data 0
MSR 0x345 is MSR_IA32_PERF_CAPABILITIES; the 0x6XX MSRs are LBR from/to.
Question: is something like this acceptable/make sense -- to only setup lbr values if the BTS is detected?
arch/x86/kernel/cpu/perf_event_intel.c +2 -1
@@ -1783,7 +1784,8 @@ __init int intel_pmu_init(void)memcpy (hw_cache_extra_regs, nehalem_hw_cache_extra_regs,sizeof (hw_cache_extra_regs)); -intel_pmu_lbr_init_nhm (); +if (x86_pmu.bts) +intel_pmu_lbr_init_nhm (); x86_pmu.event_constraints = intel_nehalem_event_constraints; x86_pmu.pebs_constraints = intel_nehalem_pebs_event_constraints;
David
Quoted 39 lines · David Ahern
KVM recently added support for a version 2 PMU. When passing -cpu host as the CPU model for the guest we get an abnormal configuration from perf's perspective in that the guest identifies the processor as a Westmere or Nehalem (etc):
[ 0.013998] Performance Events: Westmere events, Intel PMU driver.
but yet the processor does not have the debug store mechanisms (
X86_FEATURE_DTES64is not set) meaning there is no PEBS or BTS.Right now the LBR init functions are run based on processor model which leads to attempts to write to LBR MSRs generating messages like:
[ 1170.125605] kvm: 6873: cpu0 unhandled rdmsr: 0x345 [ 1170.135354] kvm_set_msr_common: 54 callbacks suppressed [ 1170.135612] kvm: 6873: cpu0 unhandled wrmsr: 0x680 data 0 [ 1170.135900] kvm: 6873: cpu0 unhandled wrmsr: 0x6c0 data 0 [ 1170.136163] kvm: 6873: cpu0 unhandled wrmsr: 0x681 data 0 [ 1170.136437] kvm: 6873: cpu0 unhandled wrmsr: 0x6c1 data 0 [ 1170.136703] kvm: 6873: cpu0 unhandled wrmsr: 0x682 data 0 [ 1170.136968] kvm: 6873: cpu0 unhandled wrmsr: 0x6c2 data 0 [ 1170.137227] kvm: 6873: cpu0 unhandled wrmsr: 0x683 data 0 [ 1170.137502] kvm: 6873: cpu0 unhandled wrmsr: 0x6c3 data 0 [ 1170.137797] kvm: 6873: cpu0 unhandled wrmsr: 0x684 data 0 [ 1170.138061] kvm: 6873: cpu0 unhandled wrmsr: 0x6c4 data 0MSR
0x345isMSR_IA32_PERF_CAPABILITIES; the 0x6XX MSRs are LBR from/to.Question: is something like this acceptable/make sense -- to only setup lbr values if the BTS is detected?
arch/x86/kernel/cpu/perf_event_intel.c +0
-0
arch/x86/kernel/cpu/perf_event_intel.c __init int intel_pmu_init(void) · line 1789
Quoted 7 lines
arch/x86/kernel/cpu/perf_event_intel.c +2
-1@@ -1783,7 +1784,8 @@ __init int intel_pmu_init(void) memcpy(hw_cache_extra_regs, nehalem_hw_cache_extra_regs, sizeof(hw_cache_extra_regs)); - intel_pmu_lbr_init_nhm(); + if (x86_pmu.bts) + intel_pmu_lbr_init_nhm();
Well, no. There is no connection between BTS and LBR and you're creating one.
Where is the wrmsr coming from? What we need to do is ensure that LBR is not touched if we don't actually use it.
Quoted 4 lines
arch/x86/kernel/cpu/perf_event_intel.c +0
-0x86_pmu.event_constraints = intel_nehalem_event_constraints; x86_pmu.pebs_constraints = intel_nehalem_pebs_event_constraints;David
Quoted 1 line · Stephane Eranian
Well, no. There is no connection between BTS and LBR and you're creating one.
Ok. That was not clear to me from skimming the manual. Then should it be tied to X86_FEATURE_DTES64?
arch/x86/kernel/cpu/perf_event_intel_lbr.c +3 -0
@@ -640,6 +640,9 @@ void intel_pmu_lbr_init_core(void)/* nehalem/westmere */ void intel_pmu_lbr_init_nhm (void ) { +if (!boot_cpu_has (X86_FEATURE_DTES64 )) +return ; + x86_pmu.lbr_nr =16 ; x86_pmu.lbr_tos =MSR_LBR_TOS ; x86_pmu.lbr_from =MSR_LBR_NHM_FROM ;
Quoted 2 lines
Where is the wrmsr coming from? What we need to do is ensure that LBR is not touched if we don't actually use it.
e.g., intel_pmu_lbr_init_nhm sets up lbr_nr, lbr_from, lbr_to and from, etc. Fhat I can tell intel_pmu_lbr_reset() gets invoked some where during the VM boot; I haven't traced how/when yet.
David
Quoted 8 lines · David Ahern
Quoted 3 lines · Stephane Eranian
Where is the wrmsr coming from? What we need to do is ensure that LBR is not
touched if we don't actually use it.e.g.,
intel_pmu_lbr_init_nhmsets uplbr_nr,lbr_from,lbr_toand from, etc. Fhat I can tellintel_pmu_lbr_reset()gets invoked some where during the VM boot; I haven't traced how/when yet.
[ 0.012998] [<c04220ca>] intel_pmu_lbr_reset+0x2a/0xa0 [ 0.012998] [<c0423beb>] intel_pmu_cpu_starting+0x3b/0x100 [ 0.012998] [<c0424bd0>] ? allocate_shared_regs+0x20/0x50 [ 0.012998] [<c0932714>] x86_pmu_notifier+0xb3/0xc0 [ 0.012998] [<c0bed54c>] init_hw_perf_events+0x42c/0x456 [ 0.012998] [<c0403034>] do_one_initcall+0x34/0x170 [ 0.012998] [<c0bf2388>] ? native_smp_prepare_cpus+0x2b7/0x2f2 [ 0.012998] [<c0bed120>] ? check_bugs+0xf9/0xf9 [ 0.012998] [<c0be4833>] kernel_init+0x77/0x1a4 [ 0.012998] [<c0be47bc>] ? start_kernel+0x36d/0x36d [ 0.012998] [<c094af7e>] kernel_thread_helper+0x6/0x10
Quoted 9 lines · David Ahern
Quoted 3 lines · Stephane Eranian
Well, no. There is no connection between BTS and LBR and you're creating one.
Ok. That was not clear to me from skimming the manual. Then should it be tied to
X86_FEATURE_DTES64?
No, it is unrelated to the Debug Store.
There is really nothing you can use to figure that out, not event IA32_DEBUGCTL. I wish Intel had something in PERF_CAPABILITIES, but LBR was originally considered
as a debug feature, I think. To the point that it is described in the debug section not in
vol 3b with performance monitoring.
Quoted 28 lines
arch/x86/kernel/cpu/perf_event_intel_lbr.c +3
-0@@ -640,6 +640,9 @@ void intel_pmu_lbr_init_core(void) /* nehalem/westmere */ void intel_pmu_lbr_init_nhm(void) { + if (!boot_cpu_has(X86_FEATURE_DTES64)) + return; + x86_pmu.lbr_nr = 16; x86_pmu.lbr_tos = MSR_LBR_TOS; x86_pmu.lbr_from = MSR_LBR_NHM_FROM;Quoted 6 lines
Where is the wrmsr coming from? What we need to do is ensure that LBR is not
touched if we don't actually use it.e.g.,
intel_pmu_lbr_init_nhmsets uplbr_nr,lbr_from,lbr_toand from, etc. Fhat I can tellintel_pmu_lbr_reset()gets invoked some where during the VM boot; I haven't traced how/when yet.David
Quoted 7 lines · Stephane Eranian
Quoted 6 lines · David Ahern
Quoted 3 lines · Stephane Eranian
Well, no. There is no connection between BTS and LBR and you're creating one.
Quoted 5 lines
Quoted 3 lines
Ok. That was not clear to me from skimming the manual. Then should it be tied to
X86_FEATURE_DTES64?No, it is unrelated to the Debug Store.
There is really nothing you can use to figure that out, not eventIA32_DEBUGCTL.
One way to do it would be to reread DEBUGCTL after writing and see if the bit really changed to 1. If not assume LBR is not available.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only
Peter Zijlstra
maintainerQuoted 5 lines · David Ahern
KVM recently added support for a version 2 PMU. When passing -cpu host as the CPU model for the guest we get an abnormal configuration from perf's perspective in that the guest identifies the processor as a Westmere or Nehalem (etc):
Quoted 8 lines
[ 0.013998] Performance Events: Westmere events, Intel PMU driver.but yet the processor does not have the debug store mechanisms (
X86_FEATURE_DTES64is not set) meaning there is no PEBS or BTS.Right now the LBR init functions are run based on processor model which leads to attempts to write to LBR MSRs generating messages like:
Right, as Stephane already noted, LBR is not an CPUID enumerated feature, hence the only thing you can do is go by model. So if you set qemu -cpu host but fail to implement all the architectural MSRs that go with that model you get to keep the pieces.
Not really our problem.
Peter Zijlstra
maintainerQuoted 4 lines · Peter Zijlstra
Right, as Stephane already noted, LBR is not an CPUID enumerated feature, hence the only thing you can do is go by model. So if you set qemu -cpu host but fail to implement all the architectural MSRs that go with that model you get to keep the pieces.
s/architectural//
They're called Model-Specific-Registers for a reason ;-)