Skip to contentSkip to conversation

perf, x86: only do lbr init if bts is available

PATCH David Ahern 4 participants 8 messages May 24, 2012 – May 25, 2012

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

KVM recently added support for a version 2 PMU. When passing -cpu host as the CPU model for the guest we get an abnormal configuration from perf's perspective in that the guest identifies the processor as a Westmere or Nehalem (etc):

[    0.013998] Performance Events: Westmere events, Intel PMU driver.

but yet the processor does not have the debug store mechanisms (X86_FEATURE_DTES64 is not set) meaning there is no PEBS or BTS.

Right now the LBR init functions are run based on processor model which leads to attempts to write to LBR MSRs generating messages like:

[ 1170.125605] kvm: 6873: cpu0 unhandled rdmsr: 0x345
[ 1170.135354] kvm_set_msr_common: 54 callbacks suppressed
[ 1170.135612] kvm: 6873: cpu0 unhandled wrmsr: 0x680 data 0
[ 1170.135900] kvm: 6873: cpu0 unhandled wrmsr: 0x6c0 data 0
[ 1170.136163] kvm: 6873: cpu0 unhandled wrmsr: 0x681 data 0
[ 1170.136437] kvm: 6873: cpu0 unhandled wrmsr: 0x6c1 data 0
[ 1170.136703] kvm: 6873: cpu0 unhandled wrmsr: 0x682 data 0
[ 1170.136968] kvm: 6873: cpu0 unhandled wrmsr: 0x6c2 data 0
[ 1170.137227] kvm: 6873: cpu0 unhandled wrmsr: 0x683 data 0
[ 1170.137502] kvm: 6873: cpu0 unhandled wrmsr: 0x6c3 data 0
[ 1170.137797] kvm: 6873: cpu0 unhandled wrmsr: 0x684 data 0
[ 1170.138061] kvm: 6873: cpu0 unhandled wrmsr: 0x6c4 data 0

MSR 0x345 is MSR_IA32_PERF_CAPABILITIES; the 0x6XX MSRs are LBR from/to.

Question: is something like this acceptable/make sense -- to only setup lbr values if the BTS is detected?

arch/x86/kernel/cpu/perf_event_intel.c +2 -1
index 166546e..7ba4afa 100644
@@ -1783,7 +1784,8 @@ __init int intel_pmu_init(void)
        memcpy(hw_cache_extra_regs, nehalem_hw_cache_extra_regs,
               sizeof(hw_cache_extra_regs));
 
-       intel_pmu_lbr_init_nhm();
+       if (x86_pmu.bts)
+           intel_pmu_lbr_init_nhm();
 
        x86_pmu.event_constraints = intel_nehalem_event_constraints;
        x86_pmu.pebs_constraints = intel_nehalem_pebs_event_constraints;

David

#1 lore raw
Quoted 39 lines · David Ahern

KVM recently added support for a version 2 PMU. When passing -cpu host as the CPU model for the guest we get an abnormal configuration from perf's perspective in that the guest identifies the processor as a Westmere or Nehalem (etc):

[    0.013998] Performance Events: Westmere events, Intel PMU driver.

but yet the processor does not have the debug store mechanisms (X86_FEATURE_DTES64 is not set) meaning there is no PEBS or BTS.

Right now the LBR init functions are run based on processor model which leads to attempts to write to LBR MSRs generating messages like:

[ 1170.125605] kvm: 6873: cpu0 unhandled rdmsr: 0x345
[ 1170.135354] kvm_set_msr_common: 54 callbacks suppressed
[ 1170.135612] kvm: 6873: cpu0 unhandled wrmsr: 0x680 data 0
[ 1170.135900] kvm: 6873: cpu0 unhandled wrmsr: 0x6c0 data 0
[ 1170.136163] kvm: 6873: cpu0 unhandled wrmsr: 0x681 data 0
[ 1170.136437] kvm: 6873: cpu0 unhandled wrmsr: 0x6c1 data 0
[ 1170.136703] kvm: 6873: cpu0 unhandled wrmsr: 0x682 data 0
[ 1170.136968] kvm: 6873: cpu0 unhandled wrmsr: 0x6c2 data 0
[ 1170.137227] kvm: 6873: cpu0 unhandled wrmsr: 0x683 data 0
[ 1170.137502] kvm: 6873: cpu0 unhandled wrmsr: 0x6c3 data 0
[ 1170.137797] kvm: 6873: cpu0 unhandled wrmsr: 0x684 data 0
[ 1170.138061] kvm: 6873: cpu0 unhandled wrmsr: 0x6c4 data 0

MSR 0x345 is MSR_IA32_PERF_CAPABILITIES; the 0x6XX MSRs are LBR from/to.

Question: is something like this acceptable/make sense -- to only setup lbr values if the BTS is detected?

arch/x86/kernel/cpu/perf_event_intel.c +0 -0
index 166546e..7ba4afa 100644

arch/x86/kernel/cpu/perf_event_intel.c __init int intel_pmu_init(void) · line 1789

Quoted 7 lines
arch/x86/kernel/cpu/perf_event_intel.c +2 -1
@@ -1783,7 +1784,8 @@ __init int intel_pmu_init(void)
        memcpy(hw_cache_extra_regs, nehalem_hw_cache_extra_regs,
               sizeof(hw_cache_extra_regs));
 
-       intel_pmu_lbr_init_nhm();
+       if (x86_pmu.bts)
+           intel_pmu_lbr_init_nhm();
 

Well, no. There is no connection between BTS and LBR and you're creating one.

Where is the wrmsr coming from? What we need to do is ensure that LBR is not touched if we don't actually use it.

Quoted 4 lines
arch/x86/kernel/cpu/perf_event_intel.c +0 -0
        x86_pmu.event_constraints = intel_nehalem_event_constraints;
        x86_pmu.pebs_constraints = intel_nehalem_pebs_event_constraints;

David

#2 lore raw
Quoted 1 line · Stephane Eranian

Well, no. There is no connection between BTS and LBR and you're creating one.

Ok. That was not clear to me from skimming the manual. Then should it be tied to X86_FEATURE_DTES64?

arch/x86/kernel/cpu/perf_event_intel_lbr.c +3 -0
index 520b426..1090ae6 100644
@@ -640,6 +640,9 @@ void intel_pmu_lbr_init_core(void)
 /* nehalem/westmere */
 void intel_pmu_lbr_init_nhm(void)
 {
+   if (!boot_cpu_has(X86_FEATURE_DTES64))
+       return;
+
    x86_pmu.lbr_nr     = 16;
    x86_pmu.lbr_tos    = MSR_LBR_TOS;
    x86_pmu.lbr_from   = MSR_LBR_NHM_FROM;
Quoted 2 lines

Where is the wrmsr coming from? What we need to do is ensure that LBR is not touched if we don't actually use it.

e.g., intel_pmu_lbr_init_nhm sets up lbr_nr, lbr_from, lbr_to and from, etc. Fhat I can tell intel_pmu_lbr_reset() gets invoked some where during the VM boot; I haven't traced how/when yet.

David

#3 lore raw
Quoted 8 lines · David Ahern
Quoted 3 lines · Stephane Eranian

Where is the wrmsr coming from? What we need to do is ensure that LBR is not
touched if we don't actually use it.

e.g., intel_pmu_lbr_init_nhm sets up lbr_nr, lbr_from, lbr_to and from, etc. Fhat I can tell intel_pmu_lbr_reset() gets invoked some where during the VM boot; I haven't traced how/when yet.

[    0.012998]  [<c04220ca>] intel_pmu_lbr_reset+0x2a/0xa0
[    0.012998]  [<c0423beb>] intel_pmu_cpu_starting+0x3b/0x100
[    0.012998]  [<c0424bd0>] ? allocate_shared_regs+0x20/0x50
[    0.012998]  [<c0932714>] x86_pmu_notifier+0xb3/0xc0
[    0.012998]  [<c0bed54c>] init_hw_perf_events+0x42c/0x456
[    0.012998]  [<c0403034>] do_one_initcall+0x34/0x170
[    0.012998]  [<c0bf2388>] ? native_smp_prepare_cpus+0x2b7/0x2f2
[    0.012998]  [<c0bed120>] ? check_bugs+0xf9/0xf9
[    0.012998]  [<c0be4833>] kernel_init+0x77/0x1a4
[    0.012998]  [<c0be47bc>] ? start_kernel+0x36d/0x36d
[    0.012998]  [<c094af7e>] kernel_thread_helper+0x6/0x10
#4 lore raw
In reply to David Ahern
Quoted 9 lines · David Ahern
Quoted 3 lines · Stephane Eranian

Well, no. There is no connection between BTS and LBR and you're creating one.

Ok. That was not clear to me from skimming the manual. Then should it be tied to X86_FEATURE_DTES64?

No, it is unrelated to the Debug Store.
There is really nothing you can use to figure that out, not event IA32_DEBUGCTL. I wish Intel had something in PERF_CAPABILITIES, but LBR was originally considered
as a debug feature, I think. To the point that it is described in the debug section not in
vol 3b with performance monitoring.

Quoted 28 lines
arch/x86/kernel/cpu/perf_event_intel_lbr.c +3 -0
index 520b426..1090ae6 100644
@@ -640,6 +640,9 @@ void intel_pmu_lbr_init_core(void)
  /* nehalem/westmere */
  void intel_pmu_lbr_init_nhm(void)
  {
+   if (!boot_cpu_has(X86_FEATURE_DTES64))
+       return;
+
    x86_pmu.lbr_nr     = 16;
    x86_pmu.lbr_tos    = MSR_LBR_TOS;
    x86_pmu.lbr_from   = MSR_LBR_NHM_FROM;
Quoted 6 lines

Where is the wrmsr coming from? What we need to do is ensure that LBR is not
touched if we don't actually use it.

e.g., intel_pmu_lbr_init_nhm sets up lbr_nr, lbr_from, lbr_to and from, etc. Fhat I can tell intel_pmu_lbr_reset() gets invoked some where during the VM boot; I haven't traced how/when yet.

David

maintainer
#6 lore raw
In reply to David Ahern
Quoted 5 lines · David Ahern

KVM recently added support for a version 2 PMU. When passing -cpu host as the CPU model for the guest we get an abnormal configuration from perf's perspective in that the guest identifies the processor as a Westmere or Nehalem (etc):

Quoted 8 lines
[    0.013998] Performance Events: Westmere events, Intel PMU driver.

but yet the processor does not have the debug store mechanisms (X86_FEATURE_DTES64 is not set) meaning there is no PEBS or BTS.

Right now the LBR init functions are run based on processor model which leads to attempts to write to LBR MSRs generating messages like:

Right, as Stephane already noted, LBR is not an CPUID enumerated feature, hence the only thing you can do is go by model. So if you set qemu -cpu host but fail to implement all the architectural MSRs that go with that model you get to keep the pieces.

Not really our problem.

maintainer
#7 lore raw
Quoted 4 lines · Peter Zijlstra

Right, as Stephane already noted, LBR is not an CPUID enumerated feature, hence the only thing you can do is go by model. So if you set qemu -cpu host but fail to implement all the architectural MSRs that go with that model you get to keep the pieces.

s/architectural//

They're called Model-Specific-Registers for a reason ;-)

#5 lore raw
In reply to Stephane Eranian
Quoted 7 lines · Stephane Eranian
Quoted 6 lines · David Ahern
Quoted 3 lines · Stephane Eranian

Well, no. There is no connection between BTS and LBR and you're creating one.

Quoted 5 lines
Quoted 3 lines

Ok. That was not clear to me from skimming the manual. Then should it be tied to X86_FEATURE_DTES64?

No, it is unrelated to the Debug Store.
There is really nothing you can use to figure that out, not event IA32_DEBUGCTL.

One way to do it would be to reread DEBUGCTL after writing and see if the bit really changed to 1. If not assume LBR is not available.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only