perf/x86: Don't write PEBS_ENABLED on KVM transitions
Subsystems (4)
Sean Christopherson
maintainerv2 cover perf/x86: Don't write PEBS_ENABLED on KVM transitions
Rework the handling of PEBS_ENABLED (and related PEBS MSRs) to *never* touch PEBS_ENABLED if the CPU provides PEBS isolation, in which case disabling counters via PERF_GLOBAL_CTRL is sufficient to prevent generation of unwanted PEBS records. For vCPUs without PEBS enabled, this saves upwards of 7 MSR writes on each roundtrip between the guest and host (KVM performs an immediate WRMSR to zero out PEBS_ENABLED if it's in the load list). For vCPUS with PEBS, this saves 3 MSR writes per roundtrip.
However, performance isn't the underlying motiviation. We (more accurately, Jim, Mingwei, and Stephane) have been chasing issues where PEBS_ENABLED bits can get "stuck" in a '1' state when running KVM guests while profiling the host with PEBS events. The working theory is that perf throttles PEBS events in NMI context, and thus clears bits in cpuc->pebs_enabled and PEBS_ENABLED, after generating the list of PMU MSRs to context switch but before VM-Entry. And so when the host's PEBS_ENABLED is loaded on VM-Exit, the CPU ends up with a stale PEBS_ENABLED that doesn't get reset until something triggers an explicit reload in perf.
Testing this against our "PEBS_ENABLED is stuck" reproducer is (still) a work in-progress (largely because the "reproducer" is currently "throw the kernel in a big test pool"), i.e. I don't know if this actually resolves the problems we are seeing. But even if it doesn't fully resolve our woes, it seems like a no-brainer improvement, and if we're missing something with respect to "stuck" PEBS_ENABLED, it'd be nice to get feedback/input asap.
Note, if the throttling theory is correct (which is looking unlikely at the moment), then there are likely more fixes that need to be done, e.g. for CPUs without isolation, and/or if PERF_GLOBAL_CTRL can be modified from NMI context too.
Patch 4 is a clean up that I posted as a standalone patch almost a year ago. I included it here because it's very related, and because I needed to refresh it anyways.
v2:
- "Load" the host value for the guest when an MSR should remain unchanged, instead of omitting the MSR from the list entirely, as KVM may need to _remove_ the MSR from the list. [Sashiko, Jim]
- Collect Jim's reviews. [Jim]
- Call out that the bug being fixed is theoretical at this point.
- Dropping PEBS_ENABLED from the lists save three MSR writes, not two, as KVM performs an explicit WRMSR prior to VM-Entry to guarantee PEBS is quiesced.
v1: https://lore.kernel.org/all/[email protected]
Sean Christopherson (4):
perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU
has isolation
perf/x86/intel: Don't context switch DS_AREA (and PEBS config) if PEBS
is unused
perf/x86/intel: Make @data a mandatory param for
intel_guest_get_msrs()
perf/x86: KVM: Have perf define a dedicated struct for getting guest
PEBS data
arch/x86/events/core.c | 5 ++- arch/x86/events/intel/core.c | 69 +++++++++++++++++++------------ arch/x86/events/perf_event.h | 3 +- arch/x86/include/asm/kvm_host.h | 9 ---- arch/x86/include/asm/perf_event.h | 12 +++++- arch/x86/kvm/vmx/pmu_intel.c | 20 +++++++-- arch/x86/kvm/vmx/vmx.c | 11 +++-- arch/x86/kvm/vmx/vmx.h | 2 +- 8 files changed, 82 insertions(+), 49 deletions(-)
base-commit: 6b802031877a995456c528095c41d1948546bf45
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Sean Christopherson
maintainerPATCH v2 1/4 perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU has isolation
When filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit, *never* insert an entry for PEBS_ENABLED if the CPU properly isolates PEBS events, in which case disabling counters via PERF_GLOBAL_CTRL is sufficient to prevent unwanted PEBS events in the guest (or host). Because perf loads PEBS_ENABLE with the unfiltered cpu_hw_events.pebs_enabled, i.e. with both host and guest masks, there is no need to load different values for the guest versus host, perf+KVM can and should simply control which counters are enabled/disabled via PERF_GLOBAL_CTRL.
Avoiding touching PEBS_ENABLED fixes a theorized bug where PEBS_ENABLED can end up with "stuck" bits if a PEBS event is throttled better generating the list and actually entering the guest (Intel CPUs can't arbtitrarily block NMIs). And stating the obvious, leaving PEBS_ENABLED as-is avoids three MSR writes on every VMX transition: one each on entry/exit, and one more explicit WRMSR to zero PEBS_ENABLED before VM-Entry (KVM assumes the only reason PEBS_ENABLED is in the load list is if the CPU lacks isolation and thus needs a quiescent period).
--- arch/x86/events/intel/core.c | 42 ++++++++++++++++++++---------------- 1 file changed, 23 insertions(+), 19 deletions(-)
arch/x86/events/intel/core.c +23 -19
@@ -4999,12 +4999,15 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;u64 intel_ctrl =hybrid (cpuc->pmu, intel_ctrl);u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable; -int global_ctrl, pebs_enable; +u64 guest_pebs_mask = pebs_mask & ~cpuc->intel_ctrl_host_mask; +int global_ctrl;/* * In addition to obeying exclude_guest/exclude_host, remove bits being * used for PEBS when running a guest, because PEBS writes to virtual -* addresses (not physical addresses). +* addresses (not physical addresses). If the guest wants to utilize +* PEBS, and PEBS can safely enabled in the guest, bits for the guest's +* PEBS-enabled counters will be OR'd back in as appropriate. */ *nr =0 ; global_ctrl = (*nr)++;
@@ -5051,24 +5054,25 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) }; } - pebs_enable = (*nr)++; - arr[pebs_enable] = (struct perf_guest_switch_msr ){ - .msr =MSR_IA32_PEBS_ENABLE , - .host = cpuc->pebs_enabled & ~cpuc->intel_ctrl_guest_mask, - .guest = pebs_mask & ~cpuc->intel_ctrl_host_mask & kvm_pmu->pebs_enable, - }; - -if (arr[pebs_enable].host) { -/* Disable guest PEBS if host PEBS is enabled. */ - arr[pebs_enable].guest =0 ; - }else { -/* Disable guest PEBS thoroughly for cross-mapped PEBS counters. */ - arr[pebs_enable].guest &= ~kvm_pmu->host_cross_mapped_mask; - arr[global_ctrl].guest &= ~kvm_pmu->host_cross_mapped_mask; -/* Set hw GLOBAL_CTRL bits for PEBS counter when it runs for guest */ - arr[global_ctrl].guest |= arr[pebs_enable].guest; - } +/* +* Disable counters where the guest PMC is different than the host PMC +* being used on behalf of the guest, as the PEBS record includes +* PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the +* wrong counter(s). Similarly, disallow PEBS in the guest if the host +* is using PEBS, to avoid bleeding host state into PEBS records. +*/ + guest_pebs_mask &= kvm_pmu->pebs_enable & ~kvm_pmu->host_cross_mapped_mask; +if (pebs_mask & ~cpuc->intel_ctrl_guest_mask) + guest_pebs_mask =0 ; +/* +* Do NOT mess with PEBS_ENABLED. As above, disabling counters via +* PERF_GLOBAL_CTRL is sufficient, and loading a stale PEBS_ENABLED, +* e.g. on VM-Exit, can put the system in a bad state. Simply enable +* counters in PERF_GLOBAL_CTRL, as perf load PEBS_ENABLED with the +* full value, i.e. perf *also* relies on PERF_GLOBAL_CTRL. +*/ + arr[global_ctrl].guest |= guest_pebs_mask;return arr; }
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Peter Zijlstra
maintainerRe: [PATCH v2 1/4] perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU has isolation
Quoted 9 lines · Sean Christopherson
When filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit, *never* insert an entry for
PEBS_ENABLEDif the CPU properly isolates PEBS events, in which case disabling counters viaPERF_GLOBAL_CTRLis sufficient to prevent unwanted PEBS events in the guest (or host). Because perf loadsPEBS_ENABLEwith the unfiltered cpu_hw_events.pebs_enabled, i.e. with both host and guest masks, there is no need to load different values for the guest versus host, perf+KVM can and should simply control which counters are enabled/disabled viaPERF_GLOBAL_CTRL.
Quoted 4 lines
Avoiding touching
PEBS_ENABLEDfixes a theorized bug wherePEBS_ENABLEDcan end up with "stuck" bits if a PEBS event is throttled better generating the list and actually entering the guest (Intel CPUs can't arbtitrarily block NMIs).
Supposedly writing 0 to GLOBAL_CTRL *should* serialize vs PMI on most chips IIRC. That is something perf itself relies on. Once we've cleared GLOBAL_CTRL we do not expect any PMIs.
Re: [PATCH v2 1/4] perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU has isolation
In reply to Sean ChristophersonQuoted 33 lines · Sean Christopherson
When filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit, *never* insert an entry for
PEBS_ENABLEDif the CPU properly isolates PEBS events, in which case disabling counters viaPERF_GLOBAL_CTRLis sufficient to prevent unwanted PEBS events in the guest (or host). Because perf loadsPEBS_ENABLEwith the unfiltered cpu_hw_events.pebs_enabled, i.e. with both host and guest masks, there is no need to load different values for the guest versus host, perf+KVM can and should simply control which counters are enabled/disabled viaPERF_GLOBAL_CTRL.Avoiding touching
PEBS_ENABLEDfixes a theorized bug wherePEBS_ENABLEDcan end up with "stuck" bits if a PEBS event is throttled better generating the list and actually entering the guest (Intel CPUs can't arbtitrarily block NMIs). And stating the obvious, leavingPEBS_ENABLEDas-is avoids three MSR writes on every VMX transition: one each on entry/exit, and one more explicit WRMSR to zeroPEBS_ENABLEDbefore VM-Entry (KVM assumes the only reasonPEBS_ENABLEDis in the load list is if the CPU lacks isolation and thus needs a quiescent period).Fixes c59a1f106f5c ("KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS")Cc Jim Mattson <[email protected]>Cc Mingwei Zhang <[email protected]>Cc Stephane Eranian <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/events/intel/core.c | 42 ++++++++++++++++++++---------------- 1 file changed, 23 insertions(+), 19 deletions(-)arch/x86/events/intel/core.c +0
-0
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5003
Quoted 6 lines
arch/x86/events/intel/core.c +2
-1@@ -4999,12 +4999,15 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;u64 intel_ctrl =hybrid (cpuc->pmu, intel_ctrl);u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable; -int global_ctrl, pebs_enable; +u64 guest_pebs_mask = pebs_mask & ~cpuc->intel_ctrl_host_mask; +int global_ctrl;
Is it worth noting somewhere that pebs_ept is not supported on any CPUs with PMU version < 5, where a single event can set two PEBS_ENABLE bits (cf. intel_pmu_pebs_enable)?
Quoted 31 lines
arch/x86/events/intel/core.c +3
-18/* * In addition to obeying exclude_guest/exclude_host, remove bits being * used for PEBS when running a guest, because PEBS writes to virtual -* addresses (not physical addresses). +* addresses (not physical addresses). If the guest wants to utilize +* PEBS, and PEBS can safely enabled in the guest, bits for the guest's +* PEBS-enabled counters will be OR'd back in as appropriate. */ *nr =0 ; global_ctrl = (*nr)++;@@ -5051,24 +5054,25 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) }; } - pebs_enable = (*nr)++; - arr[pebs_enable] = (struct perf_guest_switch_msr ){ - .msr =MSR_IA32_PEBS_ENABLE , - .host = cpuc->pebs_enabled & ~cpuc->intel_ctrl_guest_mask, - .guest = pebs_mask & ~cpuc->intel_ctrl_host_mask & kvm_pmu->pebs_enable, - }; - -if (arr[pebs_enable].host) { -/* Disable guest PEBS if host PEBS is enabled. */ - arr[pebs_enable].guest =0 ; - }else { -/* Disable guest PEBS thoroughly for cross-mapped PEBS counters. */ - arr[pebs_enable].guest &= ~kvm_pmu->host_cross_mapped_mask; - arr[global_ctrl].guest &= ~kvm_pmu->host_cross_mapped_mask; -/* Set hw GLOBAL_CTRL bits for PEBS counter when it runs for guest */ - arr[global_ctrl].guest |= arr[pebs_enable].guest; - }
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5066
Quoted 10 lines
arch/x86/events/intel/core.c +10
-0+/* +* Disable counters where the guest PMC is different than the host PMC +* being used on behalf of the guest, as the PEBS record includes +* PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the +* wrong counter(s). Similarly, disallow PEBS in the guest if the host +* is using PEBS, to avoid bleeding host state into PEBS records. +*/ + guest_pebs_mask &= kvm_pmu->pebs_enable & ~kvm_pmu->host_cross_mapped_mask; +if (pebs_mask & ~cpuc->intel_ctrl_guest_mask) + guest_pebs_mask =0 ;
I don't understand this clause. IIUC, it says that if we don't have any exclude-host PEBS events, then clear PEBS_ENABLE for the guest.
Yes, any guest-programmed PEBS event should be exclude-host, but if there is an inconsistency, shouldn't we apply a mask? What if there is only one exclude-host PEBS event, but there are two bits set in guest_pebs_mask?
Quoted 14 lines
arch/x86/events/intel/core.c +8
-1+/* +* Do NOT mess with PEBS_ENABLED. As above, disabling counters via +* PERF_GLOBAL_CTRL is sufficient, and loading a stale PEBS_ENABLED, +* e.g. on VM-Exit, can put the system in a bad state. Simply enable +* counters in PERF_GLOBAL_CTRL, as perf load PEBS_ENABLED with the +* full value, i.e. perf *also* relies on PERF_GLOBAL_CTRL. +*/ + arr[global_ctrl].guest |= guest_pebs_mask;return arr; } --2.54.0.545.g6539524ca2-goog
Re: [PATCH v2 1/4] perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU has isolation
Quoted 78 lines · Jim Mattson
Quoted 32 lines · Sean Christopherson
When filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit, *never* insert an entry for
PEBS_ENABLEDif the CPU properly isolates PEBS events, in which case disabling counters viaPERF_GLOBAL_CTRLis sufficient to prevent unwanted PEBS events in the guest (or host). Because perf loadsPEBS_ENABLEwith the unfiltered cpu_hw_events.pebs_enabled, i.e. with both host and guest masks, there is no need to load different values for the guest versus host, perf+KVM can and should simply control which counters are enabled/disabled viaPERF_GLOBAL_CTRL.Avoiding touching
PEBS_ENABLEDfixes a theorized bug wherePEBS_ENABLEDcan end up with "stuck" bits if a PEBS event is throttled better generating the list and actually entering the guest (Intel CPUs can't arbtitrarily block NMIs). And stating the obvious, leavingPEBS_ENABLEDas-is avoids three MSR writes on every VMX transition: one each on entry/exit, and one more explicit WRMSR to zeroPEBS_ENABLEDbefore VM-Entry (KVM assumes the only reasonPEBS_ENABLEDis in the load list is if the CPU lacks isolation and thus needs a quiescent period).Fixes c59a1f106f5c ("KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS")Cc Jim Mattson <[email protected]>Cc Mingwei Zhang <[email protected]>Cc Stephane Eranian <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/events/intel/core.c | 42 ++++++++++++++++++++---------------- 1 file changed, 23 insertions(+), 19 deletions(-)arch/x86/events/intel/core.c +0
-0arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5003
Quoted 6 lines
arch/x86/events/intel/core.c +2
-1@@ -4999,12 +4999,15 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;u64 intel_ctrl =hybrid (cpuc->pmu, intel_ctrl);u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable; -int global_ctrl, pebs_enable; +u64 guest_pebs_mask = pebs_mask & ~cpuc->intel_ctrl_host_mask; +int global_ctrl;Is it worth noting somewhere that
pebs_eptis not supported on any CPUs with PMU version < 5, where a single event can set twoPEBS_ENABLEbits (cf.intel_pmu_pebs_enable)?Quoted 35 lines
arch/x86/events/intel/core.c +7
-18/* * In addition to obeying exclude_guest/exclude_host, remove bits being * used for PEBS when running a guest, because PEBS writes to virtual -* addresses (not physical addresses). +* addresses (not physical addresses). If the guest wants to utilize +* PEBS, and PEBS can safely enabled in the guest, bits for the guest's +* PEBS-enabled counters will be OR'd back in as appropriate. */ *nr =0 ; global_ctrl = (*nr)++;@@ -5051,24 +5054,25 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) }; } - pebs_enable = (*nr)++; - arr[pebs_enable] = (struct perf_guest_switch_msr ){ - .msr =MSR_IA32_PEBS_ENABLE , - .host = cpuc->pebs_enabled & ~cpuc->intel_ctrl_guest_mask, - .guest = pebs_mask & ~cpuc->intel_ctrl_host_mask & kvm_pmu->pebs_enable, - }; - -if (arr[pebs_enable].host) { -/* Disable guest PEBS if host PEBS is enabled. */ - arr[pebs_enable].guest =0 ; - }else { -/* Disable guest PEBS thoroughly for cross-mapped PEBS counters. */ - arr[pebs_enable].guest &= ~kvm_pmu->host_cross_mapped_mask; - arr[global_ctrl].guest &= ~kvm_pmu->host_cross_mapped_mask; -/* Set hw GLOBAL_CTRL bits for PEBS counter when it runs for guest */ - arr[global_ctrl].guest |= arr[pebs_enable].guest; - } +/* +* Disable counters where the guest PMC is different than the host PMC +* being used on behalf of the guest, as the PEBS record includes +* PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
Quoted 8 lines
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5066
Quoted 6 lines
arch/x86/events/intel/core.c +6
-0+* wrong counter(s). Similarly, disallow PEBS in the guest if the host +* is using PEBS, to avoid bleeding host state into PEBS records. +*/ + guest_pebs_mask &= kvm_pmu->pebs_enable & ~kvm_pmu->host_cross_mapped_mask; +if (pebs_mask & ~cpuc->intel_ctrl_guest_mask) + guest_pebs_mask =0 ;I don't understand this clause. IIUC, it says that if we don't have any exclude-host PEBS events, then clear
PEBS_ENABLEfor the guest.
I suppose it says all guest PEBS events need to be disabled if there is any event using PEBS on host side, and it's clearing GLOBAL_CTRL instead of PEBS_ENABLE to disable guest PEBS events.
Quoted 20 lines
Yes, any guest-programmed PEBS event should be exclude-host, but if there is an inconsistency, shouldn't we apply a mask? What if there is only one exclude-host PEBS event, but there are two bits set in
guest_pebs_mask?Quoted 14 lines
arch/x86/events/intel/core.c +8
-1+/* +* Do NOT mess with PEBS_ENABLED. As above, disabling counters via +* PERF_GLOBAL_CTRL is sufficient, and loading a stale PEBS_ENABLED, +* e.g. on VM-Exit, can put the system in a bad state. Simply enable +* counters in PERF_GLOBAL_CTRL, as perf load PEBS_ENABLED with the +* full value, i.e. perf *also* relies on PERF_GLOBAL_CTRL. +*/ + arr[global_ctrl].guest |= guest_pebs_mask;return arr; } --2.54.0.545.g6539524ca2-goog
Sean Christopherson
maintainerRe: [PATCH v2 1/4] perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU has isolation
Quoted 4 lines · Dapeng Mi
Quoted 3 lines · Jim Mattson
Quoted 3 lines
arch/x86/events/intel/core.c | 42 ++++++++++++++++++++---------------- 1 file changed, 23 insertions(+), 19 deletions(-)
Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
arch/x86/events/intel/core.c +0
-0@@ -4999,12 +4999,15 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;Quoted 8 lines
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5003
Quoted 5 lines
arch/x86/events/intel/core.c +2
-1u64 intel_ctrl =hybrid (cpuc->pmu, intel_ctrl);u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable; -int global_ctrl, pebs_enable; +u64 guest_pebs_mask = pebs_mask & ~cpuc->intel_ctrl_host_mask; +int global_ctrl;Is it worth noting somewhere that
pebs_eptis not supported on any CPUs with PMU version < 5, where a single event can set twoPEBS_ENABLEbits (cf.intel_pmu_pebs_enable)?
Is that a hardware limitation, or a "perf hasn't added pebs_ept for PMU v5 yet" thing? I assume it's the latter?
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
arch/x86/events/intel/core.c +10
-0+/* +* Disable counters where the guest PMC is different than the host PMC +* being used on behalf of the guest, as the PEBS record includes +* PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the +* wrong counter(s). Similarly, disallow PEBS in the guest if the host +* is using PEBS, to avoid bleeding host state into PEBS records. +*/ + guest_pebs_mask &= kvm_pmu->pebs_enable & ~kvm_pmu->host_cross_mapped_mask; +if (pebs_mask & ~cpuc->intel_ctrl_guest_mask) + guest_pebs_mask =0 ;
Quoted 6 lines
Quoted 2 lines
I don't understand this clause. IIUC, it says that if we don't have any exclude-host PEBS events, then clear
PEBS_ENABLEfor the guest.I suppose it says all guest PEBS events need to be disabled if there is any event using PEBS on host side, and it's clearing
GLOBAL_CTRLinstead ofPEBS_ENABLEto disable guest PEBS events.
Yeah, but why disable _everything_?
Quoted 4 lines
Quoted 4 lines
Yes, any guest-programmed PEBS event should be exclude-host, but if there is an inconsistency, shouldn't we apply a mask? What if there is only one exclude-host PEBS event, but there are two bits set in
guest_pebs_mask?
I'm confused about this as well. The comment above about not bleeding host state into the PEBS records is my best guess (and it's probably not a very good guess) as to why the code does what it does. The changelog from commit 854250329c02 ("KVM: x86/pmu: Disable guest PEBS temporarily in two rare situations") just says:
The guest PEBS will be disabled when some users try to perf KVM and its user-space through the same PEBS facility
That doesn't entirely make sense to me though, because I would think disabling the host counters iva GLOBAL_CTRL would suffice. I.e. there's no need to disallow PEBS in the guest just because the host is also using PEBS. But I can't think of any other reason to fully disable PEBS.
FWIW, I was going off the previous code which effectively did:
if (cpuc->pebs_enabled & ~cpuc->intel_ctrl_guest_mask) {
/* Disable guest PEBS if host PEBS is enabled. */
arr[pebs_enable].guest = 0;
}One idea would be to add a FIXME, and then address the FIXME in a follow-up patch (in the same series)? And then see what breaks? :-)
Re: [PATCH v2 1/4] perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU has isolation
Quoted 58 lines · Sean Christopherson
Quoted 4 lines · Dapeng Mi
Quoted 3 lines · Jim Mattson
Quoted 3 lines
arch/x86/events/intel/core.c | 42 ++++++++++++++++++++---------------- 1 file changed, 23 insertions(+), 19 deletions(-)Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
arch/x86/events/intel/core.c +0
-0@@ -4999,12 +4999,15 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;Quoted 8 lines
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5003
Quoted 5 lines
arch/x86/events/intel/core.c +2
-1u64 intel_ctrl =hybrid (cpuc->pmu, intel_ctrl);u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable; -int global_ctrl, pebs_enable; +u64 guest_pebs_mask = pebs_mask & ~cpuc->intel_ctrl_host_mask; +int global_ctrl;Is it worth noting somewhere that
pebs_eptis not supported on any CPUs with PMU version < 5, where a single event can set twoPEBS_ENABLEbits (cf.intel_pmu_pebs_enable)?Is that a hardware limitation, or a "perf hasn't added
pebs_eptfor PMU v5 yet" thing? I assume it's the latter?Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
arch/x86/events/intel/core.c +10
-0+/* +* Disable counters where the guest PMC is different than the host PMC +* being used on behalf of the guest, as the PEBS record includes +* PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the +* wrong counter(s). Similarly, disallow PEBS in the guest if the host +* is using PEBS, to avoid bleeding host state into PEBS records. +*/ + guest_pebs_mask &= kvm_pmu->pebs_enable & ~kvm_pmu->host_cross_mapped_mask; +if (pebs_mask & ~cpuc->intel_ctrl_guest_mask) + guest_pebs_mask =0 ;Quoted 5 lines
Quoted 2 lines
I don't understand this clause. IIUC, it says that if we don't have any exclude-host PEBS events, then clear
PEBS_ENABLEfor the guest.I suppose it says all guest PEBS events need to be disabled if there is any event using PEBS on host side, and it's clearing
GLOBAL_CTRLinstead ofPEBS_ENABLEto disable guest PEBS events.Yeah, but why disable _everything_?
Quoted 4 lines
Quoted 4 lines
Yes, any guest-programmed PEBS event should be exclude-host, but if there is an inconsistency, shouldn't we apply a mask? What if there is only one exclude-host PEBS event, but there are two bits set in
guest_pebs_mask?I'm confused about this as well. The comment above about not bleeding host state into the PEBS records is my best guess (and it's probably not a very good guess) as to why the code does what it does. The changelog from commit
854250329c02("KVM: x86/pmu: Disable guest PEBS temporarily in two rare situations") just says:The guest PEBS will be disabled when some users try to perf KVM and its user-space through the same PEBS facility
That doesn't entirely make sense to me though, because I would think disabling the host counters iva
GLOBAL_CTRLwould suffice. I.e. there's no need to disallow PEBS in the guest just because the host is also using PEBS. But I can't think of any other reason to fully disable PEBS.FWIW, I was going off the previous code which effectively did:
Quoted 7 lines
if (cpuc->pebs_enabled & ~cpuc->intel_ctrl_guest_mask) { /* Disable guest PEBS if host PEBS is enabled. */ arr[pebs_enable].guest = 0; }One idea would be to add a FIXME, and then address the FIXME in a follow-up patch (in the same series)? And then see what breaks? :-)
I'm confused with this. does this go back to manipulate PEBS_ENABLE again?
I suppose what Jim mentioned is that the PEBS_LDLAT or PEBS_ST events could set multiple bits in PEBS_ENABLE MSR before Perfmon v5 as below code shows.
```
void intel_pmu_pebs_enable(struct perf_event *event)
{
......
if ((event->hw.flags & PERF_X86_EVENT_PEBS_LDLAT) && (x86_pmu.version < 5))
cpuc->pebs_enabled |= 1ULL << (hwc->idx + 32);
else if (event->hw.flags & PERF_X86_EVENT_PEBS_ST)
cpuc->pebs_enabled |= 1ULL << 63;
......
```This would lead to multiple bits set in pebs_mask and may cause incorrect results.
If so, maybe we can bit-and the pebs_mask with intel_ctrl to filter out these extra bits?
Sean Christopherson
maintainerPATCH v2 2/4 perf/x86/intel: Don't context switch DS_AREA (and PEBS config) if PEBS is unused
In reply to Sean ChristophersonWhen filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit, load the guest values for DS_AREA and (conditionally) MSR_PEBS_DATA_CFG if and only if PEBS will be active in the guest, i.e. only if a PEBS record may be generated while running the guest. As shown by the !pebs_ept path, it's perfectly safe to run with the host's DS_AREA, so long as PEBS-enabled counters are disabled via PERF_GLOBAL_CTRL.
Omitting DS_AREA and MSR_PEBS_DATA_CFG when PEBS is unused saves two MSR writes per MSR on each VMX transition, i.e. eliminates two/four pointless MSR writes on each VMX roundtrip when PEBS isn't being used by the guest.
--- arch/x86/events/intel/core.c | 39 +++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 14 deletions(-)
arch/x86/events/intel/core.c +25 -14
@@ -5037,23 +5037,14 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)return arr; } +/* +* If the guest won't use PEBS or the CPU doesn't support PEBS in the +* guest, then there's nothing more to do as disabling PMCs via +* PERF_GLOBAL_CTRL is sufficient on CPUs with guest/host isolation. +*/ if (!kvm_pmu || !x86_pmu.pebs_ept)return arr; - arr[(*nr)++] = (struct perf_guest_switch_msr ){ - .msr =MSR_IA32_DS_AREA , - .host = (unsigned long )cpuc->ds, - .guest = kvm_pmu->ds_area, - }; - -if (x86_pmu.intel_cap.pebs_baseline) { - arr[(*nr)++] = (struct perf_guest_switch_msr ){ - .msr =MSR_PEBS_DATA_CFG , - .host = cpuc->active_pebs_data_cfg, - .guest = kvm_pmu->pebs_data_cfg, - }; - } -/* * Disable counters where the guest PMC is different than the host PMC * being used on behalf of the guest, as the PEBS record includes
@@ -5065,6 +5056,26 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)if (pebs_mask & ~cpuc->intel_ctrl_guest_mask) guest_pebs_mask =0 ; +/* +* Context switch DS_AREA and PEBS_DATA_CFG if and only if PEBS will be +* active in the guest; if no records will be generated while the guest +* is running, then simply keep the host values resident in hardware. +*/ + arr[(*nr)++] = (struct perf_guest_switch_msr ){ + .msr =MSR_IA32_DS_AREA , + .host = (unsigned long )cpuc->ds, + .guest = guest_pebs_mask ? kvm_pmu->ds_area : (unsigned long )cpuc->ds, + }; + +if (x86_pmu.intel_cap.pebs_baseline) { + arr[(*nr)++] = (struct perf_guest_switch_msr ){ + .msr =MSR_PEBS_DATA_CFG , + .host = cpuc->active_pebs_data_cfg, + .guest = guest_pebs_mask ? kvm_pmu->pebs_data_cfg : + cpuc->active_pebs_data_cfg, + }; + } +/* * Do NOT mess with PEBS_ENABLED. As above, disabling counters via * PERF_GLOBAL_CTRL is sufficient, and loading a stale PEBS_ENABLED,
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Re: [PATCH v2 2/4] perf/x86/intel: Don't context switch DS_AREA (and PEBS config) if PEBS is unused
Quoted 58 lines · Sean Christopherson
When filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit, load the guest values for
DS_AREAand (conditionally)MSR_PEBS_DATA_CFGif and only if PEBS will be active in the guest, i.e. only if a PEBS record may be generated while running the guest. As shown by the !pebs_ept path, it's perfectly safe to run with the host'sDS_AREA, so long as PEBS-enabled counters are disabled viaPERF_GLOBAL_CTRL.Omitting
DS_AREAandMSR_PEBS_DATA_CFGwhen PEBS is unused saves two MSR writes per MSR on each VMX transition, i.e. eliminates two/four pointless MSR writes on each VMX roundtrip when PEBS isn't being used by the guest.Fixes c59a1f106f5c ("KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS")Cc Jim Mattson <[email protected]>Cc Mingwei Zhang <[email protected]>Cc Stephane Eranian <[email protected]>Reviewed-by Jim Mattson <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/events/intel/core.c | 39 +++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 14 deletions(-)arch/x86/events/intel/core.c +5
-14@@ -5037,23 +5037,14 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)return arr; } +/* +* If the guest won't use PEBS or the CPU doesn't support PEBS in the +* guest, then there's nothing more to do as disabling PMCs via +* PERF_GLOBAL_CTRL is sufficient on CPUs with guest/host isolation. +*/ if (!kvm_pmu || !x86_pmu.pebs_ept)return arr; - arr[(*nr)++] = (struct perf_guest_switch_msr ){ - .msr =MSR_IA32_DS_AREA , - .host = (unsigned long )cpuc->ds, - .guest = kvm_pmu->ds_area, - }; - -if (x86_pmu.intel_cap.pebs_baseline) { - arr[(*nr)++] = (struct perf_guest_switch_msr ){ - .msr =MSR_PEBS_DATA_CFG , - .host = cpuc->active_pebs_data_cfg, - .guest = kvm_pmu->pebs_data_cfg, - }; - } -/* * Disable counters where the guest PMC is different than the host PMC * being used on behalf of the guest, as the PEBS record includes @@ -5065,6 +5056,26 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)if (pebs_mask & ~cpuc->intel_ctrl_guest_mask) guest_pebs_mask =0 ;
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5068
Quoted 10 lines
arch/x86/events/intel/core.c +10
-0+/* +* Context switch DS_AREA and PEBS_DATA_CFG if and only if PEBS will be +* active in the guest; if no records will be generated while the guest +* is running, then simply keep the host values resident in hardware. +*/ + arr[(*nr)++] = (struct perf_guest_switch_msr ){ + .msr =MSR_IA32_DS_AREA , + .host = (unsigned long )cpuc->ds, + .guest = guest_pebs_mask ? kvm_pmu->ds_area : (unsigned long )cpuc->ds, + };
The code looks good to me.
one nit: people may be curious on why set same host value for guest if they don't notice that the same value on host and guest would lead to the MSRs would be removed from MSR load-list. Maybe add comments to further explain it.
Thanks.
Quoted 13 lines
arch/x86/events/intel/core.c +10
-0+ +if (x86_pmu.intel_cap.pebs_baseline) { + arr[(*nr)++] = (struct perf_guest_switch_msr ){ + .msr =MSR_PEBS_DATA_CFG , + .host = cpuc->active_pebs_data_cfg, + .guest = guest_pebs_mask ? kvm_pmu->pebs_data_cfg : + cpuc->active_pebs_data_cfg, + }; + } +/* * Do NOT mess with PEBS_ENABLED. As above, disabling counters via * PERF_GLOBAL_CTRL is sufficient, and loading a stale PEBS_ENABLED,
Sean Christopherson
maintainerRe: [PATCH v2 2/4] perf/x86/intel: Don't context switch DS_AREA (and PEBS config) if PEBS is unused
Quoted 17 lines · Dapeng Mi
Quoted 4 lines
arch/x86/events/intel/core.c +0
-0@@ -5065,6 +5056,26 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)if (pebs_mask & ~cpuc->intel_ctrl_guest_mask) guest_pebs_mask =0 ;arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5068
Quoted 10 lines
arch/x86/events/intel/core.c +10
-0+/* +* Context switch DS_AREA and PEBS_DATA_CFG if and only if PEBS will be +* active in the guest; if no records will be generated while the guest +* is running, then simply keep the host values resident in hardware. +*/ + arr[(*nr)++] = (struct perf_guest_switch_msr ){ + .msr =MSR_IA32_DS_AREA , + .host = (unsigned long )cpuc->ds, + .guest = guest_pebs_mask ? kvm_pmu->ds_area : (unsigned long )cpuc->ds, + };The code looks good to me.
Quoted 3 lines
one nit: people may be curious on why set same host value for guest if they don't notice that the same value on host and guest would lead to the MSRs would be removed from MSR load-list. Maybe add comments to further explain it.
+1. In the same comment (maybe on the function?), I'll also call out that the inclusion/exclusion of MSRs must only ever be based on constant host state, specifically because of the need to _remove_ MSRs from the list when KVM doesn't need to load a guest value on entry.
Sean Christopherson
maintainerPATCH v2 3/4 perf/x86/intel: Make @data a mandatory param for intel_guest_get_msrs()
In reply to Sean ChristophersonDrop "support" for passing a NULL @data/@kvm_pmu param when getting guest MSRs. KVM, the only in-tree user, unconditionally passes a non-NULL pointer, and carrying code that suggests @data may be NULL is confusing, e.g. incorrectly implies that there are scenarios where KVM doesn't pass a PMU context.
--- arch/x86/events/intel/core.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/events/intel/core.c +4 -4
@@ -5038,11 +5038,11 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) }/* -* If the guest won't use PEBS or the CPU doesn't support PEBS in the -* guest, then there's nothing more to do as disabling PMCs via -* PERF_GLOBAL_CTRL is sufficient on CPUs with guest/host isolation. +* If the CPU doesn't support PEBS in the guest, then there's nothing +* more to do as disabling PMCs via PERF_GLOBAL_CTRL is sufficient on +* CPUs with guest/host isolation. */ -if (!kvm_pmu || !x86_pmu.pebs_ept) +if (!x86_pmu.pebs_ept)return arr;/*
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Re: [PATCH v2 3/4] perf/x86/intel: Make @data a mandatory param for intel_guest_get_msrs()
Quoted 26 lines · Sean Christopherson
Drop "support" for passing a NULL @data/@kvm_pmu param when getting guest MSRs. KVM, the only in-tree user, unconditionally passes a non-NULL pointer, and carrying code that suggests @data may be NULL is confusing, e.g. incorrectly implies that there are scenarios where KVM doesn't pass a PMU context.
Fixes 8183a538cd95 ("KVM: x86/pmu: Add IA32_DS_AREA MSR emulation to support guest DS")Cc Jim Mattson <[email protected]>Cc Mingwei Zhang <[email protected]>Cc Stephane Eranian <[email protected]>Reviewed-by Jim Mattson <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/events/intel/core.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)arch/x86/events/intel/core.c +0
-2@@ -5038,11 +5038,11 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) }/* -* If the guest won't use PEBS or the CPU doesn't support PEBS in the -* guest, then there's nothing more to do as disabling PMCs via
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5048
Quoted 10 lines
arch/x86/events/intel/core.c +4
-2-* PERF_GLOBAL_CTRL is sufficient on CPUs with guest/host isolation. +* If the CPU doesn't support PEBS in the guest, then there's nothing +* more to do as disabling PMCs via PERF_GLOBAL_CTRL is sufficient on +* CPUs with guest/host isolation. */ -if (!kvm_pmu || !x86_pmu.pebs_ept) +if (!x86_pmu.pebs_ept)return arr;/*
LGTM. Thanks.
Sean Christopherson
maintainerPATCH v2 4/4 perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data
In reply to Sean ChristophersonHave perf define a struct for getting guest PEBS data from KVM instead of poking into the kvm_pmu structure. Passing in an entire "struct kvm_pmu" _as an opaque pointer_ to get at four fields is silly, especially since one of the fields exists purely to convey information to perf, i.e. isn't used by KVM.
Perf should also own its APIs, i.e. define what fields/data it needs, not rely on KVM to throw fields into data structures that effectively hold KVM-internal state.
Opportunistically rephrase the comment about cross-mapped counters to explain *why* PEBS needs to be disabled.
--- arch/x86/events/core.c | 5 +++-- arch/x86/events/intel/core.c | 14 +++++++------- arch/x86/events/perf_event.h | 3 ++- arch/x86/include/asm/kvm_host.h | 9 --------- arch/x86/include/asm/perf_event.h | 12 ++++++++++-- arch/x86/kvm/vmx/pmu_intel.c | 20 +++++++++++++++++--- arch/x86/kvm/vmx/vmx.c | 11 +++++++---- arch/x86/kvm/vmx/vmx.h | 2 +- 8 files changed, 47 insertions(+), 29 deletions(-)
arch/x86/events/core.c +3 -2
@@ -723,9 +723,10 @@ void x86_pmu_disable_all(void) } } -struct perf_guest_switch_msr *perf_guest_get_msrs (int *nr,void *data) +struct perf_guest_switch_msr *perf_guest_get_msrs (int *nr, +struct x86_guest_pebs *guest_pebs) { -return static_call (x86_pmu_guest_get_msrs)(nr, data); +return static_call (x86_pmu_guest_get_msrs)(nr, guest_pebs); }EXPORT_SYMBOL_FOR_KVM (perf_guest_get_msrs);
arch/x86/events/intel/core.c +7 -7
@@ -14,7 +14,6 @@#include <linux/slab.h> #include <linux/export.h> #include <linux/nmi.h> -#include <linux/kvm_host.h> #include <asm/cpufeature.h> #include <asm/debugreg.h>
@@ -4992,11 +4991,11 @@ static int intel_pmu_hw_config(struct perf_event *event)* when it uses {RD,WR}MSR, which should be handled by the KVM context, * specifically in the intel_pmu_{get,set}_msr(). */ -static struct perf_guest_switch_msr *intel_guest_get_msrs (int *nr,void *data) +static struct perf_guest_switch_msr *intel_guest_get_msrs (int *nr, +struct x86_guest_pebs *guest_pebs) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs; -struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;u64 intel_ctrl =hybrid (cpuc->pmu, intel_ctrl);u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;u64 guest_pebs_mask = pebs_mask & ~cpuc->intel_ctrl_host_mask;
@@ -5052,7 +5051,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)* wrong counter(s). Similarly, disallow PEBS in the guest if the host * is using PEBS, to avoid bleeding host state into PEBS records. */ - guest_pebs_mask &= kvm_pmu->pebs_enable & ~kvm_pmu->host_cross_mapped_mask; + guest_pebs_mask &= guest_pebs->enable & ~guest_pebs->cross_mapped_mask;if (pebs_mask & ~cpuc->intel_ctrl_guest_mask) guest_pebs_mask =0 ;
@@ -5064,14 +5063,14 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) arr[(*nr)++] = (struct perf_guest_switch_msr ){ .msr =MSR_IA32_DS_AREA , .host = (unsigned long )cpuc->ds, - .guest = guest_pebs_mask ? kvm_pmu->ds_area : (unsigned long )cpuc->ds, + .guest = guest_pebs_mask ? guest_pebs->ds_area : (unsigned long )cpuc->ds, };if (x86_pmu.intel_cap.pebs_baseline) { arr[(*nr)++] = (struct perf_guest_switch_msr ){ .msr =MSR_PEBS_DATA_CFG , .host = cpuc->active_pebs_data_cfg, - .guest = guest_pebs_mask ? kvm_pmu->pebs_data_cfg : + .guest = guest_pebs_mask ? guest_pebs->data_cfg : cpuc->active_pebs_data_cfg, }; }
@@ -5087,7 +5086,8 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)return arr; } -static struct perf_guest_switch_msr *core_guest_get_msrs (int *nr,void *data) +static struct perf_guest_switch_msr *core_guest_get_msrs (int *nr, +struct x86_guest_pebs *guest_pebs) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs;
arch/x86/events/perf_event.h +2 -1
@@ -1023,7 +1023,8 @@ struct x86_pmu {/* * Intel host/guest support (KVM) */ -struct perf_guest_switch_msr *(*guest_get_msrs)(int *nr,void *data); +struct perf_guest_switch_msr *(*guest_get_msrs)(int *nr, +struct x86_guest_pebs *guest_pebs);/* * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
arch/x86/include/asm/kvm_host.h +0 -9
@@ -600,15 +600,6 @@ struct kvm_pmu {u64 pebs_data_cfg;u64 pebs_data_cfg_rsvd; -/* -* If a guest counter is cross-mapped to host counter with different -* index, its PEBS capability will be temporarily disabled. -* -* The user should make sure that this mask is updated -* after disabling interrupts and before perf_guest_get_msrs(); -*/ -u64 host_cross_mapped_mask; -/* * The gate to release perf_events not marked in * pmc_in_use only once in a vcpu time slice.
arch/x86/include/asm/perf_event.h +10 -2
@@ -769,11 +769,19 @@ extern void perf_load_guest_lvtpc(u32 guest_lvtpc);extern void perf_put_guest_lvtpc (void );#endif +struct x86_guest_pebs { +u64 enable; +u64 ds_area; +u64 data_cfg; +u64 cross_mapped_mask; +};#if defined(CONFIG_PERF_EVENTS ) && defined(CONFIG_CPU_SUP_INTEL ) -extern struct perf_guest_switch_msr *perf_guest_get_msrs (int *nr,void *data); +extern struct perf_guest_switch_msr *perf_guest_get_msrs (int *nr, +struct x86_guest_pebs *guest_pebs);extern void x86_perf_get_lbr (struct x86_pmu_lbr *lbr);#else -struct perf_guest_switch_msr *perf_guest_get_msrs (int *nr,void *data); +struct perf_guest_switch_msr *perf_guest_get_msrs (int *nr, +struct x86_guest_pebs *guest_pebs);static inline void x86_perf_get_lbr (struct x86_pmu_lbr *lbr) {memset (lbr,0 ,sizeof (*lbr));
arch/x86/kvm/vmx/pmu_intel.c +17 -3
@@ -736,11 +736,24 @@ static void intel_pmu_cleanup(struct kvm_vcpu *vcpu)intel_pmu_release_guest_lbr_event (vcpu); } -void intel_pmu_cross_mapped_check (struct kvm_pmu *pmu) +u64 intel_pmu_get_cross_mapped_mask (struct kvm_pmu *pmu) { -struct kvm_pmc *pmc =NULL ; +u64 host_cross_mapped_mask; +struct kvm_pmc *pmc;int bit, hw_idx; +if (!(pmu->pebs_enable & pmu->global_ctrl)) +return 0 ; + +/* +* Provide a mask of counters that are cross-mapped between the guest +* and the host, i.e. where a guest PMC is mapped to a host PMC with a +* different index. PEBS records hold a PERF_GLOBAL_STATUS snapshot, +* and so PEBS-enabled counters need to hold the correct index so as +* not to confuse the guest. +*/ + host_cross_mapped_mask =0 ; +kvm_for_each_pmc (pmu, pmc, bit, (unsigned long *)&pmu->global_ctrl) {if (!pmc_is_locally_enabled (pmc) || !pmc_is_globally_enabled (pmc) || !pmc->perf_event)
@@ -752,8 +765,9 @@ void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu)*/ hw_idx = pmc->perf_event->hw.idx;if (hw_idx != pmc->idx && hw_idx >-1 ) - pmu->host_cross_mapped_mask |=BIT_ULL (hw_idx); + host_cross_mapped_mask |=BIT_ULL (hw_idx); } +return host_cross_mapped_mask; }static bool intel_pmu_is_mediated_pmu_supported (struct x86_pmu_capability *host_pmu)
arch/x86/kvm/vmx/vmx.c +7 -4
@@ -7313,12 +7313,15 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)if (kvm_vcpu_has_mediated_pmu (&vmx->vcpu))return ; - pmu->host_cross_mapped_mask =0 ; -if (pmu->pebs_enable & pmu->global_ctrl) -intel_pmu_cross_mapped_check (pmu); +struct x86_guest_pebs guest_pebs = { + .enable = pmu->pebs_enable, + .ds_area = pmu->ds_area, + .data_cfg = pmu->pebs_data_cfg, + .cross_mapped_mask =intel_pmu_get_cross_mapped_mask (pmu), + };/* Note, nr_msrs may be garbage if perf_guest_get_msrs() returns NULL. */ - msrs =perf_guest_get_msrs (&nr_msrs, (void *)pmu); + msrs =perf_guest_get_msrs (&nr_msrs, &guest_pebs);if (!msrs)return ;
arch/x86/kvm/vmx/vmx.h +1 -1
@@ -659,7 +659,7 @@ static __always_inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_vmx, vcpu); } -void intel_pmu_cross_mapped_check (struct kvm_pmu *pmu); +u64 intel_pmu_get_cross_mapped_mask (struct kvm_pmu *pmu);int intel_pmu_create_guest_lbr_event (struct kvm_vcpu *vcpu);void vmx_passthrough_lbr_msrs (struct kvm_vcpu *vcpu);
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Re: [PATCH v2 4/4] perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data
Quoted 11 lines · Sean Christopherson
Have perf define a struct for getting guest PEBS data from KVM instead of poking into the
kvm_pmustructure. Passing in an entire "structkvm_pmu"_asan opaquepointer_to get at four fields is silly, especially since one of the fields exists purely to convey information to perf, i.e. isn't used by KVM.Perf should also own its APIs, i.e. define what fields/data it needs, not rely on KVM to throw fields into data structures that effectively hold KVM-internal state.
Quoted 5 lines
Opportunistically rephrase the comment about cross-mapped counters to explain *why* PEBS needs to be disabled.
Reviewed-by Dapeng Mi <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
[email protected]
AI reviewRe: [PATCH v2 4/4] perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data
In reply to Sean ChristophersonSashiko AI review found 1 potential issue(s): - [Medium] Unconditional external function call to intel_pmu_get_cross_mapped_mask() introduced in the VMX hot path. --
Sashiko AI review found 1 potential issue(s):
- [Medium] Unconditional external function call to intel_pmu_get_cross_mapped_mask() introduced in the VMX hot path. --
commit bf596eb4f1ae159ccdd3b9e0517c0eaa4c80ce4a
Author: Sean Christopherson <[email protected]>
perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data
This commit introduces a dedicated struct (x86_guest_pebs) for passing guest PEBS data from KVM to perf. It replaces the previous method of passing an opaque pointer to kvm_pmu, allowing perf to define its own data requirements independently of KVM-internal state.
Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +0
-0@@ -7313,12 +7313,15 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)if (kvm_vcpu_has_mediated_pmu (&vmx->vcpu))return ;
arch/x86/kvm/vmx/vmx.c static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) · line 7321
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +6
-3- pmu->host_cross_mapped_mask =0 ; -if (pmu->pebs_enable & pmu->global_ctrl) -intel_pmu_cross_mapped_check (pmu); +struct x86_guest_pebs guest_pebs = { + .enable = pmu->pebs_enable, + .ds_area = pmu->ds_area, + .data_cfg = pmu->pebs_data_cfg, + .cross_mapped_mask =intel_pmu_get_cross_mapped_mask (pmu), + };
Does this introduce an unnecessary function call overhead on the VMX hot path?
Prior to this patch, the external function intel_pmu_cross_mapped_check() was conditionally called only when PEBS was enabled, which avoided an external function call in the vast majority of cases.
Because intel_pmu_get_cross_mapped_mask() is in a different compilation unit (pmu_intel.c), calling it unconditionally here results in a function call on every VM-entry.
Would it be better to restore the inline condition check, perhaps using a ternary operator when initializing cross_mapped_mask?
Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +1
-1/* Note, nr_msrs may be garbage if perf_guest_get_msrs() returns NULL. */ - msrs =perf_guest_get_msrs (&nr_msrs, (void *)pmu); + msrs =perf_guest_get_msrs (&nr_msrs, &guest_pebs);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Sean Christopherson
maintainerRe: [PATCH v2 4/4] perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data
Quoted 22 lines · On Thu, Apr 23, 2026, [email protected] wrote:
Sashiko AI review found 1 potential issue(s):
- [Medium] Unconditional external function call tointel_pmu_get_cross_mapped_mask()introduced in the VMX hot path. --commit
bf596eb4f1ae159ccdd3b9e0517c0eaa4c80ce4a
Author: Sean Christopherson <[email protected]>perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data
This commit introduces a dedicated struct (
x86_guest_pebs) for passing guest PEBS data from KVM to perf. It replaces the previous method of passing an opaque pointer tokvm_pmu, allowing perf to define its own data requirements independently of KVM-internal state.Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +0
-0@@ -7313,12 +7313,15 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)if (kvm_vcpu_has_mediated_pmu (&vmx->vcpu))return ;
Quoted 11 lines
arch/x86/kvm/vmx/vmx.c static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) · line 7321
Quoted 9 lines
arch/x86/kvm/vmx/vmx.c +6
-3- pmu->host_cross_mapped_mask =0 ; -if (pmu->pebs_enable & pmu->global_ctrl) -intel_pmu_cross_mapped_check (pmu); +struct x86_guest_pebs guest_pebs = { + .enable = pmu->pebs_enable, + .ds_area = pmu->ds_area, + .data_cfg = pmu->pebs_data_cfg, + .cross_mapped_mask =intel_pmu_get_cross_mapped_mask (pmu), + };Does this introduce an unnecessary function call overhead on the VMX hot path?
Probably. I'm not sure how much we care. Though what does matter and should be fixed is that intel_pmu_get_cross_mapped_mask() is a bit of a lie. It doesn't check _all_ PMCs, only those that are globally enabled and configured for PEBS.
I'll figure out a way to address both in the next version.
Quoted 10 lines
Prior to this patch, the external function
intel_pmu_cross_mapped_check()was conditionally called only when PEBS was enabled, which avoided an external function call in the vast majority of cases.Because
intel_pmu_get_cross_mapped_mask()is in a different compilation unit (pmu_intel.c), calling it unconditionally here results in a function call on every VM-entry.Would it be better to restore the inline condition check, perhaps using a ternary operator when initializing
cross_mapped_mask?
Re: [PATCH v2 4/4] perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data
In reply to Sean Christopherson.
Quoted 45 lines · Sean Christopherson
Have perf define a struct for getting guest PEBS data from KVM instead of poking into the
kvm_pmustructure. Passing in an entire "structkvm_pmu"_asan opaquepointer_to get at four fields is silly, especially since one of the fields exists purely to convey information to perf, i.e. isn't used by KVM.Perf should also own its APIs, i.e. define what fields/data it needs, not rely on KVM to throw fields into data structures that effectively hold KVM-internal state.
Opportunistically rephrase the comment about cross-mapped counters to explain *why* PEBS needs to be disabled.
Reviewed-by Dapeng Mi <[email protected]>Signed-off-by Sean Christopherson <[email protected]>---
...arch/x86/events/intel/core.c +2
-3@@ -14,7 +14,6 @@#include <linux/slab.h> #include <linux/export.h> #include <linux/nmi.h> -#include <linux/kvm_host.h> #include <asm/cpufeature.h> #include <asm/debugreg.h> @@ -4992,11 +4991,11 @@ static int intel_pmu_hw_config(struct perf_event *event)* when it uses {RD,WR}MSR, which should be handled by the KVM context, * specifically in the intel_pmu_{get,set}_msr(). */ -static struct perf_guest_switch_msr *intel_guest_get_msrs (int *nr,void *data) +static struct perf_guest_switch_msr *intel_guest_get_msrs (int *nr, +struct x86_guest_pebs *guest_pebs) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs; -struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;u64 intel_ctrl =hybrid (cpuc->pmu, intel_ctrl);u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;u64 guest_pebs_mask = pebs_mask & ~cpuc->intel_ctrl_host_mask;
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5054
Quoted 5 lines
arch/x86/events/intel/core.c +1
-1@@ -5052,7 +5051,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)* wrong counter(s). Similarly, disallow PEBS in the guest if the host * is using PEBS, to avoid bleeding host state into PEBS records. */ - guest_pebs_mask &= kvm_pmu->pebs_enable & ~kvm_pmu->host_cross_mapped_mask; + guest_pebs_mask &= guest_pebs->enable & ~guest_pebs->cross_mapped_mask;
It would be helpful to save this mask somewhere, so that it can be used when calculating guest_pebs_idxs in x86_pmu_handle_guest_pebs(). I think that code needs a fix similar to the one in commit 58f6217e5d01 ("perf/x86/intel: KVM: Mask PEBS_ENABLE loaded for guest with vCPU's value.").
Sean Christopherson
maintainerRe: [PATCH v2 4/4] perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data
Quoted 31 lines · Jim Mattson
Quoted 30 lines · Sean Christopherson
arch/x86/events/intel/core.c +2
-3@@ -14,7 +14,6 @@#include <linux/slab.h> #include <linux/export.h> #include <linux/nmi.h> -#include <linux/kvm_host.h> #include <asm/cpufeature.h> #include <asm/debugreg.h> @@ -4992,11 +4991,11 @@ static int intel_pmu_hw_config(struct perf_event *event)* when it uses {RD,WR}MSR, which should be handled by the KVM context, * specifically in the intel_pmu_{get,set}_msr(). */ -static struct perf_guest_switch_msr *intel_guest_get_msrs (int *nr,void *data) +static struct perf_guest_switch_msr *intel_guest_get_msrs (int *nr, +struct x86_guest_pebs *guest_pebs) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs; -struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;u64 intel_ctrl =hybrid (cpuc->pmu, intel_ctrl);u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;u64 guest_pebs_mask = pebs_mask & ~cpuc->intel_ctrl_host_mask;@@ -5052,7 +5051,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)* wrong counter(s). Similarly, disallow PEBS in the guest if the host * is using PEBS, to avoid bleeding host state into PEBS records. */
Quoted 8 lines
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5054
Quoted 2 lines
arch/x86/events/intel/core.c +1
-1- guest_pebs_mask &= kvm_pmu->pebs_enable & ~kvm_pmu->host_cross_mapped_mask; + guest_pebs_mask &= guest_pebs->enable & ~guest_pebs->cross_mapped_mask;It would be helpful to save this mask somewhere, so that it can be used when calculating
guest_pebs_idxsinx86_pmu_handle_guest_pebs(). I think that code needs a fix similar to the one in commit58f6217e5d01("perf/x86/intel: KVM: MaskPEBS_ENABLEloaded for guest with vCPU's value.").
Blech. This all feels like a losing game of whack-a-mole. Proxying the PMU through perf is a mediocre approximation for non-PEBS events, and it seems like it's downright awful for PEBS. Ideally, we'd just rip out all of the perf-based PEBS virtualization support, and only support PEBS through the mediated PMU. :-/
Absent drastic measures though, saving the effective guest_pebs_enable in the per-CPU tracking does seem like the least awful approach. Though I don't quite understand why we can't use GLOBAL_STATUS for x86_pmu_handle_guest_pebs(). I.e. what happens if x86_pmu_handle_guest_pebs() only processes counters that actually got marked as overflowing?
Regardless, I'm not going to try and address that mess in this series. AFAICT, it's not urgent, and I don't want to snowball into a broader cleanup.
Re: [PATCH v2 4/4] perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data
Quoted 47 lines · Sean Christopherson
Quoted 31 lines · Jim Mattson
Quoted 30 lines · Sean Christopherson
arch/x86/events/intel/core.c +2
-3@@ -14,7 +14,6 @@#include <linux/slab.h> #include <linux/export.h> #include <linux/nmi.h> -#include <linux/kvm_host.h> #include <asm/cpufeature.h> #include <asm/debugreg.h> @@ -4992,11 +4991,11 @@ static int intel_pmu_hw_config(struct perf_event *event)* when it uses {RD,WR}MSR, which should be handled by the KVM context, * specifically in the intel_pmu_{get,set}_msr(). */ -static struct perf_guest_switch_msr *intel_guest_get_msrs (int *nr,void *data) +static struct perf_guest_switch_msr *intel_guest_get_msrs (int *nr, +struct x86_guest_pebs *guest_pebs) {struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events);struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs; -struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;u64 intel_ctrl =hybrid (cpuc->pmu, intel_ctrl);u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;u64 guest_pebs_mask = pebs_mask & ~cpuc->intel_ctrl_host_mask;@@ -5052,7 +5051,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)* wrong counter(s). Similarly, disallow PEBS in the guest if the host * is using PEBS, to avoid bleeding host state into PEBS records. */ Quoted 8 lines
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5054
Quoted 2 lines
arch/x86/events/intel/core.c +1
-1- guest_pebs_mask &= kvm_pmu->pebs_enable & ~kvm_pmu->host_cross_mapped_mask; + guest_pebs_mask &= guest_pebs->enable & ~guest_pebs->cross_mapped_mask;It would be helpful to save this mask somewhere, so that it can be used when calculating
guest_pebs_idxsinx86_pmu_handle_guest_pebs(). I think that code needs a fix similar to the one in commit58f6217e5d01("perf/x86/intel: KVM: MaskPEBS_ENABLEloaded for guest with vCPU's value.").Blech. This all feels like a losing game of whack-a-mole. Proxying the PMU through perf is a mediocre approximation for non-PEBS events, and it seems like it's downright awful for PEBS. Ideally, we'd just rip out all of the perf-based PEBS virtualization support, and only support PEBS through the mediated PMU. :-/
Quoted 5 lines
Absent drastic measures though, saving the effective
guest_pebs_enablein the per-CPU tracking does seem like the least awful approach. Though I don't quite understand why we can't useGLOBAL_STATUSforx86_pmu_handle_guest_pebs(). I.e. what happens ifx86_pmu_handle_guest_pebs()only processes counters that actually got marked as overflowing?
x86_pmu_handle_guest_pebs() is called in the path where we are handling GLOBAL_STATUS bit 62 (GLOBAL_STATUS_BUFFER_OVF_BIT). Individual PEBS PMCs are not configured to raise PMI on overflow.
Moreover, I believe kvm_arch_pmi_in_guest() returns a false positive if perf is using NMI for PMI, and a PMI occurs while between kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ) and kvm_after_interrupt(vcpu).
Quoted 2 lines
Regardless, I'm not going to try and address that mess in this series. AFAICT, it's not urgent, and I don't want to snowball into a broader cleanup.
I'll let you know if our instrumentation changes that sense of urgency. :)
Sean Christopherson
maintainerRe: [PATCH v2 4/4] perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data
Quoted 27 lines · Jim Mattson
Quoted 20 lines · Sean Christopherson
Quoted 4 lines · Jim Mattson
Quoted 4 lines
arch/x86/events/intel/core.c +0
-0@@ -5052,7 +5051,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)* wrong counter(s). Similarly, disallow PEBS in the guest if the host * is using PEBS, to avoid bleeding host state into PEBS records. */ Quoted 8 lines
arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5054
Quoted 2 lines
arch/x86/events/intel/core.c +1
-1- guest_pebs_mask &= kvm_pmu->pebs_enable & ~kvm_pmu->host_cross_mapped_mask; + guest_pebs_mask &= guest_pebs->enable & ~guest_pebs->cross_mapped_mask;It would be helpful to save this mask somewhere, so that it can be used when calculating
guest_pebs_idxsinx86_pmu_handle_guest_pebs(). I think that code needs a fix similar to the one in commit58f6217e5d01("perf/x86/intel: KVM: MaskPEBS_ENABLEloaded for guest with vCPU's value.").Blech. This all feels like a losing game of whack-a-mole. Proxying the PMU through perf is a mediocre approximation for non-PEBS events, and it seems like it's downright awful for PEBS. Ideally, we'd just rip out all of the perf-based PEBS virtualization support, and only support PEBS through the mediated PMU. :-/
Quoted 5 lines
Absent drastic measures though, saving the effective
guest_pebs_enablein the per-CPU tracking does seem like the least awful approach. Though I don't quite understand why we can't useGLOBAL_STATUSforx86_pmu_handle_guest_pebs(). I.e. what happens ifx86_pmu_handle_guest_pebs()only processes counters that actually got marked as overflowing?
Quoted 3 lines
x86_pmu_handle_guest_pebs()is called in the path where we are handlingGLOBAL_STATUSbit 62 (GLOBAL_STATUS_BUFFER_OVF_BIT). Individual PEBS PMCs are not configured to raise PMI on overflow.
Right, but aren't the per-counter OVF bits still set in GLOBAL_STATUS? Ah, no, at least that's not supposed to happen, per commit 8077eca079a2 ("perf/x86/pebs:
Add workaround for broken OVFL status on HSW+"). Well that's lame.
Quoted 4 lines · Sean Christopherson
...
- DroppingPEBS_ENABLEDfrom the lists save three MSR writes, not two, as KVM performs an explicit WRMSR prior to VM-Entry to guarantee PEBS is quiesced.
Is the wrmsrq(MSR_IA32_PEBS_ENABLED, 0) in add_atomic_switch_msr() necessary when the CPU has PEBS isolation?
Peter Zijlstra
maintainerQuoted 11 lines · Sean Christopherson
Testing this against our "
PEBS_ENABLEDis stuck" reproducer is (still) a work in-progress (largely because the "reproducer" is currently "throw the kernel in a big test pool"), i.e. I don't know if this actually resolves the problems we are seeing. But even if it doesn't fully resolve our woes, it seems like a no-brainer improvement, and if we're missing something with respect to "stuck"PEBS_ENABLED, it'd be nice to get feedback/input asap.Note, if the throttling theory is correct (which is looking unlikely at the moment), then there are likely more fixes that need to be done, e.g. for CPUs without isolation, and/or if
PERF_GLOBAL_CTRLcan be modified from NMI context too.
Throttle does: pmu->stop() := x86_pmu_stop() -> intel_pmu_disable_event()
Which in turn should:
x86_pmu_disable_event()
wrmsrq(config_base, config & ~EN);
x86_pmu_pebs_disable() := intel_pmu_pebs_disable()
wrmsr(PEBS_ENABLE, pebs_enabled & ~(1<<idx));So that's just the counter EN bit and PEBS_ENABLED cleared. However, if this is from PMI, then the PMI handler should also update GLOBAL_CTRL -- provided it wasn't 0.
See intel_pmu_handle_irq():
if (pmu_enabled)
__intel_pmu_enable_all()
wrmsrq(GLOBAL_CTRL, intel_ctrl);
Quoted 25 lines · Peter Zijlstra
Quoted 11 lines · Sean Christopherson
Testing this against our "
PEBS_ENABLEDis stuck" reproducer is (still) a work in-progress (largely because the "reproducer" is currently "throw the kernel in a big test pool"), i.e. I don't know if this actually resolves the problems we are seeing. But even if it doesn't fully resolve our woes, it seems like a no-brainer improvement, and if we're missing something with respect to "stuck"PEBS_ENABLED, it'd be nice to get feedback/input asap.Note, if the throttling theory is correct (which is looking unlikely at the moment), then there are likely more fixes that need to be done, e.g. for CPUs without isolation, and/or if
PERF_GLOBAL_CTRLcan be modified from NMI context too.Throttle does:
pmu->stop():=x86_pmu_stop()->intel_pmu_disable_event()Which in turn should:
x86_pmu_disable_event() wrmsrq(config_base, config & ~EN); x86_pmu_pebs_disable() := intel_pmu_pebs_disable() wrmsr(PEBS_ENABLE, pebs_enabled & ~(1<<idx));So that's just the counter EN bit and
PEBS_ENABLEDcleared. However, if this is from PMI, then the PMI handler should also updateGLOBAL_CTRL-- provided it wasn't 0.
Quoted 6 lines
See
intel_pmu_handle_irq():if (
pmu_enabled)
__intel_pmu_enable_all()wrmsrq(GLOBAL_CTRL, intel_ctrl);
Yes, currently all valid bits in GLOBAL_CTRL would be set by default on Intel platforms. IIUC, this issue looks more like a race condition between Perf and KVM.
1. KVM saves the value of host PEBS_ENABLE before VM-entry.
2. PMI is triggered and interrupts the upcoming VM-entry. PEBS events are throttled and PEBS_ENABLE MSR is updated in the PMI handler, then the KVM saved host PEBS_ENABLE value gets stale.
3. VM entry continues and then the next VM-exit occurs, the stale PEBS_ENABLE value is restored.
4. The PEBS_ENABLE MSR keeps the stale value until next write.
Seems an alternative way to fix this issue is to disable the PMU (Clearing GLOBAL_CTRL) before KVM saving the PMU MSRs?
Thanks.
Peter Zijlstra
maintainerQuoted 43 lines · Dapeng
Quoted 25 lines · Peter Zijlstra
Quoted 11 lines · Sean Christopherson
Testing this against our "
PEBS_ENABLEDis stuck" reproducer is (still) a work in-progress (largely because the "reproducer" is currently "throw the kernel in a big test pool"), i.e. I don't know if this actually resolves the problems we are seeing. But even if it doesn't fully resolve our woes, it seems like a no-brainer improvement, and if we're missing something with respect to "stuck"PEBS_ENABLED, it'd be nice to get feedback/input asap.Note, if the throttling theory is correct (which is looking unlikely at the moment), then there are likely more fixes that need to be done, e.g. for CPUs without isolation, and/or if
PERF_GLOBAL_CTRLcan be modified from NMI context too.Throttle does:
pmu->stop():=x86_pmu_stop()->intel_pmu_disable_event()Which in turn should:
x86_pmu_disable_event() wrmsrq(config_base, config & ~EN); x86_pmu_pebs_disable() := intel_pmu_pebs_disable() wrmsr(PEBS_ENABLE, pebs_enabled & ~(1<<idx));So that's just the counter EN bit and
PEBS_ENABLEDcleared. However, if this is from PMI, then the PMI handler should also updateGLOBAL_CTRL-- provided it wasn't 0.Quoted 6 lines
See
intel_pmu_handle_irq():if (
pmu_enabled)
__intel_pmu_enable_all()wrmsrq(GLOBAL_CTRL, intel_ctrl);Yes, currently all valid bits in
GLOBAL_CTRLwould be set by default on Intel platforms. IIUC, this issue looks more like a race condition between Perf and KVM.1. KVM saves the value of host
PEBS_ENABLEbefore VM-entry.2. PMI is triggered and interrupts the upcoming VM-entry. PEBS events are throttled and
PEBS_ENABLEMSR is updated in the PMI handler, then the KVM saved hostPEBS_ENABLEvalue gets stale.
Quoted 7 lines
3. VM entry continues and then the next VM-exit occurs, the stale
PEBS_ENABLEvalue is restored.4. The PEBS_ENABLE MSR keeps the stale value until next write.
Seems an alternative way to fix this issue is to disable the PMU (Clearing
GLOBAL_CTRL) before KVM saving the PMU MSRs?
Yes, that would seem a prudent thing to do.