Skip to contentSkip to conversation

perf/x86: Don't write PEBS_ENABLED on KVM transitions

PATCH Superseded Sean Christopherson 5 participants 29 messages May 8 – Sep 18

Subsystems (4)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

v3 cover perf/x86: Don't write PEBS_ENABLED on KVM transitions

Rework the handling of PEBS_ENABLED (and related PEBS MSRs) to *never* touch PEBS_ENABLED if the CPU provides PEBS isolation, in which case disabling counters via PERF_GLOBAL_CTRL is sufficient to prevent generation of unwanted PEBS records. For vCPUs without PEBS enabled, this saves upwards of 7 MSR writes on each roundtrip between the guest and host (KVM performs an immediate WRMSR to zero out PEBS_ENABLED if it's in the load list). For vCPUS with PEBS, this saves 3 MSR writes per roundtrip.

E.g. without PEBS activity in the host, for a guest with a vPMU, this reduces the roundtrip time for a fastpath exit from ~1120 => ~860 cycles on EMR. With host PEBS active, the reduction is ~1450 => ~900 cycles.

However, performance isn't the underlying motiviation (well, at least, it didn't start that way). Jim, Mingwei, and Stephane have been chasing issues where PEBS_ENABLED bits can get "stuck" in a '1' state when running KVM guests while profiling the host with PEBS events. The working theory is that perf throttles PEBS events in NMI context, and thus clears bits in cpuc->pebs_enabled and PEBS_ENABLED, after generating the list of PMU MSRs to context switch but before VM-Entry. And so when the host's PEBS_ENABLED is loaded on VM-Exit, the CPU ends up with a stale PEBS_ENABLED that doesn't get reset until something triggers an explicit reload in perf.

Note, as Peter pointed out, more than likely KVM needs to zero PERF_GLOBAL_CTRL before invoking perf_guest_get_msrs(), as that's the only way to guarantee stable output. I deliberately didn't include that here, as I want to keep this series focused on PEBS. I also wanted to let Jim and company bottom out on their investigation (still ongoing) before pursuing fixes that we'll probably want to send to stable@.

v3:
- Ensure guest PEBS_ENABLE is a subset of intel_ctrl. [Jim]
- Rename intel_ctrl_{guest,host}_mask to be less confusing. [Jim]
- Do even more cleanup of the cross-mapped handling, and specifically avoid overhead when PEBS isn't in use. [Sashiko]
- Leave behind a FIXME regarding the "disable guest PEBS if host is using PEBS" code. I still don't know for sure why that restriction is in place, and I'm too scared too change it. :-)

v2:
- https://lore.kernel.org/all/[email protected]
- "Load" the host value for the guest when an MSR should remain unchanged, instead of omitting the MSR from the list entirely, as KVM may need to _remove_ the MSR from the list. [Sashiko, Jim]
- Collect Jim's reviews. [Jim]
- Call out that the bug being fixed is theoretical at this point.
- Dropping PEBS_ENABLED from the lists save three MSR writes, not two, as KVM performs an explicit WRMSR prior to VM-Entry to guarantee PEBS is quiesced.

v1: https://lore.kernel.org/all/[email protected]

Sean Christopherson (9):
perf/x86/intel: Ensure guest PEBS path doesn't set unwanted
PERF_GLOBAL_CTRL bits
perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU
has isolation
perf/x86/intel: Don't context switch DS_AREA (and PEBS config) if PEBS
is unused
perf/x86/intel: Make @data a mandatory param for
intel_guest_get_msrs()
perf/x86/intel: Invert names of intel_ctrl_{guest,host}_mask perf/x86: KVM: Have perf define a dedicated struct for getting guest
PEBS data
perf/x86/intel: KVM: Handle cross-mapped PEBS PMCs entirely within KVM
KVM: VMX: Drop a redundant pmu->global_ctrl check when processing
pebs_enable
KVM: VMX: Only tell perf to enable PEBS counters for fully enabled
PMCs

 arch/x86/events/core.c            |  5 +-
 arch/x86/events/intel/core.c      | 92 +++++++++++++++++++------------
 arch/x86/events/intel/lbr.c       |  2 +-
 arch/x86/events/perf_event.h      |  7 ++-
 arch/x86/include/asm/kvm_host.h   |  9 ---
 arch/x86/include/asm/perf_event.h | 11 +++-
 arch/x86/kvm/vmx/pmu_intel.c      | 28 +++++++---
 arch/x86/kvm/vmx/vmx.c            | 10 ++--
 arch/x86/kvm/vmx/vmx.h            | 15 ++++-
 9 files changed, 114 insertions(+), 65 deletions(-)

base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731

Signature · 1 line
2.54.0.563.g4f69b47b94-goog
author maintainer
#1 lore raw

PATCH v3 1/9 perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

When reinstating PEBS counters into PERF_GLOBAL_CTRL for a KVM guest, mask the value with perf's desired/original PERF_GLOBAL_CTRL value to ensure KVM doesn't unintentionally enable counters. This _should_ be a nop, as arr[pebs_enable].guest is derived from cpuc->pebs_enabled, which should be a subset of x86_pmu.intel_ctrl, but paranoia is cheap in this case.

---
 arch/x86/events/intel/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -1
index d9488ade0f8e..b70dc35fcceb 100644
@@ -5066,7 +5066,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 		arr[pebs_enable].guest &= ~kvm_pmu->host_cross_mapped_mask;
 		arr[global_ctrl].guest &= ~kvm_pmu->host_cross_mapped_mask;
 		/* Set hw GLOBAL_CTRL bits for PEBS counter when it runs for guest */
-		arr[global_ctrl].guest |= arr[pebs_enable].guest;
+		arr[global_ctrl].guest |= intel_ctrl & arr[pebs_enable].guest;
 	}
 
 	return arr;
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
#2 lore raw

Re: [PATCH v3 1/9] perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The host PEBS collision check fails to mask non-counter bits, incorrectly disabling guest PEBS for…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The host PEBS collision check fails to mask non-counter bits, incorrectly disabling guest PEBS for events like Store Latency. --

Note: The format of this report has been altered due to recitation restrictions. The original patch code is not quoted directly, and findings are provided as a free-form summary.

commit 39adc53b360e88595a3be6fc59b18b048d9e9921
Author: Sean Christopherson <[email protected]>

perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

This commit masks the pebs counter values being reinstated into the global control msr for a guest with perf's desired value. This is done to ensure the hypervisor doesn't unintentionally enable counters.

In intel_guest_get_msrs(), the host pebs collision check evaluates the host field of the pebs_enable array entry to determine if the host is actively using pebs. If it is, guest pebs is disabled.

Is it possible for this check to falsely detect a host collision due to non-counter bits being set?

When a guest configures a store latency pebs event, a backing perf event is created. In intel_pmu_pebs_enable(), this sets both the counter index bit and bit 63 in cpuc->pebs_enabled.

Because the host field is calculated as cpuc->pebs_enabled masked against the inverse of cpuc->intel_ctrl_guest_mask, and the guest mask exclusively tracks valid counter indices (never including bit 63), bit 63 remains set in the host field.

Consequently, the check for whether the host is actively using pebs will evaluate to true even when only the guest is using pebs. This introduces a regression where guest pebs is completely disabled, silently breaking guest store latency events.

Should the host pebs usage evaluation also be masked by intel_ctrl to prevent non-counter bits from triggering a false collision?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

Re: [PATCH v3 1/9] perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

Quoted 33 lines · On 5/9/2026 7:40 AM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The host PEBS collision check fails to mask non-counter bits, incorrectly disabling guest PEBS for events like Store Latency. --

Note: The format of this report has been altered due to recitation restrictions. The original patch code is not quoted directly, and findings are provided as a free-form summary.

commit 39adc53b360e88595a3be6fc59b18b048d9e9921
Author: Sean Christopherson <[email protected]>

perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

This commit masks the pebs counter values being reinstated into the global control msr for a guest with perf's desired value. This is done to ensure the hypervisor doesn't unintentionally enable counters.

In intel_guest_get_msrs(), the host pebs collision check evaluates the host field of the pebs_enable array entry to determine if the host is actively using pebs. If it is, guest pebs is disabled.

Is it possible for this check to falsely detect a host collision due to non-counter bits being set?

When a guest configures a store latency pebs event, a backing perf event is created. In intel_pmu_pebs_enable(), this sets both the counter index bit and bit 63 in cpuc->pebs_enabled.

Because the host field is calculated as cpuc->pebs_enabled masked against the inverse of cpuc->intel_ctrl_guest_mask, and the guest mask exclusively tracks valid counter indices (never including bit 63), bit 63 remains set in the host field.

Quoted 7 lines

Consequently, the check for whether the host is actively using pebs will evaluate to true even when only the guest is using pebs. This introduces a regression where guest pebs is completely disabled, silently breaking guest store latency events.

Should the host pebs usage evaluation also be masked by intel_ctrl to prevent non-counter bits from triggering a false collision?

Hmm, I suspect if the issue could happen on real hardwares.

The function intel_pmu_pebs_enable() indeed sets extra bits into cpuc->pebs_enabled for load latency and specific store events, like below code shows.

```

......

    if ((event->hw.flags & PERF_X86_EVENT_PEBS_LDLAT) && (x86_pmu.version < 5))
        cpuc->pebs_enabled |= 1ULL << (hwc->idx + 32);
    else if (event->hw.flags & PERF_X86_EVENT_PEBS_ST)
        cpuc->pebs_enabled |= 1ULL << 63;

......

```

But these 2 cases should only be hit on quite old platforms (prior to Icelake). On these platforms, only GP counters support PEBS sampling and pebs_capable would be set PEBS_COUNTER_MASK, and so these extra bits would be filtered out by the pebs_capable and pebs_mask won't really contain these extra bits.

Anyway, we could optimize the code further like below and thoroughly filter away these extra bits. (only building, not test on real HW)

arch/x86/events/intel/core.c +0 -0
index 854881b5e696..6eee7636a822 100644
@@ -4997,7 +4997,7 @@ static struct perf_guest_switch_msr

*intel_guest_get_msrs(int *nr,
        struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs; u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
-       u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable; +       u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable & intel_ctrl; u64 guest_pebs_mask;
        int global_ctrl;

@@ -5049,7 +5049,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr,
         * the guest wants to use for PEBS, (c) are not excluded from counting
         * in the guest, and (d) _are_ excluded from counting in the host. */
-       guest_pebs_mask = pebs_mask & intel_ctrl & guest_pebs->enable & +       guest_pebs_mask = pebs_mask & guest_pebs->enable & ~cpuc->intel_ctrl_exclude_guest_mask & cpuc->intel_ctrl_exclude_host_mask;

Quoted 1 line
author maintainer
#4 lore raw

Re: [PATCH v3 1/9] perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

Quoted 40 lines · Dapeng Mi
Quoted 38 lines · On 5/9/2026 7:40 AM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The host PEBS collision check fails to mask non-counter bits, incorrectly disabling guest PEBS for events like Store Latency. --

Note: The format of this report has been altered due to recitation restrictions. The original patch code is not quoted directly, and findings are provided as a free-form summary.

commit 39adc53b360e88595a3be6fc59b18b048d9e9921
Author: Sean Christopherson <[email protected]>

perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

This commit masks the pebs counter values being reinstated into the global control msr for a guest with perf's desired value. This is done to ensure the hypervisor doesn't unintentionally enable counters.

In intel_guest_get_msrs(), the host pebs collision check evaluates the host field of the pebs_enable array entry to determine if the host is actively using pebs. If it is, guest pebs is disabled.

Is it possible for this check to falsely detect a host collision due to non-counter bits being set?

When a guest configures a store latency pebs event, a backing perf event is created. In intel_pmu_pebs_enable(), this sets both the counter index bit and bit 63 in cpuc->pebs_enabled.

Because the host field is calculated as cpuc->pebs_enabled masked against the inverse of cpuc->intel_ctrl_guest_mask, and the guest mask exclusively tracks valid counter indices (never including bit 63), bit 63 remains set in the host field.

Consequently, the check for whether the host is actively using pebs will evaluate to true even when only the guest is using pebs. This introduces a regression where guest pebs is completely disabled, silently breaking guest store latency events.

Quoted 4 lines
Quoted 2 lines

Should the host pebs usage evaluation also be masked by intel_ctrl to prevent non-counter bits from triggering a false collision?

Hmm, I suspect if the issue could happen on real hardwares.

OMG, it took me _so_ long to understand what Sashiko was complaining about. I'm pretty sure I read Sashiko's response like four times, and it didn't click until I read your response.

In case anyone is feeling as dense as me, Sashiko is saying this:

	/*
	 * FIXME: Allow guest and host usage of PEBS events to co-exist instead
	 *        of disabling guest PEBS entirely if the host is using PEBS.
	 *        What exactly goes wrong if guest and host are using PEBS is
	 *        unknown.
	 */
	if (pebs_mask & ~cpuc->intel_ctrl_exclude_host_mask)
		guest_pebs_mask = 0;

should probably be masked by intel_ctrl as well.

Quoted 17 lines

The function intel_pmu_pebs_enable() indeed sets extra bits into cpuc->pebs_enabled for load latency and specific store events, like below code shows.

```

......

    if ((event->hw.flags & PERF_X86_EVENT_PEBS_LDLAT) && (x86_pmu.version < 5))
        cpuc->pebs_enabled |= 1ULL << (hwc->idx + 32);
    else if (event->hw.flags & PERF_X86_EVENT_PEBS_ST)
        cpuc->pebs_enabled |= 1ULL << 63;

......

```
Quoted 5 lines

But these 2 cases should only be hit on quite old platforms (prior to Icelake). On these platforms, only GP counters support PEBS sampling and pebs_capable would be set PEBS_COUNTER_MASK, and so these extra bits would be filtered out by the pebs_capable and pebs_mask won't really contain these extra bits.

Thanks for digging into this! I was staring and just going "huh!?" over and over in my head :-)

Quoted 2 lines

Anyway, we could optimize the code further like below and thoroughly filter away these extra bits. (only building, not test on real HW)

Hmm, I think I'd rather figure out what it would take to drop the FIXME entirely. And if we need to keep the check, I'm a-ok risking false positives until we have a better understanding of why the check exists.

Quoted 25 lines
arch/x86/events/intel/core.c +2 -2
index 854881b5e696..6eee7636a822 100644
@@ -4997,7 +4997,7 @@ static struct perf_guest_switch_msr
 *intel_guest_get_msrs(int *nr,
         struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
         struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs;
         u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
-       u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;
+       u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable & intel_ctrl;
         u64 guest_pebs_mask;
         int global_ctrl;
@@ -5049,7 +5049,7 @@ static struct perf_guest_switch_msr
 *intel_guest_get_msrs(int *nr,
          * the guest wants to use for PEBS, (c) are not excluded from counting
          * in the guest, and (d) _are_ excluded from counting in the host.
          */
-       guest_pebs_mask = pebs_mask & intel_ctrl & guest_pebs->enable &
+       guest_pebs_mask = pebs_mask & guest_pebs->enable &

                          ~cpuc->intel_ctrl_exclude_guest_mask & cpuc->intel_ctrl_exclude_host_mask;

Quoted 1 line
#5 lore raw

Re: [PATCH v3 1/9] perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

Quoted 58 lines · Sean Christopherson
Quoted 34 lines · Dapeng Mi
Quoted 33 lines · On 5/9/2026 7:40 AM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The host PEBS collision check fails to mask non-counter bits, incorrectly disabling guest PEBS for events like Store Latency. --

Note: The format of this report has been altered due to recitation restrictions. The original patch code is not quoted directly, and findings are provided as a free-form summary.

commit 39adc53b360e88595a3be6fc59b18b048d9e9921
Author: Sean Christopherson <[email protected]>

perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

This commit masks the pebs counter values being reinstated into the global control msr for a guest with perf's desired value. This is done to ensure the hypervisor doesn't unintentionally enable counters.

In intel_guest_get_msrs(), the host pebs collision check evaluates the host field of the pebs_enable array entry to determine if the host is actively using pebs. If it is, guest pebs is disabled.

Is it possible for this check to falsely detect a host collision due to non-counter bits being set?

When a guest configures a store latency pebs event, a backing perf event is created. In intel_pmu_pebs_enable(), this sets both the counter index bit and bit 63 in cpuc->pebs_enabled.

Because the host field is calculated as cpuc->pebs_enabled masked against the inverse of cpuc->intel_ctrl_guest_mask, and the guest mask exclusively tracks valid counter indices (never including bit 63), bit 63 remains set in the host field.

Quoted 8 lines
Quoted 7 lines

Consequently, the check for whether the host is actively using pebs will evaluate to true even when only the guest is using pebs. This introduces a regression where guest pebs is completely disabled, silently breaking guest store latency events.

Should the host pebs usage evaluation also be masked by intel_ctrl to prevent non-counter bits from triggering a false collision?

Hmm, I suspect if the issue could happen on real hardwares.

OMG, it took me _so_ long to understand what Sashiko was complaining about. I'm pretty sure I read Sashiko's response like four times, and it didn't click until I read your response.

In case anyone is feeling as dense as me, Sashiko is saying this:

	/*
	 * FIXME: Allow guest and host usage of PEBS events to co-exist instead
	 *        of disabling guest PEBS entirely if the host is using PEBS.
	 *        What exactly goes wrong if guest and host are using PEBS is
	 *        unknown.
	 */
	if (pebs_mask & ~cpuc->intel_ctrl_exclude_host_mask)
		guest_pebs_mask = 0;
Quoted 1 line

should probably be masked by intel_ctrl as well.

Yes, the comment of Sashiko is some kind of misleading. Strictly speaking, the comment should be made for the next patch 2/9 instead of this patch.

Quoted 18 lines
Quoted 18 lines

The function intel_pmu_pebs_enable() indeed sets extra bits into cpuc->pebs_enabled for load latency and specific store events, like below code shows.

```

......

    if ((event->hw.flags & PERF_X86_EVENT_PEBS_LDLAT) && (x86_pmu.version < 5))
        cpuc->pebs_enabled |= 1ULL << (hwc->idx + 32);
    else if (event->hw.flags & PERF_X86_EVENT_PEBS_ST)
        cpuc->pebs_enabled |= 1ULL << 63;

......

```
Quoted 7 lines
Quoted 5 lines

But these 2 cases should only be hit on quite old platforms (prior to Icelake). On these platforms, only GP counters support PEBS sampling and pebs_capable would be set PEBS_COUNTER_MASK, and so these extra bits would be filtered out by the pebs_capable and pebs_mask won't really contain these extra bits.

Thanks for digging into this! I was staring and just going "huh!?" over and over in my head :-)

:)

Quoted 6 lines
Quoted 3 lines

Anyway, we could optimize the code further like below and thoroughly filter away these extra bits. (only building, not test on real HW)

Hmm, I think I'd rather figure out what it would take to drop the FIXME entirely. And if we need to keep the check, I'm a-ok risking false positives until we have a better understanding of why the check exists.

Kan Liang should be the best man who knows the history, but he has left Intel. I would check this with other guys internally and look at if they know the reason.

Thanks.

Quoted 25 lines
Quoted 25 lines
arch/x86/events/intel/core.c +2 -2
index 854881b5e696..6eee7636a822 100644
@@ -4997,7 +4997,7 @@ static struct perf_guest_switch_msr
 *intel_guest_get_msrs(int *nr,
         struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
         struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs;
         u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
-       u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;
+       u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable & intel_ctrl;
         u64 guest_pebs_mask;
         int global_ctrl;
@@ -5049,7 +5049,7 @@ static struct perf_guest_switch_msr
 *intel_guest_get_msrs(int *nr,
          * the guest wants to use for PEBS, (c) are not excluded from counting
          * in the guest, and (d) _are_ excluded from counting in the host.
          */
-       guest_pebs_mask = pebs_mask & intel_ctrl & guest_pebs->enable &
+       guest_pebs_mask = pebs_mask & guest_pebs->enable &

                          ~cpuc->intel_ctrl_exclude_guest_mask & cpuc->intel_ctrl_exclude_host_mask;

#6 lore raw

Re: [PATCH v3 1/9] perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

Quoted 91 lines · Dapeng
Quoted 58 lines · Sean Christopherson
Quoted 34 lines · Dapeng Mi
Quoted 33 lines · On 5/9/2026 7:40 AM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The host PEBS collision check fails to mask non-counter bits, incorrectly disabling guest PEBS for events like Store Latency. --

Note: The format of this report has been altered due to recitation restrictions. The original patch code is not quoted directly, and findings are provided as a free-form summary.

commit 39adc53b360e88595a3be6fc59b18b048d9e9921
Author: Sean Christopherson <[email protected]>

perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

This commit masks the pebs counter values being reinstated into the global control msr for a guest with perf's desired value. This is done to ensure the hypervisor doesn't unintentionally enable counters.

In intel_guest_get_msrs(), the host pebs collision check evaluates the host field of the pebs_enable array entry to determine if the host is actively using pebs. If it is, guest pebs is disabled.

Is it possible for this check to falsely detect a host collision due to non-counter bits being set?

When a guest configures a store latency pebs event, a backing perf event is created. In intel_pmu_pebs_enable(), this sets both the counter index bit and bit 63 in cpuc->pebs_enabled.

Because the host field is calculated as cpuc->pebs_enabled masked against the inverse of cpuc->intel_ctrl_guest_mask, and the guest mask exclusively tracks valid counter indices (never including bit 63), bit 63 remains set in the host field.

Quoted 8 lines
Quoted 7 lines

Consequently, the check for whether the host is actively using pebs will evaluate to true even when only the guest is using pebs. This introduces a regression where guest pebs is completely disabled, silently breaking guest store latency events.

Should the host pebs usage evaluation also be masked by intel_ctrl to prevent non-counter bits from triggering a false collision?

Hmm, I suspect if the issue could happen on real hardwares.

OMG, it took me _so_ long to understand what Sashiko was complaining about. I'm pretty sure I read Sashiko's response like four times, and it didn't click until I read your response.

In case anyone is feeling as dense as me, Sashiko is saying this:

	/*
	 * FIXME: Allow guest and host usage of PEBS events to co-exist instead
	 *        of disabling guest PEBS entirely if the host is using PEBS.
	 *        What exactly goes wrong if guest and host are using PEBS is
	 *        unknown.
	 */
	if (pebs_mask & ~cpuc->intel_ctrl_exclude_host_mask)
		guest_pebs_mask = 0;
Quoted 1 line

should probably be masked by intel_ctrl as well.

Yes, the comment of Sashiko is some kind of misleading. Strictly speaking, the comment should be made for the next patch 2/9 instead of this patch.

Quoted 17 lines
Quoted 17 lines

The function intel_pmu_pebs_enable() indeed sets extra bits into cpuc->pebs_enabled for load latency and specific store events, like below code shows.

```

......

    if ((event->hw.flags & PERF_X86_EVENT_PEBS_LDLAT) && (x86_pmu.version < 5))
        cpuc->pebs_enabled |= 1ULL << (hwc->idx + 32);
    else if (event->hw.flags & PERF_X86_EVENT_PEBS_ST)
        cpuc->pebs_enabled |= 1ULL << 63;

......

```
Quoted 7 lines
Quoted 5 lines

But these 2 cases should only be hit on quite old platforms (prior to Icelake). On these platforms, only GP counters support PEBS sampling and pebs_capable would be set PEBS_COUNTER_MASK, and so these extra bits would be filtered out by the pebs_capable and pebs_mask won't really contain these extra bits.

Thanks for digging into this! I was staring and just going "huh!?" over and over in my head :-)

:)

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Anyway, we could optimize the code further like below and thoroughly filter away these extra bits. (only building, not test on real HW)

Hmm, I think I'd rather figure out what it would take to drop the FIXME entirely. And if we need to keep the check, I'm a-ok risking false positives until we have a better understanding of why the check exists.

Kan Liang should be the best man who knows the history, but he has left Intel. I would check this with other guys internally and look at if they know the reason.

Just checked the commit history, It looks the reason of disabling guest PEBS here is to avoid breaking the host !exclude_guest PEBS event sampling. Host may create !exclude_guest event to profile both root and non-root mode, but it's conflicted with guest PEBS sampling in non-root mode. So guest PEBS sampling has to be disabled if there are active host PEBS events. Thanks.

Quoted 28 lines

Thanks.

Quoted 24 lines
Quoted 24 lines
arch/x86/events/intel/core.c +2 -2
index 854881b5e696..6eee7636a822 100644
@@ -4997,7 +4997,7 @@ static struct perf_guest_switch_msr
 *intel_guest_get_msrs(int *nr,
         struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
         struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs;
         u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
-       u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;
+       u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable & intel_ctrl;
         u64 guest_pebs_mask;
         int global_ctrl;
@@ -5049,7 +5049,7 @@ static struct perf_guest_switch_msr
 *intel_guest_get_msrs(int *nr,
          * the guest wants to use for PEBS, (c) are not excluded from counting
          * in the guest, and (d) _are_ excluded from counting in the host.
          */
-       guest_pebs_mask = pebs_mask & intel_ctrl & guest_pebs->enable &
+       guest_pebs_mask = pebs_mask & guest_pebs->enable &

                          ~cpuc->intel_ctrl_exclude_guest_mask & cpuc->intel_ctrl_exclude_host_mask;

author maintainer
#7 lore raw

Re: [PATCH v3 1/9] perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

Quoted 11 lines · Dapeng Mi
Quoted 9 lines · Dapeng
Quoted 5 lines · Sean Christopherson
Quoted 2 lines

Anyway, we could optimize the code further like below and thoroughly filter away these extra bits. (only building, not test on real HW)

Hmm, I think I'd rather figure out what it would take to drop the FIXME entirely. And if we need to keep the check, I'm a-ok risking false positives until we have a better understanding of why the check exists.

Kan Liang should be the best man who knows the history, but he has left Intel. I would check this with other guys internally and look at if they know the reason.

Quoted 6 lines

Just checked the commit history, It looks the reason of disabling guest PEBS here is to avoid breaking the host !exclude_guest PEBS event sampling. Host may create !exclude_guest event to profile both root and non-root mode, but it's conflicted with guest PEBS sampling in non-root mode. So guest PEBS sampling has to be disabled if there are active host PEBS events. Thanks.

But that doesn't make any sense, because the host can't use PEBS to sample non-root mode due to the DS area being virtually addressed. It's entirely possible that was pure oversight, given that commit 58f6217e5d01 ("perf/x86/intel: KVM: Mask PEBS_ENABLE loaded for guest with vCPU's value.") was necessary.

But I'm still not convinced: if that _really_ was the motivation, then I would hope that scenario was at least smoke tested, and any testing whatsoever would have crashed the guest.

Though given how buggy this code has been, I suppose it's plausible there was simply testing.

#8 lore raw

Re: [PATCH v3 1/9] perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

Quoted 22 lines · Sean Christopherson
Quoted 10 lines · Dapeng Mi
Quoted 9 lines · Dapeng
Quoted 5 lines · Sean Christopherson
Quoted 2 lines

Anyway, we could optimize the code further like below and thoroughly filter away these extra bits. (only building, not test on real HW)

Hmm, I think I'd rather figure out what it would take to drop the FIXME entirely. And if we need to keep the check, I'm a-ok risking false positives until we have a better understanding of why the check exists.

Kan Liang should be the best man who knows the history, but he has left Intel. I would check this with other guys internally and look at if they know the reason.

Quoted 6 lines

Just checked the commit history, It looks the reason of disabling guest PEBS here is to avoid breaking the host !exclude_guest PEBS event sampling. Host may create !exclude_guest event to profile both root and non-root mode, but it's conflicted with guest PEBS sampling in non-root mode. So guest PEBS sampling has to be disabled if there are active host PEBS events. Thanks.

But that doesn't make any sense, because the host can't use PEBS to sample non-root mode due to the DS area being virtually addressed. It's entirely possible that was pure oversight, given that commit 58f6217e5d01 ("perf/x86/intel: KVM: Mask PEBS_ENABLE loaded for guest with vCPU's value.") was necessary.

Quoted 6 lines

But I'm still not convinced: if that _really_ was the motivation, then I would hope that scenario was at least smoke tested, and any testing whatsoever would have crashed the guest.

Though given how buggy this code has been, I suppose it's plausible there was simply testing.

Yes, you are right. It's impossible to support PEBS sampling for both root and non-root mode for the legacy PEBS, and that's one main reason why the arch-PEBS is introduced.

But the conclusion is still correct. Before the commit 58f6217e5d01 ("perf/x86/intel: KVM: Mask PEBS_ENABLE loaded for guest with vCPU's value."), if users create a !exclude_guest event on host, the event would be enabled in IA32_PEBS_ENABLE after the vm-entry and it would cause guest crash if no such global guest PEBS disabling. That's the reason why the guest PEBS is globally disabled here if there are active PEBS events on host.

After introducing the commit 58f6217e5d01 ("perf/x86/intel: KVM: Mask PEBS_ENABLE loaded for guest with vCPU's value."), it looks this global guest PEBS disabling becomes unnecessary any more.

Thanks.

Quoted 1 line
#9 lore raw

Re: [PATCH v3 1/9] perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

In reply to Sean Christopherson
Quoted 58 lines · Sean Christopherson
Quoted 34 lines · Dapeng Mi
Quoted 33 lines · On 5/9/2026 7:40 AM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The host PEBS collision check fails to mask non-counter bits, incorrectly disabling guest PEBS for events like Store Latency. --

Note: The format of this report has been altered due to recitation restrictions. The original patch code is not quoted directly, and findings are provided as a free-form summary.

commit 39adc53b360e88595a3be6fc59b18b048d9e9921
Author: Sean Christopherson <[email protected]>

perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

This commit masks the pebs counter values being reinstated into the global control msr for a guest with perf's desired value. This is done to ensure the hypervisor doesn't unintentionally enable counters.

In intel_guest_get_msrs(), the host pebs collision check evaluates the host field of the pebs_enable array entry to determine if the host is actively using pebs. If it is, guest pebs is disabled.

Is it possible for this check to falsely detect a host collision due to non-counter bits being set?

When a guest configures a store latency pebs event, a backing perf event is created. In intel_pmu_pebs_enable(), this sets both the counter index bit and bit 63 in cpuc->pebs_enabled.

Because the host field is calculated as cpuc->pebs_enabled masked against the inverse of cpuc->intel_ctrl_guest_mask, and the guest mask exclusively tracks valid counter indices (never including bit 63), bit 63 remains set in the host field.

Quoted 8 lines
Quoted 7 lines

Consequently, the check for whether the host is actively using pebs will evaluate to true even when only the guest is using pebs. This introduces a regression where guest pebs is completely disabled, silently breaking guest store latency events.

Should the host pebs usage evaluation also be masked by intel_ctrl to prevent non-counter bits from triggering a false collision?

Hmm, I suspect if the issue could happen on real hardwares.

OMG, it took me _so_ long to understand what Sashiko was complaining about. I'm pretty sure I read Sashiko's response like four times, and it didn't click until I read your response.

In case anyone is feeling as dense as me, Sashiko is saying this:

	/*
	 * FIXME: Allow guest and host usage of PEBS events to co-exist instead
	 *        of disabling guest PEBS entirely if the host is using PEBS.
	 *        What exactly goes wrong if guest and host are using PEBS is
	 *        unknown.
	 */
	if (pebs_mask & ~cpuc->intel_ctrl_exclude_host_mask)
		guest_pebs_mask = 0;
Quoted 1 line

should probably be masked by intel_ctrl as well.

Sean, do you think we need to fix this flaw? Not sure if it's worthy to post a new version to fix it ...

Thanks.

Quoted 56 lines
Quoted 18 lines

The function intel_pmu_pebs_enable() indeed sets extra bits into cpuc->pebs_enabled for load latency and specific store events, like below code shows.

```

......

    if ((event->hw.flags & PERF_X86_EVENT_PEBS_LDLAT) && (x86_pmu.version < 5))
        cpuc->pebs_enabled |= 1ULL << (hwc->idx + 32);
    else if (event->hw.flags & PERF_X86_EVENT_PEBS_ST)
        cpuc->pebs_enabled |= 1ULL << 63;

......

```
Quoted 5 lines

But these 2 cases should only be hit on quite old platforms (prior to Icelake). On these platforms, only GP counters support PEBS sampling and pebs_capable would be set PEBS_COUNTER_MASK, and so these extra bits would be filtered out by the pebs_capable and pebs_mask won't really contain these extra bits.

Thanks for digging into this! I was staring and just going "huh!?" over and over in my head :-)

Quoted 2 lines

Anyway, we could optimize the code further like below and thoroughly filter away these extra bits. (only building, not test on real HW)

Hmm, I think I'd rather figure out what it would take to drop the FIXME entirely. And if we need to keep the check, I'm a-ok risking false positives until we have a better understanding of why the check exists.

Quoted 24 lines
arch/x86/events/intel/core.c +2 -2
index 854881b5e696..6eee7636a822 100644
@@ -4997,7 +4997,7 @@ static struct perf_guest_switch_msr
 *intel_guest_get_msrs(int *nr,
         struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
         struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs;
         u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
-       u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;
+       u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable & intel_ctrl;
         u64 guest_pebs_mask;
         int global_ctrl;
@@ -5049,7 +5049,7 @@ static struct perf_guest_switch_msr
 *intel_guest_get_msrs(int *nr,
          * the guest wants to use for PEBS, (c) are not excluded from counting
          * in the guest, and (d) _are_ excluded from counting in the host.
          */
-       guest_pebs_mask = pebs_mask & intel_ctrl & guest_pebs->enable &
+       guest_pebs_mask = pebs_mask & guest_pebs->enable &

                          ~cpuc->intel_ctrl_exclude_guest_mask & cpuc->intel_ctrl_exclude_host_mask;

Reviewed-by on v3 1/9
#10 lore raw

Re: [PATCH v3 1/9] perf/x86/intel: Ensure guest PEBS path doesn't set unwanted PERF_GLOBAL_CTRL bits

In reply to Sean Christopherson
Quoted 16 lines · Sean Christopherson

When reinstating PEBS counters into PERF_GLOBAL_CTRL for a KVM guest, mask the value with perf's desired/original PERF_GLOBAL_CTRL value to ensure KVM doesn't unintentionally enable counters. This _should_ be a nop, as arr[pebs_enable].guest is derived from cpuc->pebs_enabled, which should be a subset of x86_pmu.intel_ctrl, but paranoia is cheap in this case.

---
 arch/x86/events/intel/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/core.c +0 -0
index d9488ade0f8e..b70dc35fcceb 100644

arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5072

Quoted 8 lines
arch/x86/events/intel/core.c +1 -1
@@ -5066,7 +5066,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 		arr[pebs_enable].guest &= ~kvm_pmu->host_cross_mapped_mask;
 		arr[global_ctrl].guest &= ~kvm_pmu->host_cross_mapped_mask;
 		/* Set hw GLOBAL_CTRL bits for PEBS counter when it runs for guest */
-		arr[global_ctrl].guest |= arr[pebs_enable].guest;
+		arr[global_ctrl].guest |= intel_ctrl & arr[pebs_enable].guest;
 	}
 
 	return arr;

The change looks good to me, but it's totally removed in next patch. Not sure if it's worthy to do it.

author maintainer
#11 lore raw

PATCH v3 2/9 perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU has isolation

In reply to Sean Christopherson

When filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit, *never* insert an entry for PEBS_ENABLED if the CPU properly isolates PEBS events, in which case disabling counters via PERF_GLOBAL_CTRL is sufficient to prevent unwanted PEBS events in the guest (or host). Because perf loads PEBS_ENABLE with the unfiltered cpu_hw_events.pebs_enabled, i.e. with both host and guest masks, there is no need to load different values for the guest versus host, perf+KVM can and should simply control which counters are enabled/disabled via PERF_GLOBAL_CTRL.

Avoiding touching PEBS_ENABLED "fixes" a bug where PEBS_ENABLED can end up with "stuck" bits if a PEBS event is throttled between generating the list and actually entering the guest (Intel CPUs can't arbtitrarily block NMIs). Fixes in quotes because leaving PEBS_ENABLED as-is doesn't fix the underlying problem of perf (via PMIs) being able to modify state after the perf<=>KVM handoff.

But not writing PEBS_ENABLED is desirable no matter what, as stating the obvious, leaving PEBS_ENABLED as-is avoids three MSR writes on every VMX
transition: one each on entry/exit, and one more explicit WRMSR to zero PEBS_ENABLED before VM-Entry (KVM assumes the only reason PEBS_ENABLED is in the load list is if the CPU lacks isolation and thus needs a quiescent period).

Opportunistically add comments to (better) explain the rules for generating the set of PEBS counters that will be active while the guest is running, along with a FIXME for the suspected hack-a-fix where perf disables guest PEBS if _any_ PEBS event is configured to count in the host (commit 854250329c02 ("KVM: x86/pmu: Disable guest PEBS temporarily in two rare situations") doesn't explain the motivation, at all).

Fixes c59a1f106f5c ("KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS")
Cc Jim Mattson <[email protected]>
Cc Mingwei Zhang <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 arch/x86/events/intel/core.c | 55 ++++++++++++++++++++++++------------
 1 file changed, 37 insertions(+), 18 deletions(-)
arch/x86/events/intel/core.c +37 -18
index b70dc35fcceb..13cd12d3eeee 100644
@@ -4999,12 +4999,15 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;
 	u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
 	u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;
-	int global_ctrl, pebs_enable;
+	u64 guest_pebs_mask;
+	int global_ctrl;
 
 	/*
 	 * In addition to obeying exclude_guest/exclude_host, remove bits being
 	 * used for PEBS when running a guest, because PEBS writes to virtual
-	 * addresses (not physical addresses).
+	 * addresses (not physical addresses).  If the guest wants to utilize
+	 * PEBS, and PEBS can safely enabled in the guest, bits for the guest's
+	 * PEBS-enabled counters will be OR'd back in as appropriate.
 	 */
 	*nr = 0;
 	global_ctrl = (*nr)++;
@@ -5051,24 +5054,40 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 		};
 	}
 
-	pebs_enable = (*nr)++;
-	arr[pebs_enable] = (struct perf_guest_switch_msr){
-		.msr = MSR_IA32_PEBS_ENABLE,
-		.host = cpuc->pebs_enabled & ~cpuc->intel_ctrl_guest_mask,
-		.guest = pebs_mask & ~cpuc->intel_ctrl_host_mask & kvm_pmu->pebs_enable,
-	};
+	/*
+	 * Restrict guest PEBS events to counters that (a) perf supports, (b)
+	 * the guest wants to use for PEBS, (c) are not excluded from counting
+	 * in the guest, and (d) _are_ excluded from counting in the host.
+	 */
+	guest_pebs_mask = pebs_mask & intel_ctrl & kvm_pmu->pebs_enable &
+			  ~cpuc->intel_ctrl_host_mask &
+			  cpuc->intel_ctrl_guest_mask;
 
-	if (arr[pebs_enable].host) {
-		/* Disable guest PEBS if host PEBS is enabled. */
-		arr[pebs_enable].guest = 0;
-	} else {
-		/* Disable guest PEBS thoroughly for cross-mapped PEBS counters. */
-		arr[pebs_enable].guest &= ~kvm_pmu->host_cross_mapped_mask;
-		arr[global_ctrl].guest &= ~kvm_pmu->host_cross_mapped_mask;
-		/* Set hw GLOBAL_CTRL bits for PEBS counter when it runs for guest */
-		arr[global_ctrl].guest |= intel_ctrl & arr[pebs_enable].guest;
-	}
+	/*
+	 * Disable counters where the guest PMC is different than the host PMC
+	 * being used on behalf of the guest, as the PEBS record includes
+	 * PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
+	 * wrong counter(s).
+	 */
+	guest_pebs_mask &= ~kvm_pmu->host_cross_mapped_mask;
 
+	/*
+	 * FIXME: Allow guest and host usage of PEBS events to co-exist instead
+	 *        of disabling guest PEBS entirely if the host is using PEBS.
+	 *        What exactly goes wrong if guest and host are using PEBS is
+	 *        unknown.
+	 */
+	if (pebs_mask & ~cpuc->intel_ctrl_guest_mask)
+		guest_pebs_mask = 0;
+
+	/*
+	 * Do NOT mess with PEBS_ENABLED.  As above, disabling counters via
+	 * PERF_GLOBAL_CTRL is sufficient, and loading a stale PEBS_ENABLED,
+	 * e.g. on VM-Exit, can put the system in a bad state.  Simply enable
+	 * counters in PERF_GLOBAL_CTRL, as perf load PEBS_ENABLED with the
+	 * full value, i.e. perf *also* relies on PERF_GLOBAL_CTRL.
+	 */
+	arr[global_ctrl].guest |= guest_pebs_mask;
 	return arr;
 }
 
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Reviewed-by on v3 2/9
#12 lore raw

Re: [PATCH v3 2/9] perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU has isolation

Quoted 107 lines · Sean Christopherson

When filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit, *never* insert an entry for PEBS_ENABLED if the CPU properly isolates PEBS events, in which case disabling counters via PERF_GLOBAL_CTRL is sufficient to prevent unwanted PEBS events in the guest (or host). Because perf loads PEBS_ENABLE with the unfiltered cpu_hw_events.pebs_enabled, i.e. with both host and guest masks, there is no need to load different values for the guest versus host, perf+KVM can and should simply control which counters are enabled/disabled via PERF_GLOBAL_CTRL.

Avoiding touching PEBS_ENABLED "fixes" a bug where PEBS_ENABLED can end up with "stuck" bits if a PEBS event is throttled between generating the list and actually entering the guest (Intel CPUs can't arbtitrarily block NMIs). Fixes in quotes because leaving PEBS_ENABLED as-is doesn't fix the underlying problem of perf (via PMIs) being able to modify state after the perf<=>KVM handoff.

But not writing PEBS_ENABLED is desirable no matter what, as stating the obvious, leaving PEBS_ENABLED as-is avoids three MSR writes on every VMX
transition: one each on entry/exit, and one more explicit WRMSR to zero PEBS_ENABLED before VM-Entry (KVM assumes the only reason PEBS_ENABLED is in the load list is if the CPU lacks isolation and thus needs a quiescent period).

Opportunistically add comments to (better) explain the rules for generating the set of PEBS counters that will be active while the guest is running, along with a FIXME for the suspected hack-a-fix where perf disables guest PEBS if _any_ PEBS event is configured to count in the host (commit 854250329c02 ("KVM: x86/pmu: Disable guest PEBS temporarily in two rare situations") doesn't explain the motivation, at all).

Fixes c59a1f106f5c ("KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS")
Cc Jim Mattson <[email protected]>
Cc Mingwei Zhang <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 arch/x86/events/intel/core.c | 55 ++++++++++++++++++++++++------------
 1 file changed, 37 insertions(+), 18 deletions(-)
arch/x86/events/intel/core.c +29 -18
index b70dc35fcceb..13cd12d3eeee 100644
@@ -4999,12 +4999,15 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;
 	u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
 	u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;
-	int global_ctrl, pebs_enable;
+	u64 guest_pebs_mask;
+	int global_ctrl;
 
 	/*
 	 * In addition to obeying exclude_guest/exclude_host, remove bits being
 	 * used for PEBS when running a guest, because PEBS writes to virtual
-	 * addresses (not physical addresses).
+	 * addresses (not physical addresses).  If the guest wants to utilize
+	 * PEBS, and PEBS can safely enabled in the guest, bits for the guest's
+	 * PEBS-enabled counters will be OR'd back in as appropriate.
 	 */
 	*nr = 0;
 	global_ctrl = (*nr)++;
@@ -5051,24 +5054,40 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 		};
 	}
 
-	pebs_enable = (*nr)++;
-	arr[pebs_enable] = (struct perf_guest_switch_msr){
-		.msr = MSR_IA32_PEBS_ENABLE,
-		.host = cpuc->pebs_enabled & ~cpuc->intel_ctrl_guest_mask,
-		.guest = pebs_mask & ~cpuc->intel_ctrl_host_mask & kvm_pmu->pebs_enable,
-	};
+	/*
+	 * Restrict guest PEBS events to counters that (a) perf supports, (b)
+	 * the guest wants to use for PEBS, (c) are not excluded from counting
+	 * in the guest, and (d) _are_ excluded from counting in the host.
+	 */
+	guest_pebs_mask = pebs_mask & intel_ctrl & kvm_pmu->pebs_enable &
+			  ~cpuc->intel_ctrl_host_mask &
+			  cpuc->intel_ctrl_guest_mask;
 
-	if (arr[pebs_enable].host) {
-		/* Disable guest PEBS if host PEBS is enabled. */
-		arr[pebs_enable].guest = 0;
-	} else {
-		/* Disable guest PEBS thoroughly for cross-mapped PEBS counters. */
-		arr[pebs_enable].guest &= ~kvm_pmu->host_cross_mapped_mask;
-		arr[global_ctrl].guest &= ~kvm_pmu->host_cross_mapped_mask;
-		/* Set hw GLOBAL_CTRL bits for PEBS counter when it runs for guest */
-		arr[global_ctrl].guest |= intel_ctrl & arr[pebs_enable].guest;
-	}
+	/*
+	 * Disable counters where the guest PMC is different than the host PMC
+	 * being used on behalf of the guest, as the PEBS record includes
+	 * PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
+	 * wrong counter(s).
+	 */
+	guest_pebs_mask &= ~kvm_pmu->host_cross_mapped_mask;
 
+	/*
+	 * FIXME: Allow guest and host usage of PEBS events to co-exist instead
+	 *        of disabling guest PEBS entirely if the host is using PEBS.
+	 *        What exactly goes wrong if guest and host are using PEBS is
+	 *        unknown.
+	 */
+	if (pebs_mask & ~cpuc->intel_ctrl_guest_mask)
+		guest_pebs_mask = 0;
+

arch/x86/events/intel/core.c static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) · line 5092

Quoted 10 lines
arch/x86/events/intel/core.c +8 -0
+	/*
+	 * Do NOT mess with PEBS_ENABLED.  As above, disabling counters via
+	 * PERF_GLOBAL_CTRL is sufficient, and loading a stale PEBS_ENABLED,
+	 * e.g. on VM-Exit, can put the system in a bad state.  Simply enable
+	 * counters in PERF_GLOBAL_CTRL, as perf load PEBS_ENABLED with the
+	 * full value, i.e. perf *also* relies on PERF_GLOBAL_CTRL.
+	 */
+	arr[global_ctrl].guest |= guest_pebs_mask;
 	return arr;
 }

LGTM.

Quoted 1 line
author maintainer
#13 lore raw

PATCH v3 3/9 perf/x86/intel: Don't context switch DS_AREA (and PEBS config) if PEBS is unused

In reply to Sean Christopherson

When filling the list of MSRs to be loaded by KVM on VM-Enter and VM-Exit, load the guest values for DS_AREA and (conditionally) MSR_PEBS_DATA_CFG if and only if PEBS will be active in the guest, i.e. only if a PEBS record may be generated while running the guest. As shown by the !pebs_ept path, it's perfectly safe to run with the host's DS_AREA, so long as PEBS-enabled counters are disabled via PERF_GLOBAL_CTRL.

Omitting DS_AREA and MSR_PEBS_DATA_CFG when PEBS is unused saves two MSR writes per MSR on each VMX transition, i.e. eliminates two/four pointless MSR writes on each VMX roundtrip when PEBS isn't being used by the guest.

Fixes c59a1f106f5c ("KVM: x86/pmu: Add IA32_PEBS_ENABLE MSR emulation for extended PEBS")
Cc Jim Mattson <[email protected]>
Cc Mingwei Zhang <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 arch/x86/events/intel/core.c | 39 +++++++++++++++++++++++-------------
 1 file changed, 25 insertions(+), 14 deletions(-)
arch/x86/events/intel/core.c +25 -14
index 13cd12d3eeee..0e9ac2e9b5e7 100644
@@ -5037,23 +5037,14 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 		return arr;
 	}
 
+	/*
+	 * If the guest won't use PEBS or the CPU doesn't support PEBS in the
+	 * guest, then there's nothing more to do as disabling PMCs via
+	 * PERF_GLOBAL_CTRL is sufficient on CPUs with guest/host isolation.
+	 */
 	if (!kvm_pmu || !x86_pmu.pebs_ept)
 		return arr;
 
-	arr[(*nr)++] = (struct perf_guest_switch_msr){
-		.msr = MSR_IA32_DS_AREA,
-		.host = (unsigned long)cpuc->ds,
-		.guest = kvm_pmu->ds_area,
-	};
-
-	if (x86_pmu.intel_cap.pebs_baseline) {
-		arr[(*nr)++] = (struct perf_guest_switch_msr){
-			.msr = MSR_PEBS_DATA_CFG,
-			.host = cpuc->active_pebs_data_cfg,
-			.guest = kvm_pmu->pebs_data_cfg,
-		};
-	}
-
 	/*
 	 * Restrict guest PEBS events to counters that (a) perf supports, (b)
 	 * the guest wants to use for PEBS, (c) are not excluded from counting
@@ -5080,6 +5071,26 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	if (pebs_mask & ~cpuc->intel_ctrl_guest_mask)
 		guest_pebs_mask = 0;
 
+	/*
+	 * Context switch DS_AREA and PEBS_DATA_CFG if and only if PEBS will be
+	 * active in the guest; if no records will be generated while the guest
+	 * is running, then simply keep the host values resident in hardware.
+	 */
+	arr[(*nr)++] = (struct perf_guest_switch_msr){
+		.msr = MSR_IA32_DS_AREA,
+		.host = (unsigned long)cpuc->ds,
+		.guest = guest_pebs_mask ? kvm_pmu->ds_area : (unsigned long)cpuc->ds,
+	};
+
+	if (x86_pmu.intel_cap.pebs_baseline) {
+		arr[(*nr)++] = (struct perf_guest_switch_msr){
+			.msr = MSR_PEBS_DATA_CFG,
+			.host = cpuc->active_pebs_data_cfg,
+			.guest = guest_pebs_mask ? kvm_pmu->pebs_data_cfg :
+						   cpuc->active_pebs_data_cfg,
+		};
+	}
+
 	/*
 	 * Do NOT mess with PEBS_ENABLED.  As above, disabling counters via
 	 * PERF_GLOBAL_CTRL is sufficient, and loading a stale PEBS_ENABLED,
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
author maintainer
#14 lore raw

PATCH v3 4/9 perf/x86/intel: Make @data a mandatory param for intel_guest_get_msrs()

In reply to Sean Christopherson

Drop "support" for passing a NULL @data/@kvm_pmu param when getting guest MSRs. KVM, the only in-tree user, unconditionally passes a non-NULL pointer, and carrying code that suggests @data may be NULL is confusing, e.g. incorrectly implies that there are scenarios where KVM doesn't pass a PMU context.

Fixes 8183a538cd95 ("KVM: x86/pmu: Add IA32_DS_AREA MSR emulation to support guest DS")
Cc Jim Mattson <[email protected]>
Cc Mingwei Zhang <[email protected]>
Cc Stephane Eranian <[email protected]>
---
 arch/x86/events/intel/core.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/events/intel/core.c +4 -4
index 0e9ac2e9b5e7..e9f5a6143e71 100644
@@ -5038,11 +5038,11 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	}
 
 	/*
-	 * If the guest won't use PEBS or the CPU doesn't support PEBS in the
-	 * guest, then there's nothing more to do as disabling PMCs via
-	 * PERF_GLOBAL_CTRL is sufficient on CPUs with guest/host isolation.
+	 * If the CPU doesn't support PEBS in the guest, then there's nothing
+	 * more to do as disabling PMCs via PERF_GLOBAL_CTRL is sufficient on
+	 * CPUs with guest/host isolation.
 	 */
-	if (!kvm_pmu || !x86_pmu.pebs_ept)
+	if (!x86_pmu.pebs_ept)
 		return arr;
 
 	/*
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Reviewed-by on v3 4/9
#15 lore raw

Re: [PATCH v3 4/9] perf/x86/intel: Make @data a mandatory param for intel_guest_get_msrs()

Quoted 7 lines · Sean Christopherson

Drop "support" for passing a NULL @data/@kvm_pmu param when getting guest MSRs. KVM, the only in-tree user, unconditionally passes a non-NULL pointer, and carrying code that suggests @data may be NULL is confusing, e.g. incorrectly implies that there are scenarios where KVM doesn't pass a PMU context.

Quoted 8 lines
Fixes 8183a538cd95 ("KVM: x86/pmu: Add IA32_DS_AREA MSR emulation to support guest DS")
Cc Jim Mattson <[email protected]>
Cc Mingwei Zhang <[email protected]>
Cc Stephane Eranian <[email protected]>

---

author maintainer
#16 lore raw

PATCH v3 5/9 perf/x86/intel: Invert names of intel_ctrl_{guest,host}_mask

In reply to Sean Christopherson

Rename intel_ctrl_{guest,host}_mask to intel_ctrl_exclude_{host,guest}_mask to more accurately capture what they actually track. Specifically, an event that is excluded from the guest is NOT guaranteed to count in the host, and vice versa, as it legal (albeit bizarre) to configure an event to exclude both the host and the guest, i.e. to not count at all.

Subjectively (though anyone who disagrees is wrong), aligning with perf_event_attr.exclude_{guest,host} also makes all related code much easier to follow.

No functional change intended.

---
 arch/x86/events/intel/core.c | 22 +++++++++++-----------
 arch/x86/events/intel/lbr.c  |  2 +-
 arch/x86/events/perf_event.h |  4 ++--
 3 files changed, 14 insertions(+), 14 deletions(-)
arch/x86/events/intel/core.c +11 -11
index e9f5a6143e71..7f7c7927b70b 100644
@@ -2535,7 +2535,7 @@ static void __intel_pmu_enable_all(int added, bool pmi)
 	}
 
 	wrmsrq(MSR_CORE_PERF_GLOBAL_CTRL,
-	       intel_ctrl & ~cpuc->intel_ctrl_guest_mask);
+	       intel_ctrl & ~cpuc->intel_ctrl_exclude_host_mask);
 
 	if (test_bit(INTEL_PMC_IDX_FIXED_BTS, cpuc->active_mask)) {
 		struct perf_event *event =
@@ -2733,9 +2733,9 @@ static inline void intel_set_masks(struct perf_event *event, int idx)
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	if (event->attr.exclude_host)
-		__set_bit(idx, (unsigned long *)&cpuc->intel_ctrl_guest_mask);
+		__set_bit(idx, (unsigned long *)&cpuc->intel_ctrl_exclude_host_mask);
 	if (event->attr.exclude_guest)
-		__set_bit(idx, (unsigned long *)&cpuc->intel_ctrl_host_mask);
+		__set_bit(idx, (unsigned long *)&cpuc->intel_ctrl_exclude_guest_mask);
 	if (event_is_checkpointed(event))
 		__set_bit(idx, (unsigned long *)&cpuc->intel_cp_status);
 }
@@ -2744,8 +2744,8 @@ static inline void intel_clear_masks(struct perf_event *event, int idx)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
-	__clear_bit(idx, (unsigned long *)&cpuc->intel_ctrl_guest_mask);
-	__clear_bit(idx, (unsigned long *)&cpuc->intel_ctrl_host_mask);
+	__clear_bit(idx, (unsigned long *)&cpuc->intel_ctrl_exclude_host_mask);
+	__clear_bit(idx, (unsigned long *)&cpuc->intel_ctrl_exclude_guest_mask);
 	__clear_bit(idx, (unsigned long *)&cpuc->intel_cp_status);
 }
 
@@ -3473,7 +3473,7 @@ static void x86_pmu_handle_guest_pebs(struct pt_regs *regs,
 				      struct perf_sample_data *data)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
-	u64 guest_pebs_idxs = cpuc->pebs_enabled & ~cpuc->intel_ctrl_host_mask;
+	u64 guest_pebs_idxs = cpuc->pebs_enabled & ~cpuc->intel_ctrl_exclude_guest_mask;
 	struct perf_event *event = NULL;
 	int bit;
 
@@ -5013,8 +5013,8 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	global_ctrl = (*nr)++;
 	arr[global_ctrl] = (struct perf_guest_switch_msr){
 		.msr = MSR_CORE_PERF_GLOBAL_CTRL,
-		.host = intel_ctrl & ~cpuc->intel_ctrl_guest_mask,
-		.guest = intel_ctrl & ~cpuc->intel_ctrl_host_mask & ~pebs_mask,
+		.host = intel_ctrl & ~cpuc->intel_ctrl_exclude_host_mask,
+		.guest = intel_ctrl & ~cpuc->intel_ctrl_exclude_guest_mask & ~pebs_mask,
 	};
 
 	if (!x86_pmu.ds_pebs)
@@ -5051,8 +5051,8 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	 * in the guest, and (d) _are_ excluded from counting in the host.
 	 */
 	guest_pebs_mask = pebs_mask & intel_ctrl & kvm_pmu->pebs_enable &
-			  ~cpuc->intel_ctrl_host_mask &
-			  cpuc->intel_ctrl_guest_mask;
+			  ~cpuc->intel_ctrl_exclude_guest_mask &
+			  cpuc->intel_ctrl_exclude_host_mask;
 
 	/*
 	 * Disable counters where the guest PMC is different than the host PMC
@@ -5068,7 +5068,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	 *        What exactly goes wrong if guest and host are using PEBS is
 	 *        unknown.
 	 */
-	if (pebs_mask & ~cpuc->intel_ctrl_guest_mask)
+	if (pebs_mask & ~cpuc->intel_ctrl_exclude_host_mask)
 		guest_pebs_mask = 0;
 
 	/*
arch/x86/events/intel/lbr.c +1 -1
index 72f2adcda7c6..1298049246d7 100644
@@ -713,7 +713,7 @@ static inline bool vlbr_exclude_host(void)
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	return test_bit(INTEL_PMC_IDX_FIXED_VLBR,
-		(unsigned long *)&cpuc->intel_ctrl_guest_mask);
+		(unsigned long *)&cpuc->intel_ctrl_exclude_host_mask);
 }
 
 void intel_pmu_lbr_enable_all(bool pmi)
arch/x86/events/perf_event.h +2 -2
index fad87d3c8b2c..cc0aeeb34eb5 100644
@@ -339,8 +339,8 @@ struct cpu_hw_events {
 	/*
 	 * Intel host/guest exclude bits
 	 */
-	u64				intel_ctrl_guest_mask;
-	u64				intel_ctrl_host_mask;
+	u64				intel_ctrl_exclude_host_mask;
+	u64				intel_ctrl_exclude_guest_mask;
 	struct perf_guest_switch_msr	guest_switch_msrs[X86_PMC_IDX_MAX];
 
 	/*
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Reviewed-by on v3 5/9
#17 lore raw

Re: [PATCH v3 5/9] perf/x86/intel: Invert names of intel_ctrl_{guest,host}_mask

Quoted 114 lines · Sean Christopherson

Rename intel_ctrl_{guest,host}_mask to intel_ctrl_exclude_{host,guest}_mask to more accurately capture what they actually track. Specifically, an event that is excluded from the guest is NOT guaranteed to count in the host, and vice versa, as it legal (albeit bizarre) to configure an event to exclude both the host and the guest, i.e. to not count at all.

Subjectively (though anyone who disagrees is wrong), aligning with perf_event_attr.exclude_{guest,host} also makes all related code much easier to follow.

No functional change intended.

---
 arch/x86/events/intel/core.c | 22 +++++++++++-----------
 arch/x86/events/intel/lbr.c  |  2 +-
 arch/x86/events/perf_event.h |  4 ++--
 3 files changed, 14 insertions(+), 14 deletions(-)
arch/x86/events/intel/core.c +11 -11
index e9f5a6143e71..7f7c7927b70b 100644
@@ -2535,7 +2535,7 @@ static void __intel_pmu_enable_all(int added, bool pmi)
 	}
 
 	wrmsrq(MSR_CORE_PERF_GLOBAL_CTRL,
-	       intel_ctrl & ~cpuc->intel_ctrl_guest_mask);
+	       intel_ctrl & ~cpuc->intel_ctrl_exclude_host_mask);
 
 	if (test_bit(INTEL_PMC_IDX_FIXED_BTS, cpuc->active_mask)) {
 		struct perf_event *event =
@@ -2733,9 +2733,9 @@ static inline void intel_set_masks(struct perf_event *event, int idx)
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	if (event->attr.exclude_host)
-		__set_bit(idx, (unsigned long *)&cpuc->intel_ctrl_guest_mask);
+		__set_bit(idx, (unsigned long *)&cpuc->intel_ctrl_exclude_host_mask);
 	if (event->attr.exclude_guest)
-		__set_bit(idx, (unsigned long *)&cpuc->intel_ctrl_host_mask);
+		__set_bit(idx, (unsigned long *)&cpuc->intel_ctrl_exclude_guest_mask);
 	if (event_is_checkpointed(event))
 		__set_bit(idx, (unsigned long *)&cpuc->intel_cp_status);
 }
@@ -2744,8 +2744,8 @@ static inline void intel_clear_masks(struct perf_event *event, int idx)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
-	__clear_bit(idx, (unsigned long *)&cpuc->intel_ctrl_guest_mask);
-	__clear_bit(idx, (unsigned long *)&cpuc->intel_ctrl_host_mask);
+	__clear_bit(idx, (unsigned long *)&cpuc->intel_ctrl_exclude_host_mask);
+	__clear_bit(idx, (unsigned long *)&cpuc->intel_ctrl_exclude_guest_mask);
 	__clear_bit(idx, (unsigned long *)&cpuc->intel_cp_status);
 }
 
@@ -3473,7 +3473,7 @@ static void x86_pmu_handle_guest_pebs(struct pt_regs *regs,
 				      struct perf_sample_data *data)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
-	u64 guest_pebs_idxs = cpuc->pebs_enabled & ~cpuc->intel_ctrl_host_mask;
+	u64 guest_pebs_idxs = cpuc->pebs_enabled & ~cpuc->intel_ctrl_exclude_guest_mask;
 	struct perf_event *event = NULL;
 	int bit;
 
@@ -5013,8 +5013,8 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	global_ctrl = (*nr)++;
 	arr[global_ctrl] = (struct perf_guest_switch_msr){
 		.msr = MSR_CORE_PERF_GLOBAL_CTRL,
-		.host = intel_ctrl & ~cpuc->intel_ctrl_guest_mask,
-		.guest = intel_ctrl & ~cpuc->intel_ctrl_host_mask & ~pebs_mask,
+		.host = intel_ctrl & ~cpuc->intel_ctrl_exclude_host_mask,
+		.guest = intel_ctrl & ~cpuc->intel_ctrl_exclude_guest_mask & ~pebs_mask,
 	};
 
 	if (!x86_pmu.ds_pebs)
@@ -5051,8 +5051,8 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	 * in the guest, and (d) _are_ excluded from counting in the host.
 	 */
 	guest_pebs_mask = pebs_mask & intel_ctrl & kvm_pmu->pebs_enable &
-			  ~cpuc->intel_ctrl_host_mask &
-			  cpuc->intel_ctrl_guest_mask;
+			  ~cpuc->intel_ctrl_exclude_guest_mask &
+			  cpuc->intel_ctrl_exclude_host_mask;
 
 	/*
 	 * Disable counters where the guest PMC is different than the host PMC
@@ -5068,7 +5068,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	 *        What exactly goes wrong if guest and host are using PEBS is
 	 *        unknown.
 	 */
-	if (pebs_mask & ~cpuc->intel_ctrl_guest_mask)
+	if (pebs_mask & ~cpuc->intel_ctrl_exclude_host_mask)
 		guest_pebs_mask = 0;
 
 	/*
arch/x86/events/intel/lbr.c +1 -1
index 72f2adcda7c6..1298049246d7 100644
@@ -713,7 +713,7 @@ static inline bool vlbr_exclude_host(void)
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
 	return test_bit(INTEL_PMC_IDX_FIXED_VLBR,
-		(unsigned long *)&cpuc->intel_ctrl_guest_mask);
+		(unsigned long *)&cpuc->intel_ctrl_exclude_host_mask);
 }
 
 void intel_pmu_lbr_enable_all(bool pmi)
arch/x86/events/perf_event.h +0 -0
index fad87d3c8b2c..cc0aeeb34eb5 100644

arch/x86/events/perf_event.h struct cpu_hw_events { · line 343

Quoted 7 lines
arch/x86/events/perf_event.h +2 -2
@@ -339,8 +339,8 @@ struct cpu_hw_events {
 	/*
 	 * Intel host/guest exclude bits
 	 */
-	u64				intel_ctrl_guest_mask;
-	u64				intel_ctrl_host_mask;
+	u64				intel_ctrl_exclude_host_mask;
+	u64				intel_ctrl_exclude_guest_mask;

This looks much straightforward.

Quoted 3 lines
arch/x86/events/perf_event.h +0 -0
 	struct perf_guest_switch_msr	guest_switch_msrs[X86_PMC_IDX_MAX];
 
 	/*
author maintainer
#18 lore raw

PATCH v3 6/9 perf/x86: KVM: Have perf define a dedicated struct for getting guest PEBS data

In reply to Sean Christopherson

Have perf define a struct for getting guest PEBS data from KVM instead of poking into the kvm_pmu structure. Passing in an entire "struct kvm_pmu" _as an opaque pointer_ to get at four fields is silly, especially since one of the fields exists purely to convey information to perf, i.e. isn't used by KVM.

Perf should also own its APIs, i.e. define what fields/data it needs, not rely on KVM to throw fields into data structures that effectively hold KVM-internal state.

Opportunistically rephrase the comment about cross-mapped counters to explain *why* PEBS needs to be disabled.

---
 arch/x86/events/core.c            |  5 +++--
 arch/x86/events/intel/core.c      | 16 ++++++++--------
 arch/x86/events/perf_event.h      |  3 ++-
 arch/x86/include/asm/kvm_host.h   |  9 ---------
 arch/x86/include/asm/perf_event.h | 12 ++++++++++--
 arch/x86/kvm/vmx/pmu_intel.c      | 17 ++++++++++++++---
 arch/x86/kvm/vmx/vmx.c            | 11 ++++++++---
 arch/x86/kvm/vmx/vmx.h            |  2 +-
 8 files changed, 46 insertions(+), 29 deletions(-)
arch/x86/events/core.c +3 -2
index 810ab21ffd99..e6f788e72e72 100644
@@ -723,9 +723,10 @@ void x86_pmu_disable_all(void)
 	}
 }
 
-struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data)
+struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr,
+						  struct x86_guest_pebs *guest_pebs)
 {
-	return static_call(x86_pmu_guest_get_msrs)(nr, data);
+	return static_call(x86_pmu_guest_get_msrs)(nr, guest_pebs);
 }
 EXPORT_SYMBOL_FOR_KVM(perf_guest_get_msrs);
 
arch/x86/events/intel/core.c +8 -8
index 7f7c7927b70b..e9acfc3f3a82 100644
@@ -14,7 +14,6 @@
 #include <linux/slab.h>
 #include <linux/export.h>
 #include <linux/nmi.h>
-#include <linux/kvm_host.h>
 
 #include <asm/cpufeature.h>
 #include <asm/debugreg.h>
@@ -4992,11 +4991,11 @@ static int intel_pmu_hw_config(struct perf_event *event)
  * when it uses {RD,WR}MSR, which should be handled by the KVM context,
  * specifically in the intel_pmu_{get,set}_msr().
  */
-static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
+static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr,
+							  struct x86_guest_pebs *guest_pebs)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 	struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs;
-	struct kvm_pmu *kvm_pmu = (struct kvm_pmu *)data;
 	u64 intel_ctrl = hybrid(cpuc->pmu, intel_ctrl);
 	u64 pebs_mask = cpuc->pebs_enabled & x86_pmu.pebs_capable;
 	u64 guest_pebs_mask;
@@ -5050,7 +5049,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	 * the guest wants to use for PEBS, (c) are not excluded from counting
 	 * in the guest, and (d) _are_ excluded from counting in the host.
 	 */
-	guest_pebs_mask = pebs_mask & intel_ctrl & kvm_pmu->pebs_enable &
+	guest_pebs_mask = pebs_mask & intel_ctrl & guest_pebs->enable &
 			  ~cpuc->intel_ctrl_exclude_guest_mask &
 			  cpuc->intel_ctrl_exclude_host_mask;
 
@@ -5060,7 +5059,7 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	 * PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
 	 * wrong counter(s).
 	 */
-	guest_pebs_mask &= ~kvm_pmu->host_cross_mapped_mask;
+	guest_pebs_mask &= ~guest_pebs->cross_mapped_mask;
 
 	/*
 	 * FIXME: Allow guest and host usage of PEBS events to co-exist instead
@@ -5079,14 +5078,14 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	arr[(*nr)++] = (struct perf_guest_switch_msr){
 		.msr = MSR_IA32_DS_AREA,
 		.host = (unsigned long)cpuc->ds,
-		.guest = guest_pebs_mask ? kvm_pmu->ds_area : (unsigned long)cpuc->ds,
+		.guest = guest_pebs_mask ? guest_pebs->ds_area : (unsigned long)cpuc->ds,
 	};
 
 	if (x86_pmu.intel_cap.pebs_baseline) {
 		arr[(*nr)++] = (struct perf_guest_switch_msr){
 			.msr = MSR_PEBS_DATA_CFG,
 			.host = cpuc->active_pebs_data_cfg,
-			.guest = guest_pebs_mask ? kvm_pmu->pebs_data_cfg :
+			.guest = guest_pebs_mask ? guest_pebs->data_cfg :
 						   cpuc->active_pebs_data_cfg,
 		};
 	}
@@ -5102,7 +5101,8 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)
 	return arr;
 }
 
-static struct perf_guest_switch_msr *core_guest_get_msrs(int *nr, void *data)
+static struct perf_guest_switch_msr *core_guest_get_msrs(int *nr,
+							 struct x86_guest_pebs *guest_pebs)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 	struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs;
arch/x86/events/perf_event.h +2 -1
index cc0aeeb34eb5..9183b3607962 100644
@@ -1023,7 +1023,8 @@ struct x86_pmu {
 	/*
 	 * Intel host/guest support (KVM)
 	 */
-	struct perf_guest_switch_msr *(*guest_get_msrs)(int *nr, void *data);
+	struct perf_guest_switch_msr *(*guest_get_msrs)(int *nr,
+							struct x86_guest_pebs *guest_pebs);
 
 	/*
 	 * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
arch/x86/include/asm/kvm_host.h +0 -9
index c470e40a00aa..91b070168947 100644
@@ -600,15 +600,6 @@ struct kvm_pmu {
 	u64 pebs_data_cfg;
 	u64 pebs_data_cfg_rsvd;
 
-	/*
-	 * If a guest counter is cross-mapped to host counter with different
-	 * index, its PEBS capability will be temporarily disabled.
-	 *
-	 * The user should make sure that this mask is updated
-	 * after disabling interrupts and before perf_guest_get_msrs();
-	 */
-	u64 host_cross_mapped_mask;
-
 	/*
 	 * The gate to release perf_events not marked in
 	 * pmc_in_use only once in a vcpu time slice.
arch/x86/include/asm/perf_event.h +10 -2
index 752cb319d5ea..bc7e48f6f4a8 100644
@@ -786,11 +786,19 @@ extern void perf_load_guest_lvtpc(u32 guest_lvtpc);
 extern void perf_put_guest_lvtpc(void);
 #endif
 
+struct x86_guest_pebs {
+	u64	enable;
+	u64	ds_area;
+	u64	data_cfg;
+	u64	cross_mapped_mask;
+};
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
-extern struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data);
+extern struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr,
+							 struct x86_guest_pebs *guest_pebs);
 extern void x86_perf_get_lbr(struct x86_pmu_lbr *lbr);
 #else
-struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data);
+struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr,
+						  struct x86_guest_pebs *guest_pebs);
 static inline void x86_perf_get_lbr(struct x86_pmu_lbr *lbr)
 {
 	memset(lbr, 0, sizeof(*lbr));
arch/x86/kvm/vmx/pmu_intel.c +14 -3
index 27eb76e6b6a0..e65adb3dc066 100644
@@ -736,11 +736,21 @@ static void intel_pmu_cleanup(struct kvm_vcpu *vcpu)
 		intel_pmu_release_guest_lbr_event(vcpu);
 }
 
-void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu)
+u64 intel_pmu_get_cross_mapped_mask(struct kvm_pmu *pmu)
 {
-	struct kvm_pmc *pmc = NULL;
+	u64 host_cross_mapped_mask;
+	struct kvm_pmc *pmc;
 	int bit, hw_idx;
 
+	/*
+	 * Provide a mask of counters that are cross-mapped between the guest
+	 * and the host, i.e. where a guest PMC is mapped to a host PMC with a
+	 * different index.  PEBS records hold a PERF_GLOBAL_STATUS snapshot,
+	 * and so PEBS-enabled counters need to hold the correct index so as
+	 * not to confuse the guest.
+	 */
+	host_cross_mapped_mask = 0;
+
 	kvm_for_each_pmc(pmu, pmc, bit, (unsigned long *)&pmu->global_ctrl) {
 		if (!pmc_is_locally_enabled(pmc) ||
 		    !pmc_is_globally_enabled(pmc) || !pmc->perf_event)
@@ -752,8 +762,9 @@ void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu)
 		 */
 		hw_idx = pmc->perf_event->hw.idx;
 		if (hw_idx != pmc->idx && hw_idx > -1)
-			pmu->host_cross_mapped_mask |= BIT_ULL(hw_idx);
+			host_cross_mapped_mask |= BIT_ULL(hw_idx);
 	}
+	return host_cross_mapped_mask;
 }
 
 static bool intel_pmu_is_mediated_pmu_supported(struct x86_pmu_capability *host_pmu)
arch/x86/kvm/vmx/vmx.c +8 -3
index a29896a9ef14..9f0a028cf10b 100644
@@ -7313,12 +7313,17 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
 	if (kvm_vcpu_has_mediated_pmu(&vmx->vcpu))
 		return;
 
-	pmu->host_cross_mapped_mask = 0;
+	struct x86_guest_pebs guest_pebs = {
+		.enable = pmu->pebs_enable,
+		.ds_area = pmu->ds_area,
+		.data_cfg = pmu->pebs_data_cfg,
+	};
+
 	if (pmu->pebs_enable & pmu->global_ctrl)
-		intel_pmu_cross_mapped_check(pmu);
+		guest_pebs.cross_mapped_mask = intel_pmu_get_cross_mapped_mask(pmu);
 
 	/* Note, nr_msrs may be garbage if perf_guest_get_msrs() returns NULL. */
-	msrs = perf_guest_get_msrs(&nr_msrs, (void *)pmu);
+	msrs = perf_guest_get_msrs(&nr_msrs, &guest_pebs);
 	if (!msrs)
 		return;
 
arch/x86/kvm/vmx/vmx.h +1 -1
index db84e8001da5..0c4563472940 100644
@@ -659,7 +659,7 @@ static __always_inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu)
 	return container_of(vcpu, struct vcpu_vmx, vcpu);
 }
 
-void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu);
+u64 intel_pmu_get_cross_mapped_mask(struct kvm_pmu *pmu);
 int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu);
 void vmx_passthrough_lbr_msrs(struct kvm_vcpu *vcpu);
 
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
author maintainer
#19 lore raw

PATCH v3 7/9 perf/x86/intel: KVM: Handle cross-mapped PEBS PMCs entirely within KVM

In reply to Sean Christopherson

Now that perf operates on a KVM-provided snapshot of PMU state, handled cross-mapped PEBS counters entirely in KVM by clearing unusable counters from the to-be-enabled mask instead of foisting the work on perf.

No functional change intended.

---
 arch/x86/events/intel/core.c      |  8 --------
 arch/x86/include/asm/perf_event.h |  1 -
 arch/x86/kvm/vmx/vmx.c            | 10 ++++++++--
 3 files changed, 8 insertions(+), 11 deletions(-)
arch/x86/events/intel/core.c +0 -8
index e9acfc3f3a82..8f6be0cc4c4b 100644
@@ -5053,14 +5053,6 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr,
 			  ~cpuc->intel_ctrl_exclude_guest_mask &
 			  cpuc->intel_ctrl_exclude_host_mask;
 
-	/*
-	 * Disable counters where the guest PMC is different than the host PMC
-	 * being used on behalf of the guest, as the PEBS record includes
-	 * PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
-	 * wrong counter(s).
-	 */
-	guest_pebs_mask &= ~guest_pebs->cross_mapped_mask;
-
 	/*
 	 * FIXME: Allow guest and host usage of PEBS events to co-exist instead
 	 *        of disabling guest PEBS entirely if the host is using PEBS.
arch/x86/include/asm/perf_event.h +0 -1
index bc7e48f6f4a8..19f874a79ab0 100644
@@ -790,7 +790,6 @@ struct x86_guest_pebs {
 	u64	enable;
 	u64	ds_area;
 	u64	data_cfg;
-	u64	cross_mapped_mask;
 };
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
 extern struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr,
arch/x86/kvm/vmx/vmx.c +8 -2
index 9f0a028cf10b..fbe3ce5f5a51 100644
@@ -7319,8 +7319,14 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
 		.data_cfg = pmu->pebs_data_cfg,
 	};
 
-	if (pmu->pebs_enable & pmu->global_ctrl)
-		guest_pebs.cross_mapped_mask = intel_pmu_get_cross_mapped_mask(pmu);
+	/*
+	 * Disable counters where the guest PMC is different than the host PMC
+	 * being used on behalf of the guest, as the PEBS record includes
+	 * PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
+	 * wrong counter(s).
+	 */
+	if (guest_pebs.enable & pmu->global_ctrl)
+		guest_pebs.enable &= ~intel_pmu_get_cross_mapped_mask(pmu);
 
 	/* Note, nr_msrs may be garbage if perf_guest_get_msrs() returns NULL. */
 	msrs = perf_guest_get_msrs(&nr_msrs, &guest_pebs);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Reviewed-by on v3 7/9
#20 lore raw

Re: [PATCH v3 7/9] perf/x86/intel: KVM: Handle cross-mapped PEBS PMCs entirely within KVM

Quoted 55 lines · Sean Christopherson

Now that perf operates on a KVM-provided snapshot of PMU state, handled cross-mapped PEBS counters entirely in KVM by clearing unusable counters from the to-be-enabled mask instead of foisting the work on perf.

No functional change intended.

---
 arch/x86/events/intel/core.c      |  8 --------
 arch/x86/include/asm/perf_event.h |  1 -
 arch/x86/kvm/vmx/vmx.c            | 10 ++++++++--
 3 files changed, 8 insertions(+), 11 deletions(-)
arch/x86/events/intel/core.c +0 -8
index e9acfc3f3a82..8f6be0cc4c4b 100644
@@ -5053,14 +5053,6 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr,
 			  ~cpuc->intel_ctrl_exclude_guest_mask &
 			  cpuc->intel_ctrl_exclude_host_mask;
 
-	/*
-	 * Disable counters where the guest PMC is different than the host PMC
-	 * being used on behalf of the guest, as the PEBS record includes
-	 * PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
-	 * wrong counter(s).
-	 */
-	guest_pebs_mask &= ~guest_pebs->cross_mapped_mask;
-
 	/*
 	 * FIXME: Allow guest and host usage of PEBS events to co-exist instead
 	 *        of disabling guest PEBS entirely if the host is using PEBS.
arch/x86/include/asm/perf_event.h +0 -1
index bc7e48f6f4a8..19f874a79ab0 100644
@@ -790,7 +790,6 @@ struct x86_guest_pebs {
 	u64	enable;
 	u64	ds_area;
 	u64	data_cfg;
-	u64	cross_mapped_mask;
 };
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
 extern struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr,
arch/x86/kvm/vmx/vmx.c +1 -2
index 9f0a028cf10b..fbe3ce5f5a51 100644
@@ -7319,8 +7319,14 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
 		.data_cfg = pmu->pebs_data_cfg,
 	};
 
-	if (pmu->pebs_enable & pmu->global_ctrl)
-		guest_pebs.cross_mapped_mask = intel_pmu_get_cross_mapped_mask(pmu);
+	/*

arch/x86/kvm/vmx/vmx.c static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) · line 7332

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +7 -0
+	 * Disable counters where the guest PMC is different than the host PMC
+	 * being used on behalf of the guest, as the PEBS record includes
+	 * PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
+	 * wrong counter(s).
+	 */
+	if (guest_pebs.enable & pmu->global_ctrl)
+		guest_pebs.enable &= ~intel_pmu_get_cross_mapped_mask(pmu);
 
 	/* Note, nr_msrs may be garbage if perf_guest_get_msrs() returns NULL. */
 	msrs = perf_guest_get_msrs(&nr_msrs, &guest_pebs);

LGTM.

author maintainer
#21 lore raw

PATCH v3 8/9 KVM: VMX: Drop a redundant pmu->global_ctrl check when processing pebs_enable

In reply to Sean Christopherson

Drop a redundant check that a PMC is globally enabled when looking for PEBS counters that are cross-mapped between the guest and the host. The for-loop explicitly iterates over pmu->global_ctrl, and since PEBS requires PMU v2+, kvm_pmu_has_perf_global_ctrl() must be true, and thus pmc_is_globally_enabled() is simply checking that the bit is set in pmu->global_ctrl.

No functional change intended.

---
 arch/x86/kvm/vmx/pmu_intel.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
arch/x86/kvm/vmx/pmu_intel.c +1 -2
index e65adb3dc066..659fe097b904 100644
@@ -752,8 +752,7 @@ u64 intel_pmu_get_cross_mapped_mask(struct kvm_pmu *pmu)
 	host_cross_mapped_mask = 0;
 
 	kvm_for_each_pmc(pmu, pmc, bit, (unsigned long *)&pmu->global_ctrl) {
-		if (!pmc_is_locally_enabled(pmc) ||
-		    !pmc_is_globally_enabled(pmc) || !pmc->perf_event)
+		if (!pmc_is_locally_enabled(pmc) || !pmc->perf_event)
 			continue;
 
 		/*
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Reviewed-by on v3 8/9
#22 lore raw

Re: [PATCH v3 8/9] KVM: VMX: Drop a redundant pmu->global_ctrl check when processing pebs_enable

Quoted 19 lines · Sean Christopherson

Drop a redundant check that a PMC is globally enabled when looking for PEBS counters that are cross-mapped between the guest and the host. The for-loop explicitly iterates over pmu->global_ctrl, and since PEBS requires PMU v2+, kvm_pmu_has_perf_global_ctrl() must be true, and thus pmc_is_globally_enabled() is simply checking that the bit is set in pmu->global_ctrl.

No functional change intended.

---
 arch/x86/kvm/vmx/pmu_intel.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
arch/x86/kvm/vmx/pmu_intel.c +0 -0
index e65adb3dc066..659fe097b904 100644

arch/x86/kvm/vmx/pmu_intel.c u64 intel_pmu_get_cross_mapped_mask(struct kvm_pmu *pmu) · line 758

Quoted 9 lines
arch/x86/kvm/vmx/pmu_intel.c +1 -2
@@ -752,8 +752,7 @@ u64 intel_pmu_get_cross_mapped_mask(struct kvm_pmu *pmu)
 	host_cross_mapped_mask = 0;
 
 	kvm_for_each_pmc(pmu, pmc, bit, (unsigned long *)&pmu->global_ctrl) {
-		if (!pmc_is_locally_enabled(pmc) ||
-		    !pmc_is_globally_enabled(pmc) || !pmc->perf_event)
+		if (!pmc_is_locally_enabled(pmc) || !pmc->perf_event)
 			continue;
 
 		/*
author maintainer
#23 lore raw

PATCH v3 9/9 KVM: VMX: Only tell perf to enable PEBS counters for fully enabled PMCs

In reply to Sean Christopherson

When passing the guest's requested PEBS_ENABLE (or rather, KVM's version of PEBS_ENABLE on behalf of the guest), omit counters that are locally disable and/or don't have a perf event (due to contention), in addition to omitting counters that are cross-mapped in the host.

In practice, this should be a nop as perf will already have disabled the associated counter, i.e. cpuc->pebs_enabled should have been cleared, but paranoia is cheap, and the existing code _looks_ wrong.

---
 arch/x86/kvm/vmx/pmu_intel.c | 30 ++++++++++++++++--------------
 arch/x86/kvm/vmx/vmx.c       | 11 +----------
 arch/x86/kvm/vmx/vmx.h       | 15 ++++++++++++++-
 3 files changed, 31 insertions(+), 25 deletions(-)
arch/x86/kvm/vmx/pmu_intel.c +16 -14
index 659fe097b904..1e420c8bca9d 100644
@@ -736,34 +736,36 @@ static void intel_pmu_cleanup(struct kvm_vcpu *vcpu)
 		intel_pmu_release_guest_lbr_event(vcpu);
 }
 
-u64 intel_pmu_get_cross_mapped_mask(struct kvm_pmu *pmu)
+u64 __intel_pmu_compute_pebs_enable(struct kvm_pmu *pmu)
 {
-	u64 host_cross_mapped_mask;
+	u64 guest_pebs_enable = pmu->pebs_enable & pmu->global_ctrl;
+	u64 pebs_enable = 0;
 	struct kvm_pmc *pmc;
 	int bit, hw_idx;
 
 	/*
-	 * Provide a mask of counters that are cross-mapped between the guest
-	 * and the host, i.e. where a guest PMC is mapped to a host PMC with a
-	 * different index.  PEBS records hold a PERF_GLOBAL_STATUS snapshot,
-	 * and so PEBS-enabled counters need to hold the correct index so as
-	 * not to confuse the guest.
+	 * Omit counters that are locally disabled, don't have a perf event, or
+	 * ended up with a perf event that is using a different counter than
+	 * the guest, i.e. where the guest PMC is different than the host PMC
+	 * being used on behalf of the guest.  PEBS records include
+	 * PERF_GLOBAL_STATUS, and so using a counter with a different index
+	 * means the guest will see overflow status for the wrong counter(s).
 	 */
-	host_cross_mapped_mask = 0;
-
-	kvm_for_each_pmc(pmu, pmc, bit, (unsigned long *)&pmu->global_ctrl) {
+	kvm_for_each_pmc(pmu, pmc, bit, (unsigned long *)&guest_pebs_enable) {
 		if (!pmc_is_locally_enabled(pmc) || !pmc->perf_event)
 			continue;
 
 		/*
-		 * A negative index indicates the event isn't mapped to a
+		 * Note, a negative index indicates the event isn't mapped to a
 		 * physical counter in the host, e.g. due to contention.
 		 */
 		hw_idx = pmc->perf_event->hw.idx;
-		if (hw_idx != pmc->idx && hw_idx > -1)
-			host_cross_mapped_mask |= BIT_ULL(hw_idx);
+		if (hw_idx != pmc->idx)
+			continue;
+
+		pebs_enable |= BIT_ULL(pmc->idx);
 	}
-	return host_cross_mapped_mask;
+	return pebs_enable;
 }
 
 static bool intel_pmu_is_mediated_pmu_supported(struct x86_pmu_capability *host_pmu)
arch/x86/kvm/vmx/vmx.c +1 -10
index fbe3ce5f5a51..31675e5cf563 100644
@@ -7314,20 +7314,11 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
 		return;
 
 	struct x86_guest_pebs guest_pebs = {
-		.enable = pmu->pebs_enable,
+		.enable = intel_pmu_compute_pebs_enable(pmu),
 		.ds_area = pmu->ds_area,
 		.data_cfg = pmu->pebs_data_cfg,
 	};
 
-	/*
-	 * Disable counters where the guest PMC is different than the host PMC
-	 * being used on behalf of the guest, as the PEBS record includes
-	 * PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
-	 * wrong counter(s).
-	 */
-	if (guest_pebs.enable & pmu->global_ctrl)
-		guest_pebs.enable &= ~intel_pmu_get_cross_mapped_mask(pmu);
-
 	/* Note, nr_msrs may be garbage if perf_guest_get_msrs() returns NULL. */
 	msrs = perf_guest_get_msrs(&nr_msrs, &guest_pebs);
 	if (!msrs)
arch/x86/kvm/vmx/vmx.h +14 -1
index 0c4563472940..b055731efd2d 100644
@@ -659,7 +659,20 @@ static __always_inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu)
 	return container_of(vcpu, struct vcpu_vmx, vcpu);
 }
 
-u64 intel_pmu_get_cross_mapped_mask(struct kvm_pmu *pmu);
+u64 __intel_pmu_compute_pebs_enable(struct kvm_pmu *pmu);
+
+static inline u64 intel_pmu_compute_pebs_enable(struct kvm_pmu *pmu)
+{
+	/*
+	 * Avoid the function call overhead in the common case that the guest
+	 * isn't using PEBS.
+	 */
+	if (!(pmu->pebs_enable & pmu->global_ctrl))
+		return 0;
+
+	return __intel_pmu_compute_pebs_enable(pmu);
+}
+
 int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu);
 void vmx_passthrough_lbr_msrs(struct kvm_vcpu *vcpu);
 
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Reviewed-by on v3 9/9
#24 lore raw

Re: [PATCH v3 9/9] KVM: VMX: Only tell perf to enable PEBS counters for fully enabled PMCs

Quoted 112 lines · Sean Christopherson

When passing the guest's requested PEBS_ENABLE (or rather, KVM's version of PEBS_ENABLE on behalf of the guest), omit counters that are locally disable and/or don't have a perf event (due to contention), in addition to omitting counters that are cross-mapped in the host.

In practice, this should be a nop as perf will already have disabled the associated counter, i.e. cpuc->pebs_enabled should have been cleared, but paranoia is cheap, and the existing code _looks_ wrong.

---
 arch/x86/kvm/vmx/pmu_intel.c | 30 ++++++++++++++++--------------
 arch/x86/kvm/vmx/vmx.c       | 11 +----------
 arch/x86/kvm/vmx/vmx.h       | 15 ++++++++++++++-
 3 files changed, 31 insertions(+), 25 deletions(-)
arch/x86/kvm/vmx/pmu_intel.c +16 -14
index 659fe097b904..1e420c8bca9d 100644
@@ -736,34 +736,36 @@ static void intel_pmu_cleanup(struct kvm_vcpu *vcpu)
 		intel_pmu_release_guest_lbr_event(vcpu);
 }
 
-u64 intel_pmu_get_cross_mapped_mask(struct kvm_pmu *pmu)
+u64 __intel_pmu_compute_pebs_enable(struct kvm_pmu *pmu)
 {
-	u64 host_cross_mapped_mask;
+	u64 guest_pebs_enable = pmu->pebs_enable & pmu->global_ctrl;
+	u64 pebs_enable = 0;
 	struct kvm_pmc *pmc;
 	int bit, hw_idx;
 
 	/*
-	 * Provide a mask of counters that are cross-mapped between the guest
-	 * and the host, i.e. where a guest PMC is mapped to a host PMC with a
-	 * different index.  PEBS records hold a PERF_GLOBAL_STATUS snapshot,
-	 * and so PEBS-enabled counters need to hold the correct index so as
-	 * not to confuse the guest.
+	 * Omit counters that are locally disabled, don't have a perf event, or
+	 * ended up with a perf event that is using a different counter than
+	 * the guest, i.e. where the guest PMC is different than the host PMC
+	 * being used on behalf of the guest.  PEBS records include
+	 * PERF_GLOBAL_STATUS, and so using a counter with a different index
+	 * means the guest will see overflow status for the wrong counter(s).
 	 */
-	host_cross_mapped_mask = 0;
-
-	kvm_for_each_pmc(pmu, pmc, bit, (unsigned long *)&pmu->global_ctrl) {
+	kvm_for_each_pmc(pmu, pmc, bit, (unsigned long *)&guest_pebs_enable) {
 		if (!pmc_is_locally_enabled(pmc) || !pmc->perf_event)
 			continue;
 
 		/*
-		 * A negative index indicates the event isn't mapped to a
+		 * Note, a negative index indicates the event isn't mapped to a
 		 * physical counter in the host, e.g. due to contention.
 		 */
 		hw_idx = pmc->perf_event->hw.idx;
-		if (hw_idx != pmc->idx && hw_idx > -1)
-			host_cross_mapped_mask |= BIT_ULL(hw_idx);
+		if (hw_idx != pmc->idx)
+			continue;
+
+		pebs_enable |= BIT_ULL(pmc->idx);
 	}
-	return host_cross_mapped_mask;
+	return pebs_enable;
 }
 
 static bool intel_pmu_is_mediated_pmu_supported(struct x86_pmu_capability *host_pmu)
arch/x86/kvm/vmx/vmx.c +1 -10
index fbe3ce5f5a51..31675e5cf563 100644
@@ -7314,20 +7314,11 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
 		return;
 
 	struct x86_guest_pebs guest_pebs = {
-		.enable = pmu->pebs_enable,
+		.enable = intel_pmu_compute_pebs_enable(pmu),
 		.ds_area = pmu->ds_area,
 		.data_cfg = pmu->pebs_data_cfg,
 	};
 
-	/*
-	 * Disable counters where the guest PMC is different than the host PMC
-	 * being used on behalf of the guest, as the PEBS record includes
-	 * PERF_GLOBAL_STATUS, i.e. the guest will see overflow status for the
-	 * wrong counter(s).
-	 */
-	if (guest_pebs.enable & pmu->global_ctrl)
-		guest_pebs.enable &= ~intel_pmu_get_cross_mapped_mask(pmu);
-
 	/* Note, nr_msrs may be garbage if perf_guest_get_msrs() returns NULL. */
 	msrs = perf_guest_get_msrs(&nr_msrs, &guest_pebs);
 	if (!msrs)
arch/x86/kvm/vmx/vmx.h +6 -1
index 0c4563472940..b055731efd2d 100644
@@ -659,7 +659,20 @@ static __always_inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu)
 	return container_of(vcpu, struct vcpu_vmx, vcpu);
 }
 
-u64 intel_pmu_get_cross_mapped_mask(struct kvm_pmu *pmu);
+u64 __intel_pmu_compute_pebs_enable(struct kvm_pmu *pmu);
+
+static inline u64 intel_pmu_compute_pebs_enable(struct kvm_pmu *pmu)
+{
+	/*
+	 * Avoid the function call overhead in the common case that the guest

arch/x86/kvm/vmx/vmx.h static __always_inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) · line 677

Quoted 10 lines
arch/x86/kvm/vmx/vmx.h +8 -0
+	 * isn't using PEBS.
+	 */
+	if (!(pmu->pebs_enable & pmu->global_ctrl))
+		return 0;
+
+	return __intel_pmu_compute_pebs_enable(pmu);
+}
+
 int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu);
 void vmx_passthrough_lbr_msrs(struct kvm_vcpu *vcpu);
Quoted 1 line
author maintainer
#25 lore raw
In reply to Sean Christopherson
Quoted 18 lines · Sean Christopherson

Sean Christopherson (9):
perf/x86/intel: Ensure guest PEBS path doesn't set unwanted
PERF_GLOBAL_CTRL bits
perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU
has isolation
perf/x86/intel: Don't context switch DS_AREA (and PEBS config) if PEBS
is unused
perf/x86/intel: Make @data a mandatory param for
intel_guest_get_msrs()
perf/x86/intel: Invert names of intel_ctrl_{guest,host}_mask perf/x86: KVM: Have perf define a dedicated struct for getting guest
PEBS data
perf/x86/intel: KVM: Handle cross-mapped PEBS PMCs entirely within KVM
KVM: VMX: Drop a redundant pmu->global_ctrl check when processing
pebs_enable
KVM: VMX: Only tell perf to enable PEBS counters for fully enabled
PMCs

Quoted 10 lines
 arch/x86/events/core.c            |  5 +-
 arch/x86/events/intel/core.c      | 92 +++++++++++++++++++------------
 arch/x86/events/intel/lbr.c       |  2 +-
 arch/x86/events/perf_event.h      |  7 ++-
 arch/x86/include/asm/kvm_host.h   |  9 ---
 arch/x86/include/asm/perf_event.h | 11 +++-
 arch/x86/kvm/vmx/pmu_intel.c      | 28 +++++++---
 arch/x86/kvm/vmx/vmx.c            | 10 ++--
 arch/x86/kvm/vmx/vmx.h            | 15 ++++-
 9 files changed, 114 insertions(+), 65 deletions(-)

Ping. This still applies on Linus' tree and against tip/master.

author maintainer
#26 lore raw
Quoted 30 lines · Sean Christopherson
Quoted 18 lines · Sean Christopherson

Sean Christopherson (9):
perf/x86/intel: Ensure guest PEBS path doesn't set unwanted
PERF_GLOBAL_CTRL bits
perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU
has isolation
perf/x86/intel: Don't context switch DS_AREA (and PEBS config) if PEBS
is unused
perf/x86/intel: Make @data a mandatory param for
intel_guest_get_msrs()
perf/x86/intel: Invert names of intel_ctrl_{guest,host}_mask perf/x86: KVM: Have perf define a dedicated struct for getting guest
PEBS data
perf/x86/intel: KVM: Handle cross-mapped PEBS PMCs entirely within KVM
KVM: VMX: Drop a redundant pmu->global_ctrl check when processing
pebs_enable
KVM: VMX: Only tell perf to enable PEBS counters for fully enabled
PMCs

Quoted 10 lines
 arch/x86/events/core.c            |  5 +-
 arch/x86/events/intel/core.c      | 92 +++++++++++++++++++------------
 arch/x86/events/intel/lbr.c       |  2 +-
 arch/x86/events/perf_event.h      |  7 ++-
 arch/x86/include/asm/kvm_host.h   |  9 ---
 arch/x86/include/asm/perf_event.h | 11 +++-
 arch/x86/kvm/vmx/pmu_intel.c      | 28 +++++++---
 arch/x86/kvm/vmx/vmx.c            | 10 ++--
 arch/x86/kvm/vmx/vmx.h            | 15 ++++-
 9 files changed, 114 insertions(+), 65 deletions(-)
Quoted 1 line

Ping. This still applies on Linus' tree and against tip/master.

Ping again. This fixes a real issue and IMO provides much-needed improvements for the KVM<=>perf APIs.

maintainer
#27 lore raw
Quoted 31 lines · Sean Christopherson
Quoted 30 lines · Sean Christopherson
Quoted 28 lines · Sean Christopherson

Sean Christopherson (9):
perf/x86/intel: Ensure guest PEBS path doesn't set unwanted
PERF_GLOBAL_CTRL bits
perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU
has isolation
perf/x86/intel: Don't context switch DS_AREA (and PEBS config) if PEBS
is unused
perf/x86/intel: Make @data a mandatory param for
intel_guest_get_msrs()
perf/x86/intel: Invert names of intel_ctrl_{guest,host}_mask perf/x86: KVM: Have perf define a dedicated struct for getting guest
PEBS data
perf/x86/intel: KVM: Handle cross-mapped PEBS PMCs entirely within KVM
KVM: VMX: Drop a redundant pmu->global_ctrl check when processing
pebs_enable
KVM: VMX: Only tell perf to enable PEBS counters for fully enabled
PMCs

 arch/x86/events/core.c            |  5 +-
 arch/x86/events/intel/core.c      | 92 +++++++++++++++++++------------
 arch/x86/events/intel/lbr.c       |  2 +-
 arch/x86/events/perf_event.h      |  7 ++-
 arch/x86/include/asm/kvm_host.h   |  9 ---
 arch/x86/include/asm/perf_event.h | 11 +++-
 arch/x86/kvm/vmx/pmu_intel.c      | 28 +++++++---
 arch/x86/kvm/vmx/vmx.c            | 10 ++--
 arch/x86/kvm/vmx/vmx.h            | 15 ++++-
 9 files changed, 114 insertions(+), 65 deletions(-)
Quoted 4 lines
Quoted 1 line

Ping. This still applies on Linus' tree and against tip/master.

Ping again. This fixes a real issue and IMO provides much-needed improvements for the KVM<=>perf APIs.

Right, finally got around to staring at them. How did you want them merged?

AFAICT 2-4 have Fixes tags and should go in perf/urgent, while the rest goes in perf/core?

/me tries and finds patch 5 doesn't apply.

author maintainer
#28 lore raw
Quoted 34 lines · Peter Zijlstra
Quoted 33 lines · Sean Christopherson
Quoted 31 lines · Sean Christopherson
Quoted 18 lines · Sean Christopherson

Sean Christopherson (9):
perf/x86/intel: Ensure guest PEBS path doesn't set unwanted
PERF_GLOBAL_CTRL bits
perf/x86/intel: Don't write PEBS_ENABLED on host<=>guest xfers if CPU
has isolation
perf/x86/intel: Don't context switch DS_AREA (and PEBS config) if PEBS
is unused
perf/x86/intel: Make @data a mandatory param for
intel_guest_get_msrs()
perf/x86/intel: Invert names of intel_ctrl_{guest,host}_mask perf/x86: KVM: Have perf define a dedicated struct for getting guest
PEBS data
perf/x86/intel: KVM: Handle cross-mapped PEBS PMCs entirely within KVM
KVM: VMX: Drop a redundant pmu->global_ctrl check when processing
pebs_enable
KVM: VMX: Only tell perf to enable PEBS counters for fully enabled
PMCs

Quoted 10 lines
 arch/x86/events/core.c            |  5 +-
 arch/x86/events/intel/core.c      | 92 +++++++++++++++++++------------
 arch/x86/events/intel/lbr.c       |  2 +-
 arch/x86/events/perf_event.h      |  7 ++-
 arch/x86/include/asm/kvm_host.h   |  9 ---
 arch/x86/include/asm/perf_event.h | 11 +++-
 arch/x86/kvm/vmx/pmu_intel.c      | 28 +++++++---
 arch/x86/kvm/vmx/vmx.c            | 10 ++--
 arch/x86/kvm/vmx/vmx.h            | 15 ++++-
 9 files changed, 114 insertions(+), 65 deletions(-)

Ping. This still applies on Linus' tree and against tip/master.

Quoted 8 lines
Quoted 2 lines

Ping again. This fixes a real issue and IMO provides much-needed improvements for the KVM<=>perf APIs.

Right, finally got around to staring at them. How did you want them merged?

AFAICT 2-4 have Fixes tags and should go in perf/urgent, while the rest goes in perf/core?

I was going to say "just put everything in perf/core", but given that 7.3 is the likely next LTS, I agree that we should patch 2 in particular into 7.3. FWIW, after looking at this more, patch 1 is more of a fix than I realized (it too will have a Fixes tag on the next version). It *should* be a glorified nop, but is a decent defense-in-depth change, so is probably worth putting into perf/urgent as well. That would give a nice split of 1-4 to perf/urgent, and the rest to perf/core.

Quoted 1 line

/me tries and finds patch 5 doesn't apply.

I'll send a v6. After reading through all of this again, I agree with Dapeng[*] that it's worth fixing the flaw Sashiko pointed out (I'll tack on a patch).

Thanks!

[*] https://lore.kernel.org/all/[email protected]