Skip to contentSkip to conversation

perf/x86/intel: Expose EPT-friendly PEBS for SPR and future models

PATCH In review Like Xu 3 participants 6 messages Sep 22, 2022 – Oct 14, 2022

Subsystems (4)
lore
Share & tools
lore

Like Xu

author
#0 lore raw

PATCH v2 1/2 perf/x86/intel: Expose EPT-friendly PEBS for SPR and future models

From: Like Xu <[email protected]>

According to Intel SDM, the EPT-friendly PEBS is supported by all the platforms after ICX, ADL and the future platforms with PEBS format 5.

Currently the only in-kernel user of this capability is KVM, which has very limited support for hybrid core pmu, so ADL and its successors do not currently expose this capability. When both hybrid core and PEBS format 5 are present, KVM will decide on its own merits.

---
V1 -> V2 Changelog:
- the perf part should be a separate patch; (Kan)
- apply PEBS format 5 to avoid patching every future model; (Kan)

 arch/x86/events/intel/core.c | 1 +
 arch/x86/events/intel/ds.c   | 4 +++-
 2 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -0
index 2db93498ff71..804540ba4599 100644
@@ -6288,6 +6288,7 @@ __init int intel_pmu_init(void)
 		x86_pmu.pebs_constraints = intel_spr_pebs_event_constraints;
 		x86_pmu.extra_regs = intel_spr_extra_regs;
 		x86_pmu.limit_period = spr_limit_period;
+		x86_pmu.pebs_ept = 1;
 		x86_pmu.pebs_aliases = NULL;
 		x86_pmu.pebs_prec_dist = true;
 		x86_pmu.pebs_block = true;
arch/x86/events/intel/ds.c +3 -1
index ba60427caa6d..4e937f685cdc 100644
@@ -2253,8 +2253,10 @@ void __init intel_ds_init(void)
 			x86_pmu.large_pebs_flags |= PERF_SAMPLE_TIME;
 			break;
 
-		case 4:
 		case 5:
+			x86_pmu.pebs_ept = 1;
+			fallthrough;
+		case 4:
 			x86_pmu.drain_pebs = intel_pmu_drain_pebs_icl;
 			x86_pmu.pebs_record_size = sizeof(struct pebs_basic);
 			if (x86_pmu.intel_cap.pebs_baseline) {
Signature · 1 line
2.37.3
author
#1 lore raw

PATCH v2 2/2 KVM: x86/pmu: Add PEBS support for SPR and future non-hybird models

From: Like Xu <[email protected]>

Virtualization support for SPR PEBS has officially available in the Intel SDM (June 2022) and has been validated on late stepping machines:

Compared to Ice Lake Server, the PDIR counter available (Fixed 0) on SPR is unchanged, but the capability is enhanced to Instruction-Accurate PDIR (PDIR++), where PEBS is taken on the next instruction after the one that caused the overflow. Also, it introduces a new Precise Distribution (PDist) facility that eliminates the skid when a precise event is programmed on general programmable counter 0.

For guest usage, KVM will require the max precise level in both cases mentioned above (other conditions may apply later), requesting the correct hardware counter (PRIR++ or PDist) from host perf as usual.

---
Previous:
https://lore.kernel.org/kvm/[email protected]/

V1 -> V2 Changelog:
- move the check into a function; (Kan)

 arch/x86/kvm/pmu.c              | 25 ++++++++++++++++++++++---
 arch/x86/kvm/vmx/capabilities.h |  4 +++-
 2 files changed, 25 insertions(+), 4 deletions(-)
arch/x86/kvm/pmu.c +22 -3
index 02f9e4f245bd..5e9b0b3ea42d 100644
@@ -28,9 +28,18 @@
 struct x86_pmu_capability __read_mostly kvm_pmu_cap;
 EXPORT_SYMBOL_GPL(kvm_pmu_cap);
 
-static const struct x86_cpu_id vmx_icl_pebs_cpu[] = {
+/* Precise Distribution of Instructions Retired (PDIR) */
+static const struct x86_cpu_id vmx_pebs_pdir_cpu[] = {
 	X86_MATCH_INTEL_FAM6_MODEL(ICELAKE_D, NULL),
 	X86_MATCH_INTEL_FAM6_MODEL(ICELAKE_X, NULL),
+	/* Instruction-Accurate PDIR (PDIR++) */
+	X86_MATCH_INTEL_FAM6_MODEL(SAPPHIRERAPIDS_X, NULL),
+	{}
+};
+
+/* Precise Distribution (PDist) */
+static const struct x86_cpu_id vmx_pebs_pdist_cpu[] = {
+	X86_MATCH_INTEL_FAM6_MODEL(SAPPHIRERAPIDS_X, NULL),
 	{}
 };
 
@@ -140,6 +149,16 @@ static void kvm_perf_overflow(struct perf_event *perf_event,
 	__kvm_perf_overflow(pmc, true);
 }
 
+static bool need_max_precise(struct kvm_pmc *pmc)
+{
+	if (pmc->idx == 0 && x86_match_cpu(vmx_pebs_pdist_cpu))
+		return true;
+	if (pmc->idx == 32 && x86_match_cpu(vmx_pebs_pdir_cpu))
+		return true;
+
+	return false;
+}
+
 static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,
 				  u64 config, bool exclude_user,
 				  bool exclude_kernel, bool intr)
@@ -181,11 +200,11 @@ static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,
 		 * the accuracy of the PEBS profiling result, because the "event IP"
 		 * in the PEBS record is calibrated on the guest side.
 		 *
-		 * On Icelake everything is fine. Other hardware (GLC+, TNT+) that
+		 * On Icelake everything is fine. Other hardware (TNT+) that
 		 * could possibly care here is unsupported and needs changes.
 		 */
 		attr.precise_ip = 1;
-		if (x86_match_cpu(vmx_icl_pebs_cpu) && pmc->idx == 32)
+		if (need_max_precise(pmc))
 			attr.precise_ip = 3;
 	}
 
arch/x86/kvm/vmx/capabilities.h +3 -1
index c5e5dfef69c7..4dc4bbe18821 100644
@@ -398,7 +398,9 @@ static inline bool vmx_pt_mode_is_host_guest(void)
 
 static inline bool vmx_pebs_supported(void)
 {
-	return boot_cpu_has(X86_FEATURE_PEBS) && kvm_pmu_cap.pebs_ept;
+	return boot_cpu_has(X86_FEATURE_PEBS) &&
+		!boot_cpu_has(X86_FEATURE_HYBRID_CPU) &&
+		kvm_pmu_cap.pebs_ept;
 }
 
 static inline u64 vmx_get_perf_capabilities(void)
Signature · 1 line
2.37.3
#2 lore raw

Re: [PATCH v2 2/2] KVM: x86/pmu: Add PEBS support for SPR and future non-hybird models

Kind of a nit, but I would prefer a shortlog that talks about the pdit/pdir stuff and not a "enable PEBS" bucket.

Quoted 18 lines · Like Xu
arch/x86/kvm/pmu.c +10 -0
@@ -140,6 +149,16 @@ static void kvm_perf_overflow(struct perf_event *perf_event,
 	__kvm_perf_overflow(pmc, true);
 }
 
+static bool need_max_precise(struct kvm_pmc *pmc)
+{
+	if (pmc->idx == 0 && x86_match_cpu(vmx_pebs_pdist_cpu))
+		return true;
+	if (pmc->idx == 32 && x86_match_cpu(vmx_pebs_pdir_cpu))
+		return true;
+
+	return false;
+}
+
 static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,
 				  u64 config, bool exclude_user,
 				  bool exclude_kernel, bool intr)

arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 204

Quoted 6 lines
arch/x86/kvm/pmu.c +1 -1
@@ -181,11 +200,11 @@ static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,
 		 * the accuracy of the PEBS profiling result, because the "event IP"
 		 * in the PEBS record is calibrated on the guest side.
 		 *
-		 * On Icelake everything is fine. Other hardware (GLC+, TNT+) that
+		 * On Icelake everything is fine. Other hardware (TNT+) that
 		 * could possibly care here is unsupported and needs changes.

This part of the comment is still somewhat stale, and for me at least it's not at all helpful.

arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 208

Quoted 5 lines
arch/x86/kvm/pmu.c +1 -1
 		 */
 		attr.precise_ip = 1;
-		if (x86_match_cpu(vmx_icl_pebs_cpu) && pmc->idx == 32)
+		if (need_max_precise(pmc))
 			attr.precise_ip = 3;

What about writing this as:

		attr.precise_ip = pmc_get_pebs_precision(pmc);

(or whatever name is appropriate for "pebs_precision").

Then in the helper, add comments to explaint the magic numbers and the interaction with PDIST and PDIR. Bonus points if #defines for the the magic numbers can be added somewher

				 *  0 - SAMPLE_IP can have arbitrary skid
				 *  1 - SAMPLE_IP must have constant skid
				 *  2 - SAMPLE_IP requested to have 0 skid
				 *  3 - SAMPLE_IP must have 0 skid

static u64 pmc_get_pebs_precision(struct kvm_pmc *pmc)

{
	/* Comment that explains why PDIST/PDIR require 0 skid? */
	if ((pmc->idx == 0 && x86_match_cpu(vmx_pebs_pdist_cpu)) ||
	    (pmc->idx == 32 && x86_match_cpu(vmx_pebs_pdir_cpu)))
		return 3;

	/* Comment about constant skid? */
	return 1;
}

arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pt_mode_is_host_guest(void) · line 402

Quoted 13 lines
(quoted diff) +0 -0
 	}
 
arch/x86/kvm/vmx/capabilities.h +2 -1
index c5e5dfef69c7..4dc4bbe18821 100644
@@ -398,7 +398,9 @@ static inline bool vmx_pt_mode_is_host_guest(void)
 
 static inline bool vmx_pebs_supported(void)
 {
-	return boot_cpu_has(X86_FEATURE_PEBS) && kvm_pmu_cap.pebs_ept;
+	return boot_cpu_has(X86_FEATURE_PEBS) &&
+		!boot_cpu_has(X86_FEATURE_HYBRID_CPU) &&

This belongs in a separate patch, and it should be ordered before patch 1 so that there's no window where KVM can see pebs_ept==1 on a hybrid CPU.

Actually, shouldn't everything in this patch land before core enabling?

arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pt_mode_is_host_guest(void) · line 403

Quoted 1 line
arch/x86/kvm/vmx/capabilities.h +1 -0
+		kvm_pmu_cap.pebs_ept;

Please align indentation:

	return boot_cpu_has(X86_FEATURE_PEBS) &&
	       !boot_cpu_has(X86_FEATURE_HYBRID_CPU) &&
	       kvm_pmu_cap.pebs_ept;
author
#3 lore raw

Re: [PATCH v2 2/2] KVM: x86/pmu: Add PEBS support for SPR and future non-hybird models

Quoted 23 lines · Sean Christopherson

Kind of a nit, but I would prefer a shortlog that talks about the pdit/pdir stuff and not a "enable PEBS" bucket.

Quoted 19 lines · Like Xu
arch/x86/kvm/pmu.c +10 -0
@@ -140,6 +149,16 @@ static void kvm_perf_overflow(struct perf_event *perf_event,
  	__kvm_perf_overflow(pmc, true);
  }
 
+static bool need_max_precise(struct kvm_pmc *pmc)
+{
+	if (pmc->idx == 0 && x86_match_cpu(vmx_pebs_pdist_cpu))
+		return true;
+	if (pmc->idx == 32 && x86_match_cpu(vmx_pebs_pdir_cpu))
+		return true;
+
+	return false;
+}
+
  static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,
  				  u64 config, bool exclude_user,
  				  bool exclude_kernel, bool intr)
@@ -181,11 +200,11 @@ static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,
  		 * the accuracy of the PEBS profiling result, because the "event IP"
Quoted 8 lines

arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 204

Quoted 5 lines
arch/x86/kvm/pmu.c +1 -1
  		 * in the PEBS record is calibrated on the guest side.
  		 *
-		 * On Icelake everything is fine. Other hardware (GLC+, TNT+) that
+		 * On Icelake everything is fine. Other hardware (TNT+) that
  		 * could possibly care here is unsupported and needs changes.

This part of the comment is still somewhat stale, and for me at least it's not at all helpful.

SPR has GLC core, and this patch adds this support and removes the unsupported statements. Not sure how else I may help you.

Quoted 9 lines

arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 208

Quoted 6 lines
arch/x86/kvm/pmu.c +1 -1
  		 */
  		attr.precise_ip = 1;
-		if (x86_match_cpu(vmx_icl_pebs_cpu) && pmc->idx == 32)
+		if (need_max_precise(pmc))
  			attr.precise_ip = 3;

What about writing this as:

Quoted 3 lines
		attr.precise_ip = pmc_get_pebs_precision(pmc);

(or whatever name is appropriate for "pebs_precision").

The comment says, "the difference in the software precision levels of guest and host PEBS events will not affect the accuracy of the PEBS profiling result".

Quoted 4 lines

Then in the helper, add comments to explaint the magic numbers and the interaction with PDIST and PDIR. Bonus points if #defines for the the magic numbers can be added somewher

KVM just uses "precision_ip = 3" to request hw pdit/pdir counters from host perf, It only works at 3. For 1 or 2, there is no difference.

Quoted 7 lines
				 *  0 - SAMPLE_IP can have arbitrary skid
				 *  1 - SAMPLE_IP must have constant skid
				 *  2 - SAMPLE_IP requested to have 0 skid
				 *  3 - SAMPLE_IP must have 0 skid

static u64 pmc_get_pebs_precision(struct kvm_pmc *pmc)

Quoted 9 lines
{
	/* Comment that explains why PDIST/PDIR require 0 skid? */
	if ((pmc->idx == 0 && x86_match_cpu(vmx_pebs_pdist_cpu)) ||
	    (pmc->idx == 32 && x86_match_cpu(vmx_pebs_pdir_cpu)))
		return 3;

	/* Comment about constant skid? */
	return 1;
}

, therefore 0 or constant skid makes no difference.

Quoted 19 lines
Quoted 9 lines
(quoted diff) +0 -0
  	}
 
arch/x86/kvm/vmx/capabilities.h +0 -0
index c5e5dfef69c7..4dc4bbe18821 100644

arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pt_mode_is_host_guest(void) · line 402

Quoted 6 lines
arch/x86/kvm/vmx/capabilities.h +2 -1
@@ -398,7 +398,9 @@ static inline bool vmx_pt_mode_is_host_guest(void)
 
  static inline bool vmx_pebs_supported(void)
  {
-	return boot_cpu_has(X86_FEATURE_PEBS) && kvm_pmu_cap.pebs_ept;
+	return boot_cpu_has(X86_FEATURE_PEBS) &&
+		!boot_cpu_has(X86_FEATURE_HYBRID_CPU) &&

This belongs in a separate patch, and it should be ordered before patch 1 so that there's no window where KVM can see pebs_ept==1 on a hybrid CPU.

Quoted 1 line

Actually, shouldn't everything in this patch land before core enabling?

Perf core enabling about SPR is already upstreamed.
KVM support is not too late either, thanks to SPR's relentless delays.

Quoted 9 lines

arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pt_mode_is_host_guest(void) · line 403

Quoted 2 lines
arch/x86/kvm/vmx/capabilities.h +1 -0
+		kvm_pmu_cap.pebs_ept;

Please align indentation:

	return boot_cpu_has(X86_FEATURE_PEBS) &&
	       !boot_cpu_has(X86_FEATURE_HYBRID_CPU) &&
	       kvm_pmu_cap.pebs_ept;
#4 lore raw

Re: [PATCH v2 2/2] KVM: x86/pmu: Add PEBS support for SPR and future non-hybird models

Quoted 12 lines · Like Xu
Quoted 10 lines · Sean Christopherson

arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 204

Quoted 7 lines
arch/x86/kvm/pmu.c +1 -1
@@ -181,11 +200,11 @@ static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,
  		 * the accuracy of the PEBS profiling result, because the "event IP"
  		 * in the PEBS record is calibrated on the guest side.
  		 *
-		 * On Icelake everything is fine. Other hardware (GLC+, TNT+) that
+		 * On Icelake everything is fine. Other hardware (TNT+) that
  		 * could possibly care here is unsupported and needs changes.

This part of the comment is still somewhat stale, and for me at least it's not at all helpful.

Quoted 2 lines

SPR has GLC core, and this patch adds this support and removes the unsupported statements.

It's stale in the sense that it says "Icelake is fine", which implies that SPR is NOT fine.

Quoted 1 line

Not sure how else I may help you.

Write comments that don't require the reader to have in-depth knowledge of the PMU's magic numbers. Explain what "unsupported" means. Explain what changes might need to be made. Use search-friendly words, e.g. I assume TNT means Tremont, but Googling for "Intel TNT" doesn't yield anything useful.

The above comment essentially says "Icelake is good, Tremont is broken", but that doesn't help the reader understand what this code actually does.

Quoted 15 lines
Quoted 10 lines

arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 208

Quoted 4 lines
arch/x86/kvm/pmu.c +1 -1
  		attr.precise_ip = 1;
-		if (x86_match_cpu(vmx_icl_pebs_cpu) && pmc->idx == 32)
+		if (need_max_precise(pmc))
  			attr.precise_ip = 3;

What about writing this as:

		attr.precise_ip = pmc_get_pebs_precision(pmc);

(or whatever name is appropriate for "pebs_precision").

The comment says, "the difference in the software precision levels of guest and host PEBS events will not affect the accuracy of the PEBS profiling result".

Quoted 1 line
Quoted 6 lines
Quoted 3 lines

Then in the helper, add comments to explaint the magic numbers and the interaction with PDIST and PDIR. Bonus points if #defines for the the magic numbers can be added somewher

KVM just uses "precision_ip = 3" to request hw pdit/pdir counters from host perf, It only works at 3. For 1 or 2, there is no difference.

What only works at 3? Are 0..3 architectural PMU values, or are they arbitrary numbers perf uses?

Quoted 13 lines
Quoted 13 lines
				 *  0 - SAMPLE_IP can have arbitrary skid
				 *  1 - SAMPLE_IP must have constant skid
				 *  2 - SAMPLE_IP requested to have 0 skid
				 *  3 - SAMPLE_IP must have 0 skid

static u64 pmc_get_pebs_precision(struct kvm_pmc *pmc)

{
	/* Comment that explains why PDIST/PDIR require 0 skid? */
	if ((pmc->idx == 0 && x86_match_cpu(vmx_pebs_pdist_cpu)) ||
	    (pmc->idx == 32 && x86_match_cpu(vmx_pebs_pdir_cpu)))
		return 3;
Quoted 4 lines
Quoted 3 lines
	/* Comment about constant skid? */
	return 1;
}

, therefore 0 or constant skid makes no difference.

Then why '1'?

Quoted 14 lines
Quoted 14 lines

arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pt_mode_is_host_guest(void) · line 401

Quoted 10 lines
arch/x86/kvm/vmx/capabilities.h +2 -1
index c5e5dfef69c7..4dc4bbe18821 100644
@@ -398,7 +398,9 @@ static inline bool vmx_pt_mode_is_host_guest(void)
  static inline bool vmx_pebs_supported(void)
  {
-	return boot_cpu_has(X86_FEATURE_PEBS) && kvm_pmu_cap.pebs_ept;
+	return boot_cpu_has(X86_FEATURE_PEBS) &&
+		!boot_cpu_has(X86_FEATURE_HYBRID_CPU) &&

This belongs in a separate patch, and it should be ordered before patch 1 so that there's no window where KVM can see pebs_ept==1 on a hybrid CPU.

Quoted 3 lines
Quoted 1 line

Actually, shouldn't everything in this patch land before core enabling?

Perf core enabling about SPR is already upstreamed.

I'm referring to patch 1, "perf/x86/intel: Expose EPT-friendly PEBS for SPR and future models". Before that patch, x86_pmu.pebs_ept will be set only for Icelake, i.e. PEBS for KVM is effecitively not enabled in the core perf code until patch 1. KVM should get all its support lined up, and then enable PEBS support for SPR and beyond in core perf. E.g. if someone applied only patch 1, they would be able to use KVM PEBS on any CPU with PEBS v5, even if it were a hybrid CPU.

Reviewed-by on v2 1/2
#5 lore raw
In reply to Like Xu
Quoted 10 lines · Like Xu

From: Like Xu <[email protected]>

According to Intel SDM, the EPT-friendly PEBS is supported by all the platforms after ICX, ADL and the future platforms with PEBS format 5.

Currently the only in-kernel user of this capability is KVM, which has very limited support for hybrid core pmu, so ADL and its successors do not currently expose this capability. When both hybrid core and PEBS format 5 are present, KVM will decide on its own merits.

Quoted 4 lines

Thanks,
Kan

Quoted 37 lines

---
V1 -> V2 Changelog:
- the perf part should be a separate patch; (Kan)
- apply PEBS format 5 to avoid patching every future model; (Kan)

 arch/x86/events/intel/core.c | 1 +
 arch/x86/events/intel/ds.c   | 4 +++-
 2 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -0
index 2db93498ff71..804540ba4599 100644
@@ -6288,6 +6288,7 @@ __init int intel_pmu_init(void)
 		x86_pmu.pebs_constraints = intel_spr_pebs_event_constraints;
 		x86_pmu.extra_regs = intel_spr_extra_regs;
 		x86_pmu.limit_period = spr_limit_period;
+		x86_pmu.pebs_ept = 1;
 		x86_pmu.pebs_aliases = NULL;
 		x86_pmu.pebs_prec_dist = true;
 		x86_pmu.pebs_block = true;
arch/x86/events/intel/ds.c +3 -1
index ba60427caa6d..4e937f685cdc 100644
@@ -2253,8 +2253,10 @@ void __init intel_ds_init(void)
 			x86_pmu.large_pebs_flags |= PERF_SAMPLE_TIME;
 			break;
 
-		case 4:
 		case 5:
+			x86_pmu.pebs_ept = 1;
+			fallthrough;
+		case 4:
 			x86_pmu.drain_pebs = intel_pmu_drain_pebs_icl;
 			x86_pmu.pebs_record_size = sizeof(struct pebs_basic);
 			if (x86_pmu.intel_cap.pebs_baseline) {