perf/x86/intel: Expose EPT-friendly PEBS for SPR and future models
Subsystems (4)
PATCH v2 1/2 perf/x86/intel: Expose EPT-friendly PEBS for SPR and future models
From: Like Xu <[email protected]>
According to Intel SDM, the EPT-friendly PEBS is supported by all the platforms after ICX, ADL and the future platforms with PEBS format 5.
Currently the only in-kernel user of this capability is KVM, which has very limited support for hybrid core pmu, so ADL and its successors do not currently expose this capability. When both hybrid core and PEBS format 5 are present, KVM will decide on its own merits.
---
V1 -> V2 Changelog:
- the perf part should be a separate patch; (Kan)
- apply PEBS format 5 to avoid patching every future model; (Kan)
arch/x86/events/intel/core.c | 1 + arch/x86/events/intel/ds.c | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/events/intel/core.c +1 -0
@@ -6288,6 +6288,7 @@ __init int intel_pmu_init(void) x86_pmu.pebs_constraints = intel_spr_pebs_event_constraints; x86_pmu.extra_regs = intel_spr_extra_regs; x86_pmu.limit_period = spr_limit_period; + x86_pmu.pebs_ept =1 ; x86_pmu.pebs_aliases =NULL ; x86_pmu.pebs_prec_dist = true; x86_pmu.pebs_block = true;
arch/x86/events/intel/ds.c +3 -1
@@ -2253,8 +2253,10 @@ void __init intel_ds_init(void) x86_pmu.large_pebs_flags |=PERF_SAMPLE_TIME ;break ; -case 4 :case 5 : + x86_pmu.pebs_ept =1 ; + fallthrough; +case 4 : x86_pmu.drain_pebs = intel_pmu_drain_pebs_icl; x86_pmu.pebs_record_size =sizeof (struct pebs_basic );if (x86_pmu.intel_cap.pebs_baseline) {
Signature · 1 line
2.37.3
PATCH v2 2/2 KVM: x86/pmu: Add PEBS support for SPR and future non-hybird models
From: Like Xu <[email protected]>
Virtualization support for SPR PEBS has officially available in the Intel SDM (June 2022) and has been validated on late stepping machines:
Compared to Ice Lake Server, the PDIR counter available (Fixed 0) on SPR is unchanged, but the capability is enhanced to Instruction-Accurate PDIR (PDIR++), where PEBS is taken on the next instruction after the one that caused the overflow. Also, it introduces a new Precise Distribution (PDist) facility that eliminates the skid when a precise event is programmed on general programmable counter 0.
For guest usage, KVM will require the max precise level in both cases mentioned above (other conditions may apply later), requesting the correct hardware counter (PRIR++ or PDist) from host perf as usual.
---
Previous:
https://lore.kernel.org/kvm/[email protected]/
V1 -> V2 Changelog:
- move the check into a function; (Kan)
arch/x86/kvm/pmu.c | 25 ++++++++++++++++++++++--- arch/x86/kvm/vmx/capabilities.h | 4 +++- 2 files changed, 25 insertions(+), 4 deletions(-)
arch/x86/kvm/pmu.c +22 -3
@@ -28,9 +28,18 @@struct x86_pmu_capability __read_mostly kvm_pmu_cap;EXPORT_SYMBOL_GPL (kvm_pmu_cap); -static const struct x86_cpu_id vmx_icl_pebs_cpu[] = { +/* Precise Distribution of Instructions Retired (PDIR) */ +static const struct x86_cpu_id vmx_pebs_pdir_cpu[] = {X86_MATCH_INTEL_FAM6_MODEL (ICELAKE_D ,NULL ),X86_MATCH_INTEL_FAM6_MODEL (ICELAKE_X ,NULL ), +/* Instruction-Accurate PDIR (PDIR++) */ +X86_MATCH_INTEL_FAM6_MODEL (SAPPHIRERAPIDS_X ,NULL ), + {} +}; + +/* Precise Distribution (PDist) */ +static const struct x86_cpu_id vmx_pebs_pdist_cpu[] = { +X86_MATCH_INTEL_FAM6_MODEL (SAPPHIRERAPIDS_X ,NULL ), {} };
@@ -140,6 +149,16 @@ static void kvm_perf_overflow(struct perf_event *perf_event,__kvm_perf_overflow (pmc, true); } +static bool need_max_precise (struct kvm_pmc *pmc) +{ +if (pmc->idx ==0 &&x86_match_cpu (vmx_pebs_pdist_cpu)) +return true; +if (pmc->idx ==32 &&x86_match_cpu (vmx_pebs_pdir_cpu)) +return true; + +return false; +} +static void pmc_reprogram_counter (struct kvm_pmc *pmc,u32 type,u64 config,bool exclude_user,bool exclude_kernel,bool intr)
@@ -181,11 +200,11 @@ static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,* the accuracy of the PEBS profiling result, because the "event IP" * in the PEBS record is calibrated on the guest side. * -* On Icelake everything is fine. Other hardware (GLC+, TNT+) that +* On Icelake everything is fine. Other hardware (TNT+) that * could possibly care here is unsupported and needs changes. */ attr.precise_ip =1 ; -if (x86_match_cpu (vmx_icl_pebs_cpu) && pmc->idx ==32 ) +if (need_max_precise (pmc)) attr.precise_ip =3 ; }
arch/x86/kvm/vmx/capabilities.h +3 -1
@@ -398,7 +398,9 @@ static inline bool vmx_pt_mode_is_host_guest(void)static inline bool vmx_pebs_supported (void ) { -return boot_cpu_has (X86_FEATURE_PEBS ) && kvm_pmu_cap.pebs_ept; +return boot_cpu_has (X86_FEATURE_PEBS ) && + !boot_cpu_has (X86_FEATURE_HYBRID_CPU ) && + kvm_pmu_cap.pebs_ept; }static inline u64 vmx_get_perf_capabilities (void )
Signature · 1 line
2.37.3
Sean Christopherson
maintainerRe: [PATCH v2 2/2] KVM: x86/pmu: Add PEBS support for SPR and future non-hybird models
Kind of a nit, but I would prefer a shortlog that talks about the pdit/pdir stuff and not a "enable PEBS" bucket.
Quoted 18 lines · Like Xu
arch/x86/kvm/pmu.c +10
-0@@ -140,6 +149,16 @@ static void kvm_perf_overflow(struct perf_event *perf_event,__kvm_perf_overflow (pmc, true); } +static bool need_max_precise (struct kvm_pmc *pmc) +{ +if (pmc->idx ==0 &&x86_match_cpu (vmx_pebs_pdist_cpu)) +return true; +if (pmc->idx ==32 &&x86_match_cpu (vmx_pebs_pdir_cpu)) +return true; + +return false; +} +static void pmc_reprogram_counter (struct kvm_pmc *pmc,u32 type,u64 config,bool exclude_user,bool exclude_kernel,bool intr)
arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 204
Quoted 6 lines
arch/x86/kvm/pmu.c +1
-1@@ -181,11 +200,11 @@ static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,* the accuracy of the PEBS profiling result, because the "event IP" * in the PEBS record is calibrated on the guest side. * -* On Icelake everything is fine. Other hardware (GLC+, TNT+) that +* On Icelake everything is fine. Other hardware (TNT+) that * could possibly care here is unsupported and needs changes.
This part of the comment is still somewhat stale, and for me at least it's not at all helpful.
arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 208
Quoted 5 lines
arch/x86/kvm/pmu.c +1
-1*/ attr.precise_ip =1 ; -if (x86_match_cpu (vmx_icl_pebs_cpu) && pmc->idx ==32 ) +if (need_max_precise (pmc)) attr.precise_ip =3 ;
What about writing this as:
attr.precise_ip = pmc_get_pebs_precision(pmc);
(or whatever name is appropriate for "pebs_precision").
Then in the helper, add comments to explaint the magic numbers and the interaction with PDIST and PDIR. Bonus points if #defines for the the magic numbers can be added somewher
* 0 - SAMPLE_IP can have arbitrary skid * 1 - SAMPLE_IP must have constant skid * 2 - SAMPLE_IP requested to have 0 skid * 3 - SAMPLE_IP must have 0 skid
static u64 pmc_get_pebs_precision(struct kvm_pmc *pmc)
{
/* Comment that explains why PDIST/PDIR require 0 skid? */
if ((pmc->idx == 0 && x86_match_cpu(vmx_pebs_pdist_cpu)) ||
(pmc->idx == 32 && x86_match_cpu(vmx_pebs_pdir_cpu)))
return 3;
/* Comment about constant skid? */
return 1;
}arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pt_mode_is_host_guest(void) · line 402
Quoted 13 lines
(quoted diff) +0
-0}arch/x86/kvm/vmx/capabilities.h +2
-1@@ -398,7 +398,9 @@ static inline bool vmx_pt_mode_is_host_guest(void)static inline bool vmx_pebs_supported (void ) { -return boot_cpu_has (X86_FEATURE_PEBS ) && kvm_pmu_cap.pebs_ept; +return boot_cpu_has (X86_FEATURE_PEBS ) && + !boot_cpu_has(X86_FEATURE_HYBRID_CPU) &&
This belongs in a separate patch, and it should be ordered before patch 1 so that there's no window where KVM can see pebs_ept==1 on a hybrid CPU.
Actually, shouldn't everything in this patch land before core enabling?
arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pt_mode_is_host_guest(void) · line 403
Quoted 1 line
arch/x86/kvm/vmx/capabilities.h +1
-0+ kvm_pmu_cap.pebs_ept;
Please align indentation:
return boot_cpu_has(X86_FEATURE_PEBS) && !boot_cpu_has(X86_FEATURE_HYBRID_CPU) && kvm_pmu_cap.pebs_ept;
Re: [PATCH v2 2/2] KVM: x86/pmu: Add PEBS support for SPR and future non-hybird models
Quoted 23 lines · Sean Christopherson
Kind of a nit, but I would prefer a shortlog that talks about the pdit/pdir stuff and not a "enable PEBS" bucket.
Quoted 19 lines · Like Xu
arch/x86/kvm/pmu.c +10
-0@@ -140,6 +149,16 @@ static void kvm_perf_overflow(struct perf_event *perf_event,__kvm_perf_overflow (pmc, true); } +static bool need_max_precise (struct kvm_pmc *pmc) +{ +if (pmc->idx ==0 &&x86_match_cpu (vmx_pebs_pdist_cpu)) +return true; +if (pmc->idx ==32 &&x86_match_cpu (vmx_pebs_pdir_cpu)) +return true; + +return false; +} +static void pmc_reprogram_counter (struct kvm_pmc *pmc,u32 type,u64 config,bool exclude_user,bool exclude_kernel,bool intr)@@ -181,11 +200,11 @@ static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,* the accuracy of the PEBS profiling result, because the "event IP"
Quoted 8 lines
arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 204
Quoted 5 lines
arch/x86/kvm/pmu.c +1
-1* in the PEBS record is calibrated on the guest side. * -* On Icelake everything is fine. Other hardware (GLC+, TNT+) that +* On Icelake everything is fine. Other hardware (TNT+) that * could possibly care here is unsupported and needs changes. This part of the comment is still somewhat stale, and for me at least it's not at all helpful.
SPR has GLC core, and this patch adds this support and removes the unsupported statements. Not sure how else I may help you.
Quoted 9 lines
arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 208
Quoted 6 lines
arch/x86/kvm/pmu.c +1
-1*/ attr.precise_ip =1 ; -if (x86_match_cpu (vmx_icl_pebs_cpu) && pmc->idx ==32 ) +if (need_max_precise (pmc)) attr.precise_ip =3 ;What about writing this as:
Quoted 3 lines
attr.precise_ip = pmc_get_pebs_precision(pmc);(or whatever name is appropriate for "
pebs_precision").
The comment says, "the difference in the software precision levels of guest and host PEBS events will not affect the accuracy of the PEBS profiling result".
Quoted 4 lines
Then in the helper, add comments to explaint the magic numbers and the interaction with PDIST and PDIR. Bonus points if #defines for the the magic numbers can be added somewher
KVM just uses "precision_ip = 3" to request hw pdit/pdir counters from host perf, It only works at 3. For 1 or 2, there is no difference.
Quoted 7 lines
* 0 - SAMPLE_IP can have arbitrary skid * 1 - SAMPLE_IP must have constant skid * 2 - SAMPLE_IP requested to have 0 skid * 3 - SAMPLE_IP must have 0 skidstatic u64
pmc_get_pebs_precision(struct kvm_pmc *pmc)
Quoted 9 lines
{
/* Comment that explains why PDIST/PDIR require 0 skid? */
if ((pmc->idx == 0 && x86_match_cpu(vmx_pebs_pdist_cpu)) ||
(pmc->idx == 32 && x86_match_cpu(vmx_pebs_pdir_cpu)))
return 3;
/* Comment about constant skid? */
return 1;
}, therefore 0 or constant skid makes no difference.
Quoted 19 lines
Quoted 9 lines
(quoted diff) +0
-0}arch/x86/kvm/vmx/capabilities.h +0
-0arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pt_mode_is_host_guest(void) · line 402
Quoted 6 lines
arch/x86/kvm/vmx/capabilities.h +2
-1@@ -398,7 +398,9 @@ static inline bool vmx_pt_mode_is_host_guest(void)static inline bool vmx_pebs_supported (void ) { -return boot_cpu_has (X86_FEATURE_PEBS ) && kvm_pmu_cap.pebs_ept; +return boot_cpu_has (X86_FEATURE_PEBS ) && + !boot_cpu_has(X86_FEATURE_HYBRID_CPU) &&This belongs in a separate patch, and it should be ordered before patch 1 so that there's no window where KVM can see pebs_ept==1 on a hybrid CPU.
Quoted 1 line
Actually, shouldn't everything in this patch land before core enabling?
Perf core enabling about SPR is already upstreamed.
KVM support is not too late either, thanks to SPR's relentless delays.
Quoted 9 lines
arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pt_mode_is_host_guest(void) · line 403
Quoted 2 lines
arch/x86/kvm/vmx/capabilities.h +1
-0+ kvm_pmu_cap.pebs_ept;Please align indentation:
return boot_cpu_has(X86_FEATURE_PEBS) && !boot_cpu_has(X86_FEATURE_HYBRID_CPU) && kvm_pmu_cap.pebs_ept;
Sean Christopherson
maintainerRe: [PATCH v2 2/2] KVM: x86/pmu: Add PEBS support for SPR and future non-hybird models
Quoted 12 lines · Like Xu
Quoted 10 lines · Sean Christopherson
arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 204
Quoted 7 lines
arch/x86/kvm/pmu.c +1
-1@@ -181,11 +200,11 @@ static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,* the accuracy of the PEBS profiling result, because the "event IP" * in the PEBS record is calibrated on the guest side. * -* On Icelake everything is fine. Other hardware (GLC+, TNT+) that +* On Icelake everything is fine. Other hardware (TNT+) that * could possibly care here is unsupported and needs changes. This part of the comment is still somewhat stale, and for me at least it's not at all helpful.
Quoted 2 lines
SPR has GLC core, and this patch adds this support and removes the unsupported statements.
It's stale in the sense that it says "Icelake is fine", which implies that SPR is NOT fine.
Quoted 1 line
Not sure how else I may help you.
Write comments that don't require the reader to have in-depth knowledge of the PMU's magic numbers. Explain what "unsupported" means. Explain what changes might need to be made. Use search-friendly words, e.g. I assume TNT means Tremont, but Googling for "Intel TNT" doesn't yield anything useful.
The above comment essentially says "Icelake is good, Tremont is broken", but that doesn't help the reader understand what this code actually does.
Quoted 15 lines
Quoted 10 lines
arch/x86/kvm/pmu.c static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, · line 208
Quoted 4 lines
arch/x86/kvm/pmu.c +1
-1attr.precise_ip =1 ; -if (x86_match_cpu (vmx_icl_pebs_cpu) && pmc->idx ==32 ) +if (need_max_precise (pmc)) attr.precise_ip =3 ;What about writing this as:
attr.precise_ip = pmc_get_pebs_precision(pmc);(or whatever name is appropriate for "
pebs_precision").The comment says, "the difference in the software precision levels of guest and host PEBS events will not affect the accuracy of the PEBS profiling result".
Quoted 1 line
Quoted 6 lines
Quoted 3 lines
Then in the helper, add comments to explaint the magic numbers and the interaction with PDIST and PDIR. Bonus points if #defines for the the magic numbers can be added somewher
KVM just uses "
precision_ip= 3" to request hw pdit/pdir counters from host perf, It only works at 3. For 1 or 2, there is no difference.
What only works at 3? Are 0..3 architectural PMU values, or are they arbitrary numbers perf uses?
Quoted 13 lines
Quoted 13 lines
* 0 - SAMPLE_IP can have arbitrary skid * 1 - SAMPLE_IP must have constant skid * 2 - SAMPLE_IP requested to have 0 skid * 3 - SAMPLE_IP must have 0 skidstatic u64
pmc_get_pebs_precision(struct kvm_pmc *pmc){ /* Comment that explains why PDIST/PDIR require 0 skid? */ if ((pmc->idx == 0 && x86_match_cpu(vmx_pebs_pdist_cpu)) || (pmc->idx == 32 && x86_match_cpu(vmx_pebs_pdir_cpu))) return 3;
Quoted 4 lines
Quoted 3 lines
/* Comment about constant skid? */ return 1; }, therefore 0 or constant skid makes no difference.
Then why '1'?
Quoted 14 lines
Quoted 14 lines
arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pt_mode_is_host_guest(void) · line 401
Quoted 10 lines
arch/x86/kvm/vmx/capabilities.h +2
-1@@ -398,7 +398,9 @@ static inline bool vmx_pt_mode_is_host_guest(void)static inline bool vmx_pebs_supported (void ) { -return boot_cpu_has (X86_FEATURE_PEBS ) && kvm_pmu_cap.pebs_ept; +return boot_cpu_has (X86_FEATURE_PEBS ) && + !boot_cpu_has(X86_FEATURE_HYBRID_CPU) &&This belongs in a separate patch, and it should be ordered before patch 1 so that there's no window where KVM can see pebs_ept==1 on a hybrid CPU.
Quoted 3 lines
Quoted 1 line
Actually, shouldn't everything in this patch land before core enabling?
Perf core enabling about SPR is already upstreamed.
I'm referring to patch 1, "perf/x86/intel: Expose EPT-friendly PEBS for SPR and future models". Before that patch, x86_pmu.pebs_ept will be set only for Icelake, i.e. PEBS for KVM is effecitively not enabled in the core perf code until patch 1. KVM should get all its support lined up, and then enable PEBS support for SPR and beyond in core perf. E.g. if someone applied only patch 1, they would be able to use KVM PEBS on any CPU with PEBS v5, even if it were a hybrid CPU.
Quoted 10 lines · Like Xu
From: Like Xu <[email protected]>
According to Intel SDM, the EPT-friendly PEBS is supported by all the platforms after ICX, ADL and the future platforms with PEBS format 5.
Currently the only in-kernel user of this capability is KVM, which has very limited support for hybrid core pmu, so ADL and its successors do not currently expose this capability. When both hybrid core and PEBS format 5 are present, KVM will decide on its own merits.
Quoted 4 lines
Cc Peter Zijlstra <[email protected]>Suggested-by Kan Liang <[email protected]>Signed-off-by Like Xu <[email protected]>
Thanks,
Kan
Quoted 37 lines
---
V1 -> V2 Changelog:
- the perf part should be a separate patch; (Kan)
- apply PEBS format 5 to avoid patching every future model; (Kan)arch/x86/events/intel/core.c | 1 + arch/x86/events/intel/ds.c | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)arch/x86/events/intel/core.c +1
-0@@ -6288,6 +6288,7 @@ __init int intel_pmu_init(void) x86_pmu.pebs_constraints = intel_spr_pebs_event_constraints; x86_pmu.extra_regs = intel_spr_extra_regs; x86_pmu.limit_period = spr_limit_period; + x86_pmu.pebs_ept =1 ; x86_pmu.pebs_aliases =NULL ; x86_pmu.pebs_prec_dist = true; x86_pmu.pebs_block = true;arch/x86/events/intel/ds.c +3
-1@@ -2253,8 +2253,10 @@ void __init intel_ds_init(void) x86_pmu.large_pebs_flags |=PERF_SAMPLE_TIME ;break ; -case 4 :case 5 : + x86_pmu.pebs_ept =1 ; + fallthrough; +case 4 : x86_pmu.drain_pebs = intel_pmu_drain_pebs_icl; x86_pmu.pebs_record_size =sizeof (struct pebs_basic );if (x86_pmu.intel_cap.pebs_baseline) {