KVM: x86/pmu: Add support to enable guest PEBS via PT
Subsystems (4)
v3 cover KVM: x86/pmu: Add support to enable guest PEBS via PT
I recently noticed that some developers particularly like the PT feature. So please help review this version since a new kernel cycle has begun .
Intel new hardware (Atom processors based on the Tremont microarchitecture) introduces some Processor Event-Based Sampling (PEBS) extensions that output the PEBS record to Intel PT stream instead of DS area. The PEBS record will be packaged in a specific format when outputting to Intel PT buffer.
To use PEBS-via-PT, the guest driver will firstly check the basic support for PEBS-via-DS, so this patch set is based on the PEBS-via-DS enabling patch set [1].
We can use PEBS-via-PT feature on the Linux guest like native:
(you may need modprobe kvm-intel.ko with pt_mode=1)
Recording is selected by using the aux-output config term e.g.
$ perf record -c 10000 -e '{intel_pt/branch=0/,cycles/aux-output/ppp}' uname
Warning:
Intel Processor Trace: TSC not available
Linux
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.028 MB perf.data ]To display PEBS events from the Intel PT trace, use the itrace 'o' option e.g.
$ perf script --itrace=oe uname 853 113.230292: 10000 cycles/aux-output/ppp: ffffffff8125dcd9 perf_output_begin+0x29 ([kernel.kallsyms]) uname 853 113.230443: 10000 cycles/aux-output/ppp: ffffffff8106de86 native_write_msr+0x6 ([kernel.kallsyms]) uname 853 113.230444: 10000 cycles/aux-output/ppp: ffffffff81bd035b exc_nmi+0x10b ([kernel.kallsyms]) uname 853 113.230567: 10000 cycles/aux-output/ppp: ffffffff8106de86 native_write_msr+0x6 ([kernel.kallsyms]) uname 853 113.230567: 10000 cycles/aux-output/ppp: ffffffff8125dce0 perf_output_begin+0x30 ([kernel.kallsyms]) uname 853 113.230688: 10000 cycles/aux-output/ppp: ffffffff8106de86 native_write_msr+0x6 ([kernel.kallsyms]) uname 853 113.230689: 10000 cycles/aux-output/ppp: ffffffff81005da7 perf_event_nmi_handler+0x7 ([kernel.kallsyms]) uname 853 113.230816: 10000 cycles/aux-output/ppp: ffffffff8106de86 native_write_msr+0x6 ([kernel.kallsyms])
Please check more details in each commit and feel free to comment.
V2 -> V3 Changelog:
- Add x86_pmu.pebs_vmx to ATOM_TREMONT and support PDIR counter;
- Rewrite get_gp_pmc() and get_fixed_pmc() based on PERF_CAP_PEBS_OUTPUT_PT;
- Check and add counter reload registers in the intel_guest_get_msrs();
- Expose this capability in the vmx_get_perf_capabilities();
Previous:
https://lore.kernel.org/kvm/[email protected]/
[1] https://lore.kernel.org/kvm/[email protected]/
Like Xu (4):
KVM: x86/pmu: Add pebs_vmx support for ATOM_TREMONT
KVM: x86/pmu: Add counter reload MSR emulation for all counters
KVM: x86/pmu: Add counter reload registers to the MSR-load list
KVM: x86/pmu: Expose PEBS-via-PT in the KVM supported capabilities
Luwei Kang (1):
KVM: x86/pmu: Add the base address parameter for get_fixed_pmc()
arch/x86/events/intel/core.c | 28 +++++++++++++++++++++++++ arch/x86/events/perf_event.h | 5 ----- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/asm/msr-index.h | 6 ++++++ arch/x86/kvm/pmu.c | 5 ++--- arch/x86/kvm/pmu.h | 11 ++++++++-- arch/x86/kvm/vmx/capabilities.h | 5 ++++- arch/x86/kvm/vmx/pmu_intel.c | 35 ++++++++++++++++++++++++++------ arch/x86/kvm/vmx/vmx.h | 2 +- 9 files changed, 80 insertions(+), 18 deletions(-)
Signature · 1 line
2.31.1
PATCH v3 1/5 KVM: x86/pmu: Add pebs_vmx support for ATOM_TREMONT
The ATOM_TREMONT platform also supports the EPT-Friendly PEBS capability and we can also safely enable guest PEBS. Per Intel SDM, the PDIR counter on non-Ice Lake platforms is always GP counter 1;
--- arch/x86/events/intel/core.c | 1 + arch/x86/kvm/pmu.c | 5 ++--- 2 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/events/intel/core.c +1 -0
@@ -5826,6 +5826,7 @@ __init int intel_pmu_init(void)case INTEL_FAM6_ATOM_TREMONT_D :case INTEL_FAM6_ATOM_TREMONT :case INTEL_FAM6_ATOM_TREMONT_L : + x86_pmu.pebs_vmx =1 ; x86_pmu.late_ack = true;memcpy (hw_cache_event_ids, glp_hw_cache_event_ids,sizeof (hw_cache_event_ids));
arch/x86/kvm/pmu.c +2 -3
@@ -151,9 +151,8 @@ static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,* the accuracy of the PEBS profiling result, because the "event IP" * in the PEBS record is calibrated on the guest side. */ - attr.precise_ip =1 ; -if (x86_match_cpu (vmx_icl_pebs_cpu) && pmc->idx ==32 ) - attr.precise_ip =3 ; + attr.precise_ip =x86_match_cpu (vmx_icl_pebs_cpu) ? + ((pmc->idx ==32 ) ?3 :1 ) : ((pmc->idx ==1 ) ?3 :1 ); } event = perf_event_create_kernel_counter(&attr, -1, current,
Signature · 1 line
2.31.1
PATCH v3 2/5 KVM: x86/pmu: Add the base address parameter for get_fixed_pmc()
In reply to Like XuFrom: Luwei Kang <[email protected]>
Introduce a new base MSR address pass-in parameter for get_fixed_pmc() so that the caller can define its base for fixed counters (such as MSR_RELOAD_FIXED_CTRx that can be used in the PBES-via-PT feature).
Refactor the code using the current value MSR_CORE_PERF_FIXED_CTR0.
---
The checkpatch.pl yell "ERROR: do not use assignment in if condition", should I fix the orignal coding style ?
arch/x86/kvm/pmu.h | 5 ++--- arch/x86/kvm/vmx/pmu_intel.c | 17 +++++++++++------ 2 files changed, 13 insertions(+), 9 deletions(-)
arch/x86/kvm/pmu.h +2 -3
@@ -126,10 +126,9 @@ static inline struct kvm_pmc *get_gp_pmc(struct kvm_pmu *pmu, u32 msr, }/* returns fixed PMC with the specified MSR */ -static inline struct kvm_pmc *get_fixed_pmc (struct kvm_pmu *pmu,u32 msr) +static inline struct kvm_pmc *get_fixed_pmc (struct kvm_pmu *pmu, +u32 msr,u32 base) { -int base =MSR_CORE_PERF_FIXED_CTR0 ; -if (msr >= base && msr < base + pmu->nr_arch_fixed_counters) {u32 index =array_index_nospec (msr - base, pmu->nr_arch_fixed_counters);
arch/x86/kvm/vmx/pmu_intel.c +11 -6
@@ -44,7 +44,8 @@ static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)u8 old_ctrl =fixed_ctrl_field (pmu->fixed_ctr_ctrl, i);struct kvm_pmc *pmc; - pmc =get_fixed_pmc (pmu,MSR_CORE_PERF_FIXED_CTR0 + i); + pmc =get_fixed_pmc (pmu,MSR_CORE_PERF_FIXED_CTR0 + i, +MSR_CORE_PERF_FIXED_CTR0 );if (old_ctrl == new_ctrl)continue ;
@@ -114,7 +115,8 @@ static struct kvm_pmc *intel_pmc_idx_to_pmc(struct kvm_pmu *pmu, int pmc_idx)else {u32 idx = pmc_idx -INTEL_PMC_IDX_FIXED ; -return get_fixed_pmc (pmu, idx +MSR_CORE_PERF_FIXED_CTR0 ); +return get_fixed_pmc (pmu, idx +MSR_CORE_PERF_FIXED_CTR0 , +MSR_CORE_PERF_FIXED_CTR0 ); } }
@@ -222,7 +224,8 @@ static bool intel_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)default : ret =get_gp_pmc (pmu, msr,MSR_IA32_PERFCTR0 ) ||get_gp_pmc (pmu, msr,MSR_P6_EVNTSEL0 ) || -get_fixed_pmc (pmu, msr) ||get_fw_gp_pmc (pmu, msr) || +get_fixed_pmc (pmu, msr,MSR_CORE_PERF_FIXED_CTR0 ) || +get_fw_gp_pmc (pmu, msr) ||intel_pmu_is_valid_lbr_msr (vcpu, msr);break ; }
@@ -235,7 +238,7 @@ static struct kvm_pmc *intel_msr_idx_to_pmc(struct kvm_vcpu *vcpu, u32 msr)struct kvm_pmu *pmu =vcpu_to_pmu (vcpu);struct kvm_pmc *pmc; - pmc =get_fixed_pmc (pmu, msr); + pmc =get_fixed_pmc (pmu, msr,MSR_CORE_PERF_FIXED_CTR0 ); pmc = pmc ? pmc :get_gp_pmc (pmu, msr,MSR_P6_EVNTSEL0 ); pmc = pmc ? pmc :get_gp_pmc (pmu, msr,MSR_IA32_PERFCTR0 );
@@ -382,7 +385,8 @@ static int intel_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) msr_info->data = val & pmu->counter_bitmask[KVM_PMC_GP ];return 0 ; - }else if ((pmc =get_fixed_pmc (pmu, msr))) { + }else if ((pmc =get_fixed_pmc (pmu, msr, +MSR_CORE_PERF_FIXED_CTR0 ))) {u64 val =pmc_read_counter (pmc); msr_info->data = val & pmu->counter_bitmask[KVM_PMC_FIXED ];
@@ -470,7 +474,8 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)perf_event_period (pmc->perf_event,get_sample_period (pmc, data));return 0 ; - }else if ((pmc =get_fixed_pmc (pmu, msr))) { + }else if ((pmc =get_fixed_pmc (pmu, msr, +MSR_CORE_PERF_FIXED_CTR0 ))) { pmc->counter += data -pmc_read_counter (pmc);if (pmc->perf_event) perf_event_period(pmc->perf_event,
Signature · 1 line
2.31.1
PATCH v3 3/5 KVM: x86/pmu: Add counter reload MSR emulation for all counters
In reply to Like XuThe Intel PEBS-via-PT feature introduces a new output mechanism that directs PEBS records to the PT buffer, and after each PEBS record is generated, it automatically reloads the counter values from a new set of "reload values" MSRs (based on MSR_RELOAD_FIXED_CTRx and MSR_RELOAD_PMCx), instead of the counter reload values stored in the DS management area.
If perf_capabilities supports this capability, PEBS records will be directed to the PT buffer when the relevant bit in pebs_enable is set.
--- arch/x86/events/perf_event.h | 5 ----- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/asm/msr-index.h | 6 ++++++ arch/x86/kvm/pmu.h | 8 ++++++++ arch/x86/kvm/vmx/pmu_intel.c | 18 ++++++++++++++++++ 5 files changed, 33 insertions(+), 5 deletions(-)
arch/x86/events/perf_event.h +0 -5
@@ -115,11 +115,6 @@ struct amd_nb { };#define PEBS_COUNTER_MASK ((1ULL << MAX_PEBS_EVENTS) - 1) -#define PEBS_PMI_AFTER_EACH_RECORD BIT_ULL(60) -#define PEBS_OUTPUT_OFFSET 61 -#define PEBS_OUTPUT_MASK (3ull << PEBS_OUTPUT_OFFSET) -#define PEBS_OUTPUT_PT (1ull << PEBS_OUTPUT_OFFSET) -#define PEBS_VIA_PT_MASK (PEBS_OUTPUT_PT | PEBS_PMI_AFTER_EACH_RECORD)/* * Flags PEBS can handle without an PMI.
arch/x86/include/asm/kvm_host.h +1 -0
@@ -443,6 +443,7 @@ struct kvm_pmc {u8 idx;u64 counter;u64 eventsel; +u64 reload_counter;struct perf_event *perf_event;struct kvm_vcpu *vcpu;/*
arch/x86/include/asm/msr-index.h +6 -0
@@ -187,12 +187,18 @@#define MSR_IA32_PERF_CAPABILITIES 0x00000345#define PERF_CAP_METRICS_IDX 15#define PERF_CAP_PT_IDX 16 +#define PEBS_PMI_AFTER_EACH_RECORD BIT_ULL(60) +#define PEBS_OUTPUT_OFFSET 61 +#define PEBS_OUTPUT_MASK (3ull << PEBS_OUTPUT_OFFSET) +#define PEBS_OUTPUT_PT (1ull << PEBS_OUTPUT_OFFSET) +#define PEBS_VIA_PT_MASK (PEBS_OUTPUT_PT | PEBS_PMI_AFTER_EACH_RECORD)#define MSR_PEBS_LD_LAT_THRESHOLD 0x000003f6#define PERF_CAP_PEBS_TRAP BIT_ULL(6)#define PERF_CAP_ARCH_REG BIT_ULL(7)#define PERF_CAP_PEBS_FORMAT 0xf00#define PERF_CAP_PEBS_BASELINE BIT_ULL(14) +#define PERF_CAP_PEBS_OUTPUT_PT BIT_ULL(16)#define PERF_CAP_PEBS_MASK (PERF_CAP_PEBS_TRAP | PERF_CAP_ARCH_REG | \ PERF_CAP_PEBS_FORMAT | PERF_CAP_PEBS_BASELINE)
arch/x86/kvm/pmu.h +8 -0
@@ -115,6 +115,10 @@ static inline bool kvm_valid_perf_global_ctrl(struct kvm_pmu *pmu,static inline struct kvm_pmc *get_gp_pmc (struct kvm_pmu *pmu,u32 msr,u32 base) { +if ((msr ==MSR_RELOAD_PMC0 || msr ==MSR_RELOAD_FIXED_CTR0 ) && + !(pmu_to_vcpu (pmu)->arch.perf_capabilities &PERF_CAP_PEBS_OUTPUT_PT )) +return NULL ; +if (msr >= base && msr < base + pmu->nr_arch_gp_counters) {u32 index =array_index_nospec (msr - base, pmu->nr_arch_gp_counters);
@@ -129,6 +133,10 @@ static inline struct kvm_pmc *get_gp_pmc(struct kvm_pmu *pmu, u32 msr,static inline struct kvm_pmc *get_fixed_pmc (struct kvm_pmu *pmu,u32 msr,u32 base) { +if ((msr ==MSR_RELOAD_PMC0 || msr ==MSR_RELOAD_FIXED_CTR0 ) && + !(pmu_to_vcpu (pmu)->arch.perf_capabilities &PERF_CAP_PEBS_OUTPUT_PT )) +return NULL ; +if (msr >= base && msr < base + pmu->nr_arch_fixed_counters) {u32 index =array_index_nospec (msr - base, pmu->nr_arch_fixed_counters);
arch/x86/kvm/vmx/pmu_intel.c +18 -0
@@ -226,6 +226,8 @@ static bool intel_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)get_gp_pmc (pmu, msr,MSR_P6_EVNTSEL0 ) ||get_fixed_pmc (pmu, msr,MSR_CORE_PERF_FIXED_CTR0 ) ||get_fw_gp_pmc (pmu, msr) || +get_gp_pmc (pmu, msr,MSR_RELOAD_PMC0 ) || +get_fixed_pmc (pmu, msr,MSR_RELOAD_FIXED_CTR0 ) ||intel_pmu_is_valid_lbr_msr (vcpu, msr);break ; }
@@ -241,6 +243,8 @@ static struct kvm_pmc *intel_msr_idx_to_pmc(struct kvm_vcpu *vcpu, u32 msr) pmc =get_fixed_pmc (pmu, msr,MSR_CORE_PERF_FIXED_CTR0 ); pmc = pmc ? pmc :get_gp_pmc (pmu, msr,MSR_P6_EVNTSEL0 ); pmc = pmc ? pmc :get_gp_pmc (pmu, msr,MSR_IA32_PERFCTR0 ); + pmc = pmc ? pmc :get_gp_pmc (pmu, msr,MSR_RELOAD_PMC0 ); + pmc = pmc ? pmc :get_fixed_pmc (pmu, msr,MSR_RELOAD_FIXED_CTR0 );return pmc; }
@@ -394,6 +398,10 @@ static int intel_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) }else if ((pmc =get_gp_pmc (pmu, msr,MSR_P6_EVNTSEL0 ))) { msr_info->data = pmc->eventsel;return 0 ; + }else if ((pmc =get_gp_pmc (pmu, msr,MSR_RELOAD_PMC0 )) || + (pmc =get_fixed_pmc (pmu, msr,MSR_RELOAD_FIXED_CTR0 ))) { + msr_info->data = pmc->reload_counter; +return 0 ; }else if (intel_pmu_handle_lbr_msrs_access (vcpu, msr_info, true))return 0 ; }
@@ -488,6 +496,12 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)reprogram_gp_counter (pmc, data);return 0 ; } + }else if ((pmc =get_gp_pmc (pmu, msr,MSR_RELOAD_PMC0 )) || + (pmc =get_fixed_pmc (pmu, msr,MSR_RELOAD_FIXED_CTR0 ))) { +if (!(data & ~pmc_bitmask (pmc))) { + pmc->reload_counter = data; +return 0 ; + } }else if (intel_pmu_handle_lbr_msrs_access (vcpu, msr_info, false))return 0 ; }
@@ -595,6 +609,8 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) pmu->pebs_enable_mask = ~((1ull << pmu->nr_arch_gp_counters) -1 ); } +if (vcpu->arch.perf_capabilities &PERF_CAP_PEBS_OUTPUT_PT ) + pmu->pebs_enable_mask &= ~PEBS_VIA_PT_MASK ; }else { vcpu->arch.ia32_misc_enable_msr |=MSR_IA32_MISC_ENABLE_PEBS_UNAVAIL ; vcpu->arch.perf_capabilities &= ~PERF_CAP_PEBS_MASK ;
@@ -612,6 +628,7 @@ static void intel_pmu_init(struct kvm_vcpu *vcpu) pmu->gp_counters[i].vcpu = vcpu; pmu->gp_counters[i].idx = i; pmu->gp_counters[i].current_config =0 ; + pmu->gp_counters[i].reload_counter =0 ; }for (i =0 ; i <INTEL_PMC_MAX_FIXED ; i++) {
@@ -619,6 +636,7 @@ static void intel_pmu_init(struct kvm_vcpu *vcpu) pmu->fixed_counters[i].vcpu = vcpu; pmu->fixed_counters[i].idx = i +INTEL_PMC_IDX_FIXED ; pmu->fixed_counters[i].current_config =0 ; + pmu->fixed_counters[i].reload_counter =0 ; } vcpu->arch.perf_capabilities =vmx_get_perf_capabilities ();
Signature · 1 line
2.31.1
PATCH v3 4/5 KVM: x86/pmu: Add counter reload registers to the MSR-load list
In reply to Like XuThe guest counter reload registers need to be loaded to real HW before VM-entry. Taking into account the existing guest PT implementation, we add those counter reload registers to MSR-load list when the corresponding PEBS counters are enabled and the optimization from clear_atomic_switch_msr() can be reused.
To support that, it needs to expand the value of NR_LOADSTORE_MSRS from 8 to 16 because when all counters are enabled, up to 7 or 8 counter reload registers need to be added into the MSR-load list.
--- arch/x86/events/intel/core.c | 27 +++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.h | 2 +- 2 files changed, 28 insertions(+), 1 deletion(-)
arch/x86/events/intel/core.c +27 -0
@@ -3903,6 +3903,8 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data)u64 intel_ctrl =hybrid (cpuc->pmu, intel_ctrl);u64 pebs_mask = (x86_pmu.flags &PMU_FL_PEBS_ALL ) ? cpuc->pebs_enabled : (cpuc->pebs_enabled &PEBS_COUNTER_MASK ); +u64 guest_pebs_enable, base, idx, host_reload_ctr; +unsigned long bit; *nr =0 ; arr[(*nr)++] = (struct perf_guest_switch_msr ){
@@ -3964,7 +3966,32 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) arr[0 ].guest |= arr[*nr].guest; } + guest_pebs_enable = arr[*nr].guest; ++(*nr); + +if (!x86_pmu.intel_cap.pebs_output_pt_available || + !(guest_pebs_enable &PEBS_OUTPUT_PT )) +return arr; + +for_each_set_bit (bit, (unsigned long *)&guest_pebs_enable, +X86_PMC_IDX_MAX ) { + base = (bit <INTEL_PMC_IDX_FIXED ) ? +MSR_RELOAD_PMC0 :MSR_RELOAD_FIXED_CTR0 ; + idx = (bit <INTEL_PMC_IDX_FIXED ) ? + bit : (bit -INTEL_PMC_IDX_FIXED ); + +/* It's good when the pebs counters are not cross-mapped. */ +rdmsrl (base, host_reload_ctr); + + arr[(*nr)++] = (struct perf_guest_switch_msr ){ + .msr = base, + .host = host_reload_ctr, + .guest = (bit <INTEL_PMC_IDX_FIXED ) ? + pmu->gp_counters[bit].reload_counter : + pmu->fixed_counters[bit].reload_counter, + }; + } +return arr; }
arch/x86/kvm/vmx/vmx.h +1 -1
@@ -28,7 +28,7 @@ extern const u32 vmx_msr_index[];#define MAX_NR_USER_RETURN_MSRS 4#endif -#define MAX_NR_LOADSTORE_MSRS 8 +#define MAX_NR_LOADSTORE_MSRS 16struct vmx_msrs {unsigned int nr;
Signature · 1 line
2.31.1
PATCH v3 5/5 KVM: x86/pmu: Expose PEBS-via-PT in the KVM supported capabilities
In reply to Like XuThe hypervisor userspace can dis/enable it via the MSR-based feature "MSR_IA32_PERF_CAPABILITIES [bit 16]". If guest also has basic PT support, it can output the PEBS records to the PT buffer.
--- arch/x86/kvm/vmx/capabilities.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/capabilities.h +4 -1
@@ -398,8 +398,11 @@ static inline u64 vmx_get_perf_capabilities(void) perf_cap |= host_perf_cap &PMU_CAP_LBR_FMT ; -if (vmx_pebs_supported ()) +if (vmx_pebs_supported ()) { perf_cap |= host_perf_cap &PERF_CAP_PEBS_MASK ; +if (vmx_pt_mode_is_host_guest ()) + perf_cap |= host_perf_cap &PERF_CAP_PEBS_OUTPUT_PT ; + }return perf_cap; }
Signature · 1 line
2.31.1