Support Perf Extension on AMD KVM guests
v5 cover Support Perf Extension on AMD KVM guests
This patchset adds support for Perf Extension on AMD KVM guests.
When perf runs on a guest with family = 15h || 17h, the MSRs that are accessed, when the Perf Extension flag is made available, differ from the existing K7 MSRs. The accesses are to the AMD Core Performance Extension counters which provide 2 extra counters and new MSRs for both the event select and counter registers.
Since the new event select and counter MSRs are interleaved and K7 MSRs are contiguous, the logic to map them to the gp_counters[] is changed.
This patchset has been tested with Family 17h and Opteron G1 guests.
v1->v2:
* Rearranged MSR #defines based on Boris's suggestion.
v2->v3:
* Changed the logic of mapping MSR to gp_counters[] index based on Boris's feedback.
* Removed use of family checks based on Radim's feedback.
* Removed KVM bugfix patch since it is already applied.
v3->v4:
* Rebased to latest KVM tree.
v4->v5:
* Removed conditional check when exposing Perf Extension flag to guests based on Radim's feedback.
Janakarajan Natarajan (3):
x86/msr: Add AMD Core Perf Extension MSRs x86/kvm: Add support for AMD Core Perf Extension in guest x86/kvm: Expose AMD Core Perf Extension flag to guests
arch/x86/include/asm/msr-index.h | 14 ++++ arch/x86/kvm/cpuid.c | 2 +- arch/x86/kvm/pmu_amd.c | 140 +++++++++++++++++++++++++++++++++++---- arch/x86/kvm/x86.c | 1 + 4 files changed, 142 insertions(+), 15 deletions(-)
Signature · 1 line
2.7.4
PATCH v5 1/3 x86/msr: Add AMD Core Perf Extension MSRs
Add the EventSelect and Counter MSRs for AMD Core Perf Extension.
--- arch/x86/include/asm/msr-index.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
arch/x86/include/asm/msr-index.h +14 -0
@@ -341,7 +341,21 @@/* Fam 15h MSRs */ #define MSR_F15H_PERF_CTL 0xc0010200 +#define MSR_F15H_PERF_CTL0 MSR_F15H_PERF_CTL +#define MSR_F15H_PERF_CTL1 (MSR_F15H_PERF_CTL + 2) +#define MSR_F15H_PERF_CTL2 (MSR_F15H_PERF_CTL + 4) +#define MSR_F15H_PERF_CTL3 (MSR_F15H_PERF_CTL + 6) +#define MSR_F15H_PERF_CTL4 (MSR_F15H_PERF_CTL + 8) +#define MSR_F15H_PERF_CTL5 (MSR_F15H_PERF_CTL + 10) +#define MSR_F15H_PERF_CTR 0xc0010201 +#define MSR_F15H_PERF_CTR0 MSR_F15H_PERF_CTR +#define MSR_F15H_PERF_CTR1 (MSR_F15H_PERF_CTR + 2) +#define MSR_F15H_PERF_CTR2 (MSR_F15H_PERF_CTR + 4) +#define MSR_F15H_PERF_CTR3 (MSR_F15H_PERF_CTR + 6) +#define MSR_F15H_PERF_CTR4 (MSR_F15H_PERF_CTR + 8) +#define MSR_F15H_PERF_CTR5 (MSR_F15H_PERF_CTR + 10) +#define MSR_F15H_NB_PERF_CTL 0xc0010240#define MSR_F15H_NB_PERF_CTR 0xc0010241#define MSR_F15H_PTSC 0xc0010280
Signature · 1 line
2.7.4
Re: [PATCH v5 1/3] x86/msr: Add AMD Core Perf Extension MSRs
2018-02-05 13:24-0600, Janakarajan Natarajan:
Quoted 20 lines
Add the EventSelect and Counter MSRs for AMD Core Perf Extension.
Signed-off-by Janakarajan Natarajan <[email protected]>--- arch/x86/include/asm/msr-index.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+)arch/x86/include/asm/msr-index.h +5
-0@@ -341,7 +341,21 @@/* Fam 15h MSRs */ #define MSR_F15H_PERF_CTL 0xc0010200 +#define MSR_F15H_PERF_CTL0 MSR_F15H_PERF_CTL +#define MSR_F15H_PERF_CTL1 (MSR_F15H_PERF_CTL + 2) +#define MSR_F15H_PERF_CTL2 (MSR_F15H_PERF_CTL + 4) +#define MSR_F15H_PERF_CTL3 (MSR_F15H_PERF_CTL + 6) +#define MSR_F15H_PERF_CTL4 (MSR_F15H_PERF_CTL + 8)
arch/x86/include/asm/msr-index.h line 358
Quoted 10 lines
arch/x86/include/asm/msr-index.h +9
-0+#define MSR_F15H_PERF_CTL5 (MSR_F15H_PERF_CTL + 10) +#define MSR_F15H_PERF_CTR 0xc0010201 +#define MSR_F15H_PERF_CTR0 MSR_F15H_PERF_CTR +#define MSR_F15H_PERF_CTR1 (MSR_F15H_PERF_CTR + 2) +#define MSR_F15H_PERF_CTR2 (MSR_F15H_PERF_CTR + 4) +#define MSR_F15H_PERF_CTR3 (MSR_F15H_PERF_CTR + 6) +#define MSR_F15H_PERF_CTR4 (MSR_F15H_PERF_CTR + 8) +#define MSR_F15H_PERF_CTR5 (MSR_F15H_PERF_CTR + 10) +
x86 maintainers,
are you ok with this going through the kvm tree?
Thanks.
Quoted 6 lines
arch/x86/include/asm/msr-index.h +0
-0#define MSR_F15H_NB_PERF_CTL 0xc0010240#define MSR_F15H_NB_PERF_CTR 0xc0010241#define MSR_F15H_PTSC 0xc0010280Signature · 1 line
2.7.4
Paolo Bonzini
maintainerRe: [PATCH v5 1/3] x86/msr: Add AMD Core Perf Extension MSRs
Quoted 19 lines · Radim Krcmar
Quoted 7 lines
/* Fam 15h MSRs */
#defineMSR_F15H_PERF_CTL0xc0010200arch/x86/include/asm/msr-index.h +5
-0+#define MSR_F15H_PERF_CTL0 MSR_F15H_PERF_CTL +#define MSR_F15H_PERF_CTL1 (MSR_F15H_PERF_CTL + 2) +#define MSR_F15H_PERF_CTL2 (MSR_F15H_PERF_CTL + 4) +#define MSR_F15H_PERF_CTL3 (MSR_F15H_PERF_CTL + 6) +#define MSR_F15H_PERF_CTL4 (MSR_F15H_PERF_CTL + 8)arch/x86/include/asm/msr-index.h line 358
Quoted 10 lines
arch/x86/include/asm/msr-index.h +9
-0+#define MSR_F15H_PERF_CTL5 (MSR_F15H_PERF_CTL + 10) +#define MSR_F15H_PERF_CTR 0xc0010201 +#define MSR_F15H_PERF_CTR0 MSR_F15H_PERF_CTR +#define MSR_F15H_PERF_CTR1 (MSR_F15H_PERF_CTR + 2) +#define MSR_F15H_PERF_CTR2 (MSR_F15H_PERF_CTR + 4) +#define MSR_F15H_PERF_CTR3 (MSR_F15H_PERF_CTR + 6) +#define MSR_F15H_PERF_CTR4 (MSR_F15H_PERF_CTR + 8) +#define MSR_F15H_PERF_CTR5 (MSR_F15H_PERF_CTR + 10) +x86 maintainers,
Quoted 1 line
are you ok with this going through the kvm tree?
Boris, can you ack these new MSRs?
Thanks,
Paolo
Thomas Gleixner
Acked-byRe: [PATCH v5 1/3] x86/msr: Add AMD Core Perf Extension MSRs
Quoted 20 lines · Paolo Bonzini
Quoted 19 lines · Radim Krcmar
Quoted 7 lines
/* Fam 15h MSRs */
#defineMSR_F15H_PERF_CTL0xc0010200arch/x86/include/asm/msr-index.h +5
-0+#define MSR_F15H_PERF_CTL0 MSR_F15H_PERF_CTL +#define MSR_F15H_PERF_CTL1 (MSR_F15H_PERF_CTL + 2) +#define MSR_F15H_PERF_CTL2 (MSR_F15H_PERF_CTL + 4) +#define MSR_F15H_PERF_CTL3 (MSR_F15H_PERF_CTL + 6) +#define MSR_F15H_PERF_CTL4 (MSR_F15H_PERF_CTL + 8)arch/x86/include/asm/msr-index.h line 358
Quoted 10 lines
arch/x86/include/asm/msr-index.h +9
-0+#define MSR_F15H_PERF_CTL5 (MSR_F15H_PERF_CTL + 10) +#define MSR_F15H_PERF_CTR 0xc0010201 +#define MSR_F15H_PERF_CTR0 MSR_F15H_PERF_CTR +#define MSR_F15H_PERF_CTR1 (MSR_F15H_PERF_CTR + 2) +#define MSR_F15H_PERF_CTR2 (MSR_F15H_PERF_CTR + 4) +#define MSR_F15H_PERF_CTR3 (MSR_F15H_PERF_CTR + 6) +#define MSR_F15H_PERF_CTR4 (MSR_F15H_PERF_CTR + 8) +#define MSR_F15H_PERF_CTR5 (MSR_F15H_PERF_CTR + 10) +x86 maintainers,
Quoted 1 line
Quoted 1 line
are you ok with this going through the kvm tree?
yes.
PATCH v5 2/3 x86/kvm: Add support for AMD Core Perf Extension in guest
In reply to Janakarajan NatarajanAdd support for AMD Core Performance counters in the guest. The base event select and counter MSRs are changed. In addition, with the core extension, there are 2 extra counters available for performance measurements for a total of 6.
With the new MSRs, the logic to map them to the gp_counters[] is changed. New functions are added to check the validity of the get/set MSRs.
If the guest has the X86_FEATURE_PERFCTR_CORE cpuid flag set, the number of counters available to the vcpu is set to 6. It the flag is not set then it is 4.
--- arch/x86/kvm/pmu_amd.c | 140 ++++++++++++++++++++++++++++++++++++++++++++----- arch/x86/kvm/x86.c | 1 + 2 files changed, 127 insertions(+), 14 deletions(-)
arch/x86/kvm/pmu_amd.c +126 -14
@@ -19,6 +19,21 @@#include "lapic.h" #include "pmu.h" +enum pmu_type { +PMU_TYPE_COUNTER =0 , +PMU_TYPE_EVNTSEL , +}; + +enum index { +INDEX_ZERO =0 , +INDEX_ONE , +INDEX_TWO , +INDEX_THREE , +INDEX_FOUR , +INDEX_FIVE , +INDEX_ERROR , +}; +/* duplicated from amd_perfmon_event_map, K7 and above should work. */ static struct kvm_event_hw_type_mapping amd_event_mapping[] = { [0 ] = {0x76 ,0x00 ,PERF_COUNT_HW_CPU_CYCLES },
@@ -31,6 +46,88 @@ static struct kvm_event_hw_type_mapping amd_event_mapping[] = { [7] = { 0xd1, 0x00, PERF_COUNT_HW_STALLED_CYCLES_BACKEND }, }; +static unsigned int get_msr_base (struct kvm_pmu *pmu,enum pmu_type type) +{ +struct kvm_vcpu *vcpu =pmu_to_vcpu (pmu); + +if (guest_cpuid_has (vcpu,X86_FEATURE_PERFCTR_CORE )) { +if (type ==PMU_TYPE_COUNTER ) +return MSR_F15H_PERF_CTR ; +else +return MSR_F15H_PERF_CTL ; + }else { +if (type ==PMU_TYPE_COUNTER ) +return MSR_K7_PERFCTR0 ; +else +return MSR_K7_EVNTSEL0 ; + } +} + +static enum index msr_to_index (u32 msr) +{ +switch (msr) { +case MSR_F15H_PERF_CTL0 : +case MSR_F15H_PERF_CTR0 : +case MSR_K7_EVNTSEL0 : +case MSR_K7_PERFCTR0 : +return INDEX_ZERO ; +case MSR_F15H_PERF_CTL1 : +case MSR_F15H_PERF_CTR1 : +case MSR_K7_EVNTSEL1 : +case MSR_K7_PERFCTR1 : +return INDEX_ONE ; +case MSR_F15H_PERF_CTL2 : +case MSR_F15H_PERF_CTR2 : +case MSR_K7_EVNTSEL2 : +case MSR_K7_PERFCTR2 : +return INDEX_TWO ; +case MSR_F15H_PERF_CTL3 : +case MSR_F15H_PERF_CTR3 : +case MSR_K7_EVNTSEL3 : +case MSR_K7_PERFCTR3 : +return INDEX_THREE ; +case MSR_F15H_PERF_CTL4 : +case MSR_F15H_PERF_CTR4 : +return INDEX_FOUR ; +case MSR_F15H_PERF_CTL5 : +case MSR_F15H_PERF_CTR5 : +return INDEX_FIVE ; +default : +return INDEX_ERROR ; + } +} + +static inline struct kvm_pmc *get_gp_pmc_amd (struct kvm_pmu *pmu,u32 msr, +enum pmu_type type) +{ +switch (msr) { +case MSR_F15H_PERF_CTL0 : +case MSR_F15H_PERF_CTL1 : +case MSR_F15H_PERF_CTL2 : +case MSR_F15H_PERF_CTL3 : +case MSR_F15H_PERF_CTL4 : +case MSR_F15H_PERF_CTL5 : + case MSR_K7_EVNTSEL0 ... MSR_K7_EVNTSEL3: +if (type !=PMU_TYPE_EVNTSEL ) +return NULL ; +break ; +case MSR_F15H_PERF_CTR0 : +case MSR_F15H_PERF_CTR1 : +case MSR_F15H_PERF_CTR2 : +case MSR_F15H_PERF_CTR3 : +case MSR_F15H_PERF_CTR4 : +case MSR_F15H_PERF_CTR5 : + case MSR_K7_PERFCTR0 ... MSR_K7_PERFCTR3: +if (type !=PMU_TYPE_COUNTER ) +return NULL ; +break ; +default : +return NULL ; + } + +return &pmu->gp_counters[msr_to_index (msr)]; +} +static unsigned amd_find_arch_event (struct kvm_pmu *pmu,u8 event_select,u8 unit_mask)
@@ -64,7 +161,18 @@ static bool amd_pmc_is_enabled(struct kvm_pmc *pmc)static struct kvm_pmc *amd_pmc_idx_to_pmc (struct kvm_pmu *pmu,int pmc_idx) { -return get_gp_pmc (pmu,MSR_K7_EVNTSEL0 + pmc_idx,MSR_K7_EVNTSEL0 ); +unsigned int base =get_msr_base (pmu,PMU_TYPE_COUNTER ); +struct kvm_vcpu *vcpu =pmu_to_vcpu (pmu); + +if (guest_cpuid_has (vcpu,X86_FEATURE_PERFCTR_CORE )) { +/* +* The idx is contiguous. The MSRs are not. The counter MSRs +* are interleaved with the event select MSRs. +*/ + pmc_idx *=2 ; + } + +return get_gp_pmc_amd (pmu, base + pmc_idx,PMU_TYPE_COUNTER ); }/* returns 0 if idx's corresponding MSR exists; otherwise returns 1. */
@@ -96,8 +204,8 @@ static bool amd_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)struct kvm_pmu *pmu =vcpu_to_pmu (vcpu);int ret = false; - ret =get_gp_pmc (pmu, msr,MSR_K7_PERFCTR0 ) || -get_gp_pmc (pmu, msr,MSR_K7_EVNTSEL0 ); + ret =get_gp_pmc_amd (pmu, msr,PMU_TYPE_COUNTER ) || +get_gp_pmc_amd (pmu, msr,PMU_TYPE_EVNTSEL );return ret; }
@@ -107,14 +215,14 @@ static int amd_pmu_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *data)struct kvm_pmu *pmu =vcpu_to_pmu (vcpu);struct kvm_pmc *pmc; -/* MSR_K7_PERFCTRn */ - pmc =get_gp_pmc (pmu, msr,MSR_K7_PERFCTR0 ); +/* MSR_PERFCTRn */ + pmc =get_gp_pmc_amd (pmu, msr,PMU_TYPE_COUNTER );if (pmc) { *data =pmc_read_counter (pmc);return 0 ; } -/* MSR_K7_EVNTSELn */ - pmc =get_gp_pmc (pmu, msr,MSR_K7_EVNTSEL0 ); +/* MSR_EVNTSELn */ + pmc =get_gp_pmc_amd (pmu, msr,PMU_TYPE_EVNTSEL );if (pmc) { *data = pmc->eventsel;return 0 ;
@@ -130,14 +238,14 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)u32 msr = msr_info->index;u64 data = msr_info->data; -/* MSR_K7_PERFCTRn */ - pmc =get_gp_pmc (pmu, msr,MSR_K7_PERFCTR0 ); +/* MSR_PERFCTRn */ + pmc =get_gp_pmc_amd (pmu, msr,PMU_TYPE_COUNTER );if (pmc) { pmc->counter += data -pmc_read_counter (pmc);return 0 ; } -/* MSR_K7_EVNTSELn */ - pmc =get_gp_pmc (pmu, msr,MSR_K7_EVNTSEL0 ); +/* MSR_EVNTSELn */ + pmc =get_gp_pmc_amd (pmu, msr,PMU_TYPE_EVNTSEL );if (pmc) {if (data == pmc->eventsel)return 0 ;
@@ -154,7 +262,11 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu) {struct kvm_pmu *pmu =vcpu_to_pmu (vcpu); - pmu->nr_arch_gp_counters =AMD64_NUM_COUNTERS ; +if (guest_cpuid_has (vcpu,X86_FEATURE_PERFCTR_CORE )) + pmu->nr_arch_gp_counters =AMD64_NUM_COUNTERS_CORE ; +else + pmu->nr_arch_gp_counters =AMD64_NUM_COUNTERS ; + pmu->counter_bitmask[KVM_PMC_GP ] = ((u64 )1 <<48 ) -1 ; pmu->reserved_bits =0xffffffff00200000ull ;/* not applicable to AMD; but clean them to prevent any fall out */
@@ -169,7 +281,7 @@ static void amd_pmu_init(struct kvm_vcpu *vcpu)struct kvm_pmu *pmu =vcpu_to_pmu (vcpu);int i; -for (i =0 ; i <AMD64_NUM_COUNTERS ; i++) { +for (i =0 ; i <AMD64_NUM_COUNTERS_CORE ; i++) { pmu->gp_counters[i].type =KVM_PMC_GP ; pmu->gp_counters[i].vcpu = vcpu; pmu->gp_counters[i].idx = i;
@@ -181,7 +293,7 @@ static void amd_pmu_reset(struct kvm_vcpu *vcpu)struct kvm_pmu *pmu =vcpu_to_pmu (vcpu);int i; -for (i =0 ; i <AMD64_NUM_COUNTERS ; i++) { +for (i =0 ; i <AMD64_NUM_COUNTERS_CORE ; i++) {struct kvm_pmc *pmc = &pmu->gp_counters[i];pmc_stop_counter (pmc);
arch/x86/kvm/x86.c +1 -0
@@ -2451,6 +2451,7 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)case MSR_AMD64_DC_CFG : msr_info->data =0 ;break ; + case MSR_F15H_PERF_CTL0 ... MSR_F15H_PERF_CTR5: case MSR_K7_EVNTSEL0 ... MSR_K7_EVNTSEL3: case MSR_K7_PERFCTR0 ... MSR_K7_PERFCTR3: case MSR_P6_PERFCTR0 ... MSR_P6_PERFCTR1:
Signature · 1 line
2.7.4
PATCH v5 3/3 x86/kvm: Expose AMD Core Perf Extension flag to guests
In reply to Janakarajan NatarajanExpose the AMD Core Perf Extension flag to the guests.
--- arch/x86/kvm/cpuid.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/cpuid.c +1 -1
@@ -365,7 +365,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,F (LAHF_LM ) |F (CMP_LEGACY ) |0 /*SVM*/ |0 /* ExtApicSpace */ |F (CR8_LEGACY ) |F (ABM ) |F (SSE4A ) |F (MISALIGNSSE ) | F(3DNOWPREFETCH) | F(OSVW) | 0 /* IBS */ | F(XOP) | -0 /* SKINIT, WDT, LWP */ |F (FMA4 ) |F (TBM ); +0 /* SKINIT, WDT, LWP */ |F (FMA4 ) |F (TBM ) |F (PERFCTR_CORE );/* cpuid 0xC0000001.edx */ const u32 kvm_cpuid_C000_0001_edx_x86_features =
Signature · 1 line
2.7.4
Quoted 25 lines · Janakarajan Natarajan
This patchset adds support for Perf Extension on AMD KVM guests.
When perf runs on a guest with family = 15h || 17h, the MSRs that are accessed, when the Perf Extension flag is made available, differ from the existing K7 MSRs. The accesses are to the AMD Core Performance Extension counters which provide 2 extra counters and new MSRs for both the event select and counter registers.
Since the new event select and counter MSRs are interleaved and K7 MSRs are contiguous, the logic to map them to the gp_counters[] is changed.
This patchset has been tested with Family 17h and Opteron G1 guests.
v1->v2:
* Rearranged MSR #defines based on Boris's suggestion.
v2->v3:
* Changed the logic of mapping MSR to gp_counters[] index based on Boris's feedback.
* Removed use of family checks based on Radim's feedback.
* Removed KVM bugfix patch since it is already applied.
v3->v4:
* Rebased to latest KVM tree.
Quoted 8 lines
v4->v5:
* Removed conditional check when exposing Perf Extension flag to guests based on Radim's feedback.Janakarajan Natarajan (3):
x86/msr: Add AMD Core Perf Extension MSRs x86/kvm: Add support for AMD Core Perf Extension in guest x86/kvm: Expose AMD Core Perf Extension flag to guests
Are there any concerns regarding this patchset?
Thanks.
Quoted 7 lines
arch/x86/include/asm/msr-index.h | 14 ++++ arch/x86/kvm/cpuid.c | 2 +- arch/x86/kvm/pmu_amd.c | 140 +++++++++++++++++++++++++++++++++++---- arch/x86/kvm/x86.c | 1 + 4 files changed, 142 insertions(+), 15 deletions(-)
2018-02-16 13:26-0600, Natarajan, Janakarajan:
Quoted 30 lines · Janakarajan Natarajan
Quoted 29 lines
This patchset adds support for Perf Extension on AMD KVM guests.
When perf runs on a guest with family = 15h || 17h, the MSRs that are accessed, when the Perf Extension flag is made available, differ from the existing K7 MSRs. The accesses are to the AMD Core Performance Extension counters which provide 2 extra counters and new MSRs for both the event select and counter registers.
Since the new event select and counter MSRs are interleaved and K7 MSRs are contiguous, the logic to map them to the gp_counters[] is changed.
This patchset has been tested with Family 17h and Opteron G1 guests.
v1->v2:
* Rearranged MSR #defines based on Boris's suggestion.
v2->v3:
* Changed the logic of mapping MSR to gp_counters[] index based on Boris's feedback.
* Removed use of family checks based on Radim's feedback.
* Removed KVM bugfix patch since it is already applied.
v3->v4:
* Rebased to latest KVM tree.
v4->v5:
* Removed conditional check when exposing Perf Extension flag to guests based on Radim's feedback.
Quoted 6 lines
Quoted 4 lines
Janakarajan Natarajan (3):
x86/msr: Add AMD Core Perf Extension MSRs x86/kvm: Add support for AMD Core Perf Extension in guest x86/kvm: Expose AMD Core Perf Extension flag to guestsAre there any concerns regarding this patchset?
No, looks ok!
I have squashed the last patch, added a sanity BUILD_BUG_ON and applied to kvm/queue (until we're sure that [1/3] goes through the kvm tree), thanks.