x86/pmu: Test case optimization, fixes and additions
Sean Christopherson
reviewerv5 cover x86/pmu: Test case optimization, fixes and additions
This series is a big pile of PMU cleanups and enhancements from Like.
The changes are roughly divided into three parts: (1) fixes (2) cleanups, and (3) new test cases. The changes are bundled in a mega-series as the original, separate series was difficult to review/manage due to a number of dependencies.
There are no major changes in the test logic. The big cleanups are to add lib/x86/pmu.[c,h] and a global PMU capabilities struct to improve readability of the code and to hide some AMD vs. Intel details.
Like's v4 was tested on AMD Zen3/4 and Intel ICX/SPR machines, but this version has only been tested on AMD Zen3 (Milan) and Intel ICX and HSW, i.e. I haven't tested AMD PMU v2 or anything new in SPR (if there is anything in SPR?).
Like Xu (22):
x86/pmu: Add PDCM check before accessing PERF_CAP register x86/pmu: Test emulation instructions on full-width counters x86/pmu: Pop up FW prefix to avoid out-of-context propagation x86/pmu: Report SKIP when testing Intel LBR on AMD platforms x86/pmu: Fix printed messages for emulated instruction test x86/pmu: Introduce __start_event() to drop all of the manual zeroing x86/pmu: Introduce multiple_{one, many}() to improve readability x86/pmu: Reset the expected count of the fixed counter 0 when i386
x86: create pmu group for quick pmu-scope testing x86/pmu: Refine info to clarify the current support x86/pmu: Update rdpmc testcase to cover #GP path
x86/pmu: Rename PC_VECTOR to PMI_VECTOR for better readability x86/pmu: Add lib/x86/pmu.[c.h] and move common code to header files x86/pmu: Snapshot PMU perf_capabilities during BSP initialization x86/pmu: Track GP counter and event select base MSRs in pmu_caps x86/pmu: Add helper to get fixed counter MSR index x86/pmu: Track global status/control/clear MSRs in pmu_caps
x86: Add tests for Guest Processor Event Based Sampling (PEBS) x86/pmu: Add global helpers to cover Intel Arch PMU Version 1 x86/pmu: Add gp_events pointer to route different event tables x86/pmu: Update testcases to cover AMD PMU
x86/pmu: Add AMD Guest PerfMonV2 testcases
Sean Christopherson (5):
x86: Add a helper for the BSP's final init sequence common to all
flavors
x86/pmu: Snapshot CPUID.0xA PMU capabilities during BSP initialization x86/pmu: Drop wrappers that just passthrough pmu_caps fields x86/pmu: Reset GP and Fixed counters during pmu_init(). x86/pmu: Add pmu_caps flag to track if CPU is Intel (versus AMD)
lib/x86/asm/setup.h | 1 + lib/x86/msr.h | 30 +++ lib/x86/pmu.c | 67 +++++++ lib/x86/pmu.h | 187 +++++++++++++++++++ lib/x86/processor.h | 80 ++------ lib/x86/setup.c | 13 +- x86/Makefile.common | 1 + x86/Makefile.x86_64 | 1 + x86/cstart.S | 4 +- x86/cstart64.S | 4 +- x86/pmu.c | 360 ++++++++++++++++++++---------------- x86/pmu_lbr.c | 24 +-- x86/pmu_pebs.c | 433 ++++++++++++++++++++++++++++++++++++++++++++ x86/unittests.cfg | 10 + x86/vmx_tests.c | 1 + 15 files changed, 975 insertions(+), 241 deletions(-) create mode 100644 lib/x86/pmu.c create mode 100644 lib/x86/pmu.h create mode 100644 x86/pmu_pebs.c
base-commit: 73d9d850f1c2c9f0df321967e67acda0d2c305ea
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 1/27 x86/pmu: Add PDCM check before accessing PERF_CAP register
From: Like Xu <[email protected]>
On virtual platforms without PDCM support (e.g. AMD), #GP failure on MSR_IA32_PERF_CAPABILITIES is completely avoidable.
--- lib/x86/processor.h | 8 ++++++++ x86/pmu.c | 2 +- x86/pmu_lbr.c | 2 +- 3 files changed, 10 insertions(+), 2 deletions(-)
lib/x86/processor.h +8 -0
@@ -847,4 +847,12 @@ static inline bool pmu_gp_counter_is_available(int i) return !(cpuid(10).b & BIT(i)); } +static inline u64 this_cpu_perf_capabilities (void ) +{ +if (!this_cpu_has (X86_FEATURE_PDCM )) +return 0 ; + +return rdmsr (MSR_IA32_PERF_CAPABILITIES ); +} +#endif
x86/pmu.c +1 -1
@@ -660,7 +660,7 @@ int main(int ac, char **av)check_counters (); -if (rdmsr (MSR_IA32_PERF_CAPABILITIES ) &PMU_CAP_FW_WRITES ) { +if (this_cpu_perf_capabilities () &PMU_CAP_FW_WRITES ) { gp_counter_base =MSR_IA32_PMC0 ;report_prefix_push ("full-width writes" );check_counters ();
x86/pmu_lbr.c +1 -1
@@ -72,7 +72,7 @@ int main(int ac, char **av)return report_summary (); } - perf_cap =rdmsr (MSR_IA32_PERF_CAPABILITIES ); + perf_cap =this_cpu_perf_capabilities ();if (!(perf_cap &PMU_CAP_LBR_FMT )) {report_skip ("(Architectural) LBR is not supported." );
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 2/27 x86/pmu: Test emulation instructions on full-width counters
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Move check_emulated_instr() into check_counters() so that full-width counters could be tested with ease by the same test case.
--- x86/pmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
x86/pmu.c +3 -3
@@ -520,6 +520,9 @@ static void check_emulated_instr(void)static void check_counters (void ) { +if (is_fep_available ()) +check_emulated_instr (); +check_gp_counters ();check_fixed_counters ();check_rdpmc ();
@@ -655,9 +658,6 @@ int main(int ac, char **av)apic_write (APIC_LVTPC ,PC_VECTOR ); -if (is_fep_available ()) -check_emulated_instr (); -check_counters ();if (this_cpu_perf_capabilities () &PMU_CAP_FW_WRITES ) {
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 3/27 x86/pmu: Pop up FW prefix to avoid out-of-context propagation
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
The inappropriate prefix "full-width writes" may be propagated to later test cases if it is not popped out.
--- x86/pmu.c | 1 + 1 file changed, 1 insertion(+)
x86/pmu.c +1 -0
@@ -665,6 +665,7 @@ int main(int ac, char **av)report_prefix_push ("full-width writes" );check_counters ();check_gp_counters_write_width (); +report_prefix_pop (); }return report_summary ();
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 4/27 x86/pmu: Report SKIP when testing Intel LBR on AMD platforms
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
The test conclusion of running Intel LBR on AMD platforms should not be PASS, but SKIP, fix it.
--- x86/pmu_lbr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
x86/pmu_lbr.c +1 -1
@@ -59,7 +59,7 @@ int main(int ac, char **av)if (!is_intel ()) {report_skip ("PMU_LBR test is for intel CPU's only" ); -return 0 ; +return report_summary (); }if (!this_cpu_has_pmu ()) {
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 5/27 x86/pmu: Fix printed messages for emulated instruction test
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
This test case uses MSR_IA32_PERFCTR0 to count branch instructions and PERFCTR1 to count instruction events. The same correspondence should be maintained at report(), specifically this should use status bit 1 for instructions and bit 0 for branches.
--- x86/pmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
x86/pmu.c +2 -2
@@ -512,8 +512,8 @@ static void check_emulated_instr(void) "branch count");// Additionally check that those counters overflowed properly. status =rdmsr (MSR_CORE_PERF_GLOBAL_STATUS ); -report (status &1 ,"instruction counter overflow" ); -report (status &2 ,"branch counter overflow" ); +report (status &1 ,"branch counter overflow" ); +report (status &2 ,"instruction counter overflow" );report_prefix_pop (); }
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 6/27 x86/pmu: Introduce __start_event() to drop all of the manual zeroing
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Most invocation of start_event() and measure() first sets evt.count=0. Instead of forcing each caller to ensure count is zeroed, optimize the count to zero during start_event(), then drop all of the manual zeroing.
Accumulating counts can be handled by reading the current count before start_event(), and doing something like stuffing a high count to test an edge case could be handled by an inner helper, __start_event().
For overflow, just open code measure() for that one-off case. Requiring callers to zero out a field in most common cases isn't exactly flexible.
--- x86/pmu.c | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-)
x86/pmu.c +16 -15
@@ -137,9 +137,9 @@ static void global_disable(pmu_counter_t *cnt) ~(1ull << cnt->idx)); } - -static void start_event (pmu_counter_t *evt) +static void __start_event (pmu_counter_t *evt,uint64_t count) { + evt->count = count;wrmsr (evt->ctr, evt->count);if (is_gp (evt)) wrmsr(MSR_P6_EVNTSEL0 + event_to_global_idx(evt),
@@ -162,6 +162,11 @@ static void start_event(pmu_counter_t *evt)apic_write (APIC_LVTPC ,PC_VECTOR ); } +static void start_event (pmu_counter_t *evt) +{ +__start_event (evt,0 ); +} +static void stop_event (pmu_counter_t *evt) {global_disable (evt);
@@ -186,6 +191,13 @@ static noinline void measure(pmu_counter_t *evt, int count)stop_event (&evt[i]); } +static noinline void __measure(pmu_counter_t *evt, uint64_t count) +{ +__start_event (evt, count); +loop (); +stop_event (evt); +} +static bool verify_event (uint64_t count,struct pmu_event *e) {// printf("%d <= %ld <= %d\n", e->min, count, e->max);
@@ -208,7 +220,6 @@ static void check_gp_counter(struct pmu_event *evt)int i;for (i =0 ; i < nr_gp_counters; i++, cnt.ctr++) { - cnt.count =0 ;measure (&cnt,1 );report (verify_event (cnt.count, evt),"%s-%d" , evt->name, i); }
@@ -235,7 +246,6 @@ static void check_fixed_counters(void)int i;for (i =0 ; i < nr_fixed_counters; i++) { - cnt.count =0 ; cnt.ctr = fixed_events[i].unit_sel;measure (&cnt,1 );report (verify_event (cnt.count, &fixed_events[i]),"fixed-%d" , i);
@@ -253,14 +263,12 @@ static void check_counters_many(void)if (!pmu_gp_counter_is_available (i))continue ; - cnt[n].count =0 ; cnt[n].ctr = gp_counter_base + n; cnt[n].config =EVNTSEL_OS |EVNTSEL_USR | gp_events[i %ARRAY_SIZE (gp_events)].unit_sel; n++; }for (i =0 ; i < nr_fixed_counters; i++) { - cnt[n].count =0 ; cnt[n].ctr = fixed_events[i].unit_sel; cnt[n].config =EVNTSEL_OS |EVNTSEL_USR ; n++;
@@ -283,9 +291,8 @@ static void check_counter_overflow(void)pmu_counter_t cnt = { .ctr = gp_counter_base, .config =EVNTSEL_OS |EVNTSEL_USR | gp_events[1 ].unit_sel/* instructions */ , - .count =0 , }; -measure (&cnt,1 ); +__measure (&cnt,0 ); count = cnt.count;/* clear status before test */
@@ -311,7 +318,7 @@ static void check_counter_overflow(void)else cnt.config &= ~EVNTSEL_INT ; idx =event_to_global_idx (&cnt); -measure (&cnt,1 ); +__measure (&cnt, cnt.count);report (cnt.count ==1 ,"cntr-%d" , i); status =rdmsr (MSR_CORE_PERF_GLOBAL_STATUS );report (status & (1ull << idx),"status-%d" , i);
@@ -329,7 +336,6 @@ static void check_gp_counter_cmask(void)pmu_counter_t cnt = { .ctr = gp_counter_base, .config =EVNTSEL_OS |EVNTSEL_USR | gp_events[1 ].unit_sel/* instructions */ , - .count =0 , }; cnt.config |= (0x2 <<EVNTSEL_CMASK_SHIFT );measure (&cnt,1 );
@@ -415,7 +421,6 @@ static void check_running_counter_wrmsr(void)pmu_counter_t evt = { .ctr = gp_counter_base, .config =EVNTSEL_OS |EVNTSEL_USR | gp_events[1 ].unit_sel, - .count =0 , };report_prefix_push ("running counter wrmsr" );
@@ -430,7 +435,6 @@ static void check_running_counter_wrmsr(void)wrmsr (MSR_CORE_PERF_GLOBAL_OVF_CTRL ,rdmsr (MSR_CORE_PERF_GLOBAL_STATUS )); - evt.count =0 ;start_event (&evt); count =-1 ;
@@ -454,13 +458,11 @@ static void check_emulated_instr(void) .ctr = MSR_IA32_PERFCTR0, /* branch instructions */ .config = EVNTSEL_OS | EVNTSEL_USR | gp_events[5].unit_sel, - .count = 0, };pmu_counter_t instr_cnt = { .ctr =MSR_IA32_PERFCTR0 +1 ,/* instructions */ .config =EVNTSEL_OS |EVNTSEL_USR | gp_events[1 ].unit_sel, - .count =0 , };report_prefix_push ("emulated instruction" );
@@ -592,7 +594,6 @@ static void set_ref_cycle_expectations(void)pmu_counter_t cnt = { .ctr =MSR_IA32_PERFCTR0 , .config =EVNTSEL_OS |EVNTSEL_USR | gp_events[2 ].unit_sel, - .count =0 , };uint64_t tsc_delta;uint64_t t0, t1, t2, t3;
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 7/27 x86/pmu: Introduce multiple_{one, many}() to improve readability
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
The current measure_one() forces the common case to pass in unnecessary information in order to give flexibility to a single use case. It's just syntatic sugar, but it really does help readers as it's not obvious that the "1" specifies the number of events, whereas multiple_many() and measure_one() are relatively self-explanatory.
--- x86/pmu.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
x86/pmu.c +10 -5
@@ -181,7 +181,7 @@ static void stop_event(pmu_counter_t *evt) evt->count =rdmsr (evt->ctr); } -static noinline void measure(pmu_counter_t *evt, int count) +static noinline void measure_many(pmu_counter_t *evt, int count) {int i;for (i =0 ; i < count; i++)
@@ -191,6 +191,11 @@ static noinline void measure(pmu_counter_t *evt, int count)stop_event (&evt[i]); } +static void measure_one (pmu_counter_t *evt) +{ +measure_many (evt,1 ); +} + static noinline void __measure(pmu_counter_t *evt, uint64_t count) {__start_event (evt, count);
@@ -220,7 +225,7 @@ static void check_gp_counter(struct pmu_event *evt)int i;for (i =0 ; i < nr_gp_counters; i++, cnt.ctr++) { -measure (&cnt,1 ); +measure_one (&cnt);report (verify_event (cnt.count, evt),"%s-%d" , evt->name, i); } }
@@ -247,7 +252,7 @@ static void check_fixed_counters(void)for (i =0 ; i < nr_fixed_counters; i++) { cnt.ctr = fixed_events[i].unit_sel; -measure (&cnt,1 ); +measure_one (&cnt);report (verify_event (cnt.count, &fixed_events[i]),"fixed-%d" , i); } }
@@ -274,7 +279,7 @@ static void check_counters_many(void) n++; } -measure (cnt, n); +measure_many (cnt, n);for (i =0 ; i < n; i++)if (!verify_counter (&cnt[i]))
@@ -338,7 +343,7 @@ static void check_gp_counter_cmask(void) .config = EVNTSEL_OS | EVNTSEL_USR | gp_events[1].unit_sel /* instructions */, }; cnt.config |= (0x2 <<EVNTSEL_CMASK_SHIFT ); -measure (&cnt,1 ); +measure_one (&cnt);report (cnt.count < gp_events[1 ].min,"cmask" ); }
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 8/27 x86/pmu: Reset the expected count of the fixed counter 0 when i386
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
The pmu test check_counter_overflow() always fails with 32-bit binaries. The cnt.count obtained from the latter run of measure() (based on fixed counter 0) is not equal to the expected value (based on gp counter 0) and there is a positive error with a value of 2.
The two extra instructions come from inline wrmsr() and inline rdmsr() inside the global_disable() binary code block. Specifically, for each msr access, the i386 code will have two assembly mov instructions before rdmsr/wrmsr (mark it for fixed counter 0, bit 32), but only one assembly mov is needed for x86_64 and gp counter 0 on i386.
The sequence of instructions to count events using the #GP and #Fixed counters is different. Thus the fix is quite high level, to use the same counter (w/ same instruction sequences) to set initial value for the same counter. Fix the expected init cnt.count for fixed counter 0 overflow based on the same fixed counter 0, not always using gp counter 0.
The difference of 1 for this count enables the interrupt to be generated immediately after the selected event count has been reached, instead of waiting for the overflow to be propagation through the counter.
Adding a helper to measure/compute the overflow preset value. It provides a convenient location to document the weird behavior that's necessary to ensure immediate event delivery.
--- x86/pmu.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-)
x86/pmu.c +18 -4
@@ -288,17 +288,30 @@ static void check_counters_many(void)report (i == n,"all counters" ); } +static uint64_t measure_for_overflow (pmu_counter_t *cnt) +{ +__measure (cnt,0 ); +/* +* To generate overflow, i.e. roll over to '0', the initial count just +* needs to be preset to the negative expected count. However, as per +* Intel's SDM, the preset count needs to be incremented by 1 to ensure +* the overflow interrupt is generated immediately instead of possibly +* waiting for the overflow to propagate through the counter. +*/ +assert (cnt->count >1 ); +return 1 - cnt->count; +} +static void check_counter_overflow (void ) {int nr_gp_counters =pmu_nr_gp_counters (); -uint64_t count; +uint64_t overflow_preset;int i;pmu_counter_t cnt = { .ctr = gp_counter_base, .config =EVNTSEL_OS |EVNTSEL_USR | gp_events[1 ].unit_sel/* instructions */ , }; -__measure (&cnt,0 ); - count = cnt.count; + overflow_preset =measure_for_overflow (&cnt);/* clear status before test */ wrmsr (MSR_CORE_PERF_GLOBAL_OVF_CTRL ,rdmsr (MSR_CORE_PERF_GLOBAL_STATUS ));
@@ -309,12 +322,13 @@ static void check_counter_overflow(void)uint64_t status;int idx; - cnt.count =1 - count; + cnt.count = overflow_preset;if (gp_counter_base ==MSR_IA32_PMC0 ) cnt.count &= (1ull <<pmu_gp_counter_width ()) -1 ;if (i == nr_gp_counters) { cnt.ctr = fixed_events[0 ].unit_sel; + cnt.count =measure_for_overflow (&cnt); cnt.count &= (1ull <<pmu_fixed_counter_width ()) -1 ; }
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 9/27 x86: create pmu group for quick pmu-scope testing
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Any agent can run "./run_tests.sh -g pmu" to run all PMU tests easily, e.g. when verifying the x86/PMU KVM changes.
--- x86/unittests.cfg | 2 ++ 1 file changed, 2 insertions(+)
x86/unittests.cfg +2 -0
@@ -189,6 +189,7 @@ file = pmu.flat extra_params = -cpu max check = /proc/sys/kernel/nmi_watchdog=0 accel = kvm +groups = pmu [pmu_lbr] arch = x86_64
@@ -197,6 +198,7 @@ extra_params = -cpu host,migratable=no check = /sys/module/kvm/parameters/ignore_msrs=N check = /proc/sys/kernel/nmi_watchdog=0 accel = kvm +groups = pmu [vmware_backdoors] file = vmware_backdoors.flat
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 10/27 x86/pmu: Refine info to clarify the current support
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Existing unit tests do not cover AMD pmu, nor Intel pmu that is not architecture (on some obsolete cpu's). AMD's PMU support will be coming in subsequent commits.
--- x86/pmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
x86/pmu.c +1 -1
@@ -658,7 +658,7 @@ int main(int ac, char **av) buf =malloc (N *64 );if (!pmu_version ()) { -report_skip ("No pmu is detected!" ); +report_skip ("No Intel Arch PMU is detected!" );return report_summary (); }
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 11/27 x86/pmu: Update rdpmc testcase to cover #GP path
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Specifying an unsupported PMC encoding will cause a #GP(0).
There are multiple reasons RDPMC can #GP, the one that is being relied on to guarantee #GP is specifically that the PMC is invalid. The most extensible solution is to provide a safe variant.
--- lib/x86/processor.h | 21 ++++++++++++++++++--- x86/pmu.c | 10 ++++++++++ 2 files changed, 28 insertions(+), 3 deletions(-)
lib/x86/processor.h +18 -3
@@ -438,11 +438,26 @@ static inline int wrmsr_safe(u32 index, u64 val)return exception_vector (); } +static inline int rdpmc_safe (u32 index,uint64_t *val) +{ +uint32_t a, d; + + asm volatile (ASM_TRY("1f") + "rdpmc\n\t" + "1:" + : "=a"(a), "=d"(d) : "c"(index) : "memory"); + *val = (uint64_t )a | ((uint64_t )d <<32 ); +return exception_vector (); +} +static inline uint64_t rdpmc (uint32_t index) { -uint32_t a, d; - asmvolatile ("rdpmc" :"=a" (a),"=d" (d) :"c" (index)); -return a | ((uint64_t )d <<32 ); +uint64_t val; +int vector =rdpmc_safe (index, &val); + +assert_msg (!vector,"Unexpected %s on RDPMC(%d)" , +exception_mnemonic (vector), index); +return val; }static inline int write_cr0_safe (ulong val)
x86/pmu.c +10 -0
@@ -651,12 +651,22 @@ static void set_ref_cycle_expectations(void) gp_events[2 ].max = (gp_events[2 ].max * cnt.count) / tsc_delta; } +static void check_invalid_rdpmc_gp (void ) +{ +uint64_t val; + +report (rdpmc_safe (64 , &val) ==GP_VECTOR , +"Expected #GP on RDPMC(64)" ); +} +int main (int ac,char **av) {setup_vm ();handle_irq (PC_VECTOR , cnt_overflow); buf =malloc (N *64 ); +check_invalid_rdpmc_gp (); +if (!pmu_version ()) {report_skip ("No Intel Arch PMU is detected!" );return report_summary ();
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Re: [kvm-unit-tests PATCH v5 11/27] x86/pmu: Update rdpmc testcase to cover #GP path
Quoted 38 lines · Sean Christopherson
From: Like Xu <[email protected]>
Specifying an unsupported PMC encoding will cause a #GP(0).
There are multiple reasons RDPMC can #GP, the one that is being relied on to guarantee #GP is specifically that the PMC is invalid. The most extensible solution is to provide a safe variant.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Like Xu <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- lib/x86/processor.h | 21 ++++++++++++++++++--- x86/pmu.c | 10 ++++++++++ 2 files changed, 28 insertions(+), 3 deletions(-)lib/x86/processor.h +12
-0@@ -438,11 +438,26 @@ static inline int wrmsr_safe(u32 index, u64 val)return exception_vector (); } +static inline int rdpmc_safe (u32 index,uint64_t *val) +{ +uint32_t a, d; + + asm volatile (ASM_TRY("1f") + "rdpmc\n\t" + "1:" + : "=a"(a), "=d"(d) : "c"(index) : "memory"); + *val = (uint64_t )a | ((uint64_t )d <<32 ); +return exception_vector (); +} +static inline uint64_t rdpmc (uint32_t index) {
lib/x86/processor.h static inline int wrmsr_safe(u32 index, u64 val) · line 461
Quoted 10 lines
lib/x86/processor.h +6
-3-uint32_t a, d; - asmvolatile ("rdpmc" :"=a" (a),"=d" (d) :"c" (index)); -return a | ((uint64_t )d <<32 ); +uint64_t val; +int vector =rdpmc_safe (index, &val); + +assert_msg (!vector,"Unexpected %s on RDPMC(%d)" , +exception_mnemonic (vector), index); +return val; }
Seems like this is causing the CI to fail:
https://gitlab.com/kvm-unit-tests/kvm-unit-tests/-/jobs/3339274319#L1260
I guess you have to use PRId32 here? Could you please send a patch?
Thanks,
Thomas
Re: [kvm-unit-tests PATCH v5 11/27] x86/pmu: Update rdpmc testcase to cover #GP path
Quoted 51 lines · Thomas Huth
Quoted 37 lines · Sean Christopherson
From: Like Xu <[email protected]>
Specifying an unsupported PMC encoding will cause a #GP(0).
There are multiple reasons RDPMC can #GP, the one that is being relied on to guarantee #GP is specifically that the PMC is invalid. The most extensible solution is to provide a safe variant.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Like Xu <[email protected]>Signed-off-by Sean Christopherson <[email protected]>---
lib/x86/processor.h| 21 ++++++++++++++++++---
x86/pmu.c | 10 ++++++++++2 files changed, 28 insertions(+), 3 deletions(-)lib/x86/processor.h +12
-0@@ -438,11 +438,26 @@ static inline int wrmsr_safe(u32 index, u64 val) return exception_vector(); } +static inline int rdpmc_safe (u32 index,uint64_t *val) +{ + uint32_t a, d; + + asm volatile (ASM_TRY("1f") + "rdpmc\n\t" + "1:" + : "=a"(a), "=d"(d) : "c"(index) : "memory"); + *val = (uint64_t)a | ((uint64_t)d << 32); + return exception_vector(); +} + static inline uint64_t rdpmc(uint32_t index) {lib/x86/processor.h static inline int wrmsr_safe(u32 index, u64 val) · line 460
Quoted 10 lines
lib/x86/processor.h +6
-3- uint32_t a, d; - asm volatile ("rdpmc" : "=a"(a), "=d"(d) : "c"(index)); - return a | ((uint64_t)d << 32); + uint64_t val; + int vector = rdpmc_safe(index, &val); + + assert_msg(!vector, "Unexpected %s on RDPMC(%d)", + exception_mnemonic(vector), index); + return val; }Seems like this is causing the CI to fail:
I just now noticed that KUT can be used to validate TCG or HVF accel on macOS. I assume the functionality of the PMU counter on TCG seems to be a blank slate.
Quoted 2 lines
I guess you have to use PRId32 here? Could you please send a patch?
Sure, let me try it on the macOS.
Quoted 6 lines
Thanks,
Thomas
Sean Christopherson
reviewerPATCH v5 12/27 x86/pmu: Rename PC_VECTOR to PMI_VECTOR for better readability
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
The original name "PC_VECTOR" comes from the LVT Performance Counter Register. Rename it to PMI_VECTOR. That's much more familiar for KVM developers and it's still correct, e.g. it's the PMI vector that's programmed into the LVT PC register.
--- x86/pmu.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
x86/pmu.c +6 -4
@@ -11,7 +11,9 @@#include <stdint.h> #define FIXED_CNT_INDEX 32 -#define PC_VECTOR 32 + +/* Performance Counter Vector for the LVT PC Register */ +#define PMI_VECTOR 32#define EVNSEL_EVENT_SHIFT 0#define EVNTSEL_UMASK_SHIFT 8
@@ -159,7 +161,7 @@ static void __start_event(pmu_counter_t *evt, uint64_t count)wrmsr (MSR_CORE_PERF_FIXED_CTR_CTRL , ctrl); }global_enable (evt); -apic_write (APIC_LVTPC ,PC_VECTOR ); +apic_write (APIC_LVTPC ,PMI_VECTOR ); }static void start_event (pmu_counter_t *evt)
@@ -662,7 +664,7 @@ static void check_invalid_rdpmc_gp(void)int main (int ac,char **av) {setup_vm (); -handle_irq (PC_VECTOR , cnt_overflow); +handle_irq (PMI_VECTOR , cnt_overflow); buf =malloc (N *64 );check_invalid_rdpmc_gp ();
@@ -686,7 +688,7 @@ int main(int ac, char **av)printf ("Fixed counters: %d\n" ,pmu_nr_fixed_counters ());printf ("Fixed counter width: %d\n" ,pmu_fixed_counter_width ()); -apic_write (APIC_LVTPC ,PC_VECTOR ); +apic_write (APIC_LVTPC ,PMI_VECTOR );check_counters ();
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 13/27 x86/pmu: Add lib/x86/pmu.[c.h] and move common code to header files
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Given all the PMU stuff coming in, we need e.g. lib/x86/pmu.h to hold all of the hardware-defined stuff, e.g. #defines, accessors, helpers and structs that are dictated by hardware. This will greatly help with code reuse and reduce unnecessary vm-exit.
Opportunistically move lbr msrs definition to header processor.h.
--- lib/x86/msr.h | 7 ++++ lib/x86/pmu.c | 1 + lib/x86/pmu.h | 100 ++++++++++++++++++++++++++++++++++++++++++++ lib/x86/processor.h | 64 ---------------------------- x86/Makefile.common | 1 + x86/pmu.c | 25 +---------- x86/pmu_lbr.c | 11 +---- x86/vmx_tests.c | 1 + 8 files changed, 112 insertions(+), 98 deletions(-) create mode 100644 lib/x86/pmu.c create mode 100644 lib/x86/pmu.h
lib/x86/msr.h +7 -0
@@ -86,6 +86,13 @@#define DEBUGCTLMSR_BTS_OFF_USR (1UL << 10)#define DEBUGCTLMSR_FREEZE_LBRS_ON_PMI (1UL << 11) +#define MSR_LBR_NHM_FROM 0x00000680 +#define MSR_LBR_NHM_TO 0x000006c0 +#define MSR_LBR_CORE_FROM 0x00000040 +#define MSR_LBR_CORE_TO 0x00000060 +#define MSR_LBR_TOS 0x000001c9 +#define MSR_LBR_SELECT 0x000001c8 +#define MSR_IA32_MC0_CTL 0x00000400#define MSR_IA32_MC0_STATUS 0x00000401#define MSR_IA32_MC0_ADDR 0x00000402
lib/x86/pmu.c new file +1 -0
@@ -0,0 +1 @@ +#include "pmu.h"
lib/x86/pmu.h new file +100 -0
100 lines not on this page · raw message
lib/x86/processor.h +0 -64
68 lines not on this page · raw message
x86/Makefile.common +1 -0
@@ -22,6 +22,7 @@ cflatobjs += lib/x86/acpi.o cflatobjs += lib/x86/stack.o cflatobjs += lib/x86/fault_test.o cflatobjs += lib/x86/delay.o +cflatobjs += lib/x86/pmu.o ifeq ($(CONFIG_EFI),y) cflatobjs += lib/x86/amd_sev.o cflatobjs += lib/efi.o
x86/pmu.c +1 -24
@@ -1,6 +1,7 @@#include "x86/msr.h" #include "x86/processor.h" +#include "x86/pmu.h" #include "x86/apic-defs.h" #include "x86/apic.h" #include "x86/desc.h"
@@ -10,29 +11,6 @@#include "libcflat.h" #include <stdint.h> -#define FIXED_CNT_INDEX 32 - -/* Performance Counter Vector for the LVT PC Register */ -#define PMI_VECTOR 32 - -#define EVNSEL_EVENT_SHIFT 0 -#define EVNTSEL_UMASK_SHIFT 8 -#define EVNTSEL_USR_SHIFT 16 -#define EVNTSEL_OS_SHIFT 17 -#define EVNTSEL_EDGE_SHIFT 18 -#define EVNTSEL_PC_SHIFT 19 -#define EVNTSEL_INT_SHIFT 20 -#define EVNTSEL_EN_SHIF 22 -#define EVNTSEL_INV_SHIF 23 -#define EVNTSEL_CMASK_SHIFT 24 - -#define EVNTSEL_EN (1 << EVNTSEL_EN_SHIF) -#define EVNTSEL_USR (1 << EVNTSEL_USR_SHIFT) -#define EVNTSEL_OS (1 << EVNTSEL_OS_SHIFT) -#define EVNTSEL_PC (1 << EVNTSEL_PC_SHIFT) -#define EVNTSEL_INT (1 << EVNTSEL_INT_SHIFT) -#define EVNTSEL_INV (1 << EVNTSEL_INV_SHIF) -#define N 1000000// These values match the number of instructions and branches in the
@@ -66,7 +44,6 @@ struct pmu_event { {"fixed 3", MSR_CORE_PERF_FIXED_CTR0 + 2, 0.1*N, 30*N} }; -#define PMU_CAP_FW_WRITES (1ULL << 13)static u64 gp_counter_base =MSR_IA32_PERFCTR0 ;char *buf;
x86/pmu_lbr.c +1 -10
@@ -1,18 +1,9 @@#include "x86/msr.h" #include "x86/processor.h" +#include "x86/pmu.h" #include "x86/desc.h" #define N 1000000 -#define MAX_NUM_LBR_ENTRY 32 -#define DEBUGCTLMSR_LBR (1UL << 0) -#define PMU_CAP_LBR_FMT 0x3f - -#define MSR_LBR_NHM_FROM 0x00000680 -#define MSR_LBR_NHM_TO 0x000006c0 -#define MSR_LBR_CORE_FROM 0x00000040 -#define MSR_LBR_CORE_TO 0x00000060 -#define MSR_LBR_TOS 0x000001c9 -#define MSR_LBR_SELECT 0x000001c8volatile int count;u32 lbr_from, lbr_to;
x86/vmx_tests.c +1 -0
@@ -9,6 +9,7 @@#include "vmx.h" #include "msr.h" #include "processor.h" +#include "pmu.h" #include "vm.h" #include "pci.h" #include "fwcfg.h"
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 14/27 x86: Add a helper for the BSP's final init sequence common to all flavors
In reply to Sean ChristophersonAdd bsp_rest_init() to dedup bringing up APs and doing SMP initialization across 32-bit, 64-bit, and EFI flavors of KVM-unit-tests. The common bucket will also be used in future to patches to init things that aren't SMP related and thus don't fit in smp_init(), e.g. PMU setup.
No functional change intended.
--- lib/x86/asm/setup.h | 1 + lib/x86/setup.c | 11 ++++++++--- x86/cstart.S | 4 +--- x86/cstart64.S | 4 +--- 4 files changed, 11 insertions(+), 9 deletions(-)
lib/x86/asm/setup.h +1 -0
@@ -17,6 +17,7 @@ void setup_5level_page_table(void);#endif /* CONFIG_EFI */ void save_id (void ); +void bsp_rest_init (void );void ap_start64 (void );#endif /* _X86_ASM_SETUP_H_ */
lib/x86/setup.c +8 -3
@@ -356,9 +356,7 @@ efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo)setup_page_table ();enable_apic ();save_id (); -bringup_aps (); -enable_x2apic (); -smp_init (); +bsp_rest_init ();return EFI_SUCCESS ; }
@@ -394,3 +392,10 @@ void ap_start64(void)enable_x2apic ();ap_online (); } + +void bsp_rest_init (void ) +{ +bringup_aps (); +enable_x2apic (); +smp_init (); +}
x86/cstart.S +1 -3
@@ -112,9 +112,7 @@ start32: call save_id call mask_pic_interrupts call enable_apic - call bringup_aps - call enable_x2apic - call smp_init + call bsp_rest_init push $__environ push $__argv push __argc
x86/cstart64.S +1 -3
@@ -118,9 +118,7 @@ start64: mov %rax, __args(%rip) call __setup_args - call bringup_aps - call enable_x2apic - call smp_init + call bsp_rest_init mov __argc(%rip), %edi lea __argv(%rip), %rsi
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 15/27 x86/pmu: Snapshot PMU perf_capabilities during BSP initialization
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Add a global "struct pmu_caps pmu" to snapshot PMU capabilities during the final stages of BSP initialization. Use the new hooks to snapshot PERF_CAPABILITIES instead of re-reading the MSR every time a test wants to query capabilities. A software-defined struct will also simplify extending support to AMD CPUs, as many of the differences between AMD and Intel can be handled during pmu_init().
Init the PMU caps for all tests so that tests don't need to remember to call pmu_init() before using any of the PMU helpers, e.g. the nVMX test uses this_cpu_has_pmu(), which will be converted to rely on the global struct in a future patch.
--- lib/x86/pmu.c | 8 ++++++++ lib/x86/pmu.h | 21 ++++++++++++++++++--- lib/x86/setup.c | 2 ++ x86/pmu.c | 2 +- x86/pmu_lbr.c | 7 ++----- 5 files changed, 31 insertions(+), 9 deletions(-)
lib/x86/pmu.c +8 -0
@@ -1 +1,9 @@#include "pmu.h" + +struct pmu_caps pmu; + +void pmu_init (void ) +{ +if (this_cpu_has (X86_FEATURE_PDCM )) + pmu.perf_cap =rdmsr (MSR_IA32_PERF_CAPABILITIES ); +}
lib/x86/pmu.h +18 -3
@@ -33,6 +33,14 @@#define EVNTSEL_INT (1 << EVNTSEL_INT_SHIFT)#define EVNTSEL_INV (1 << EVNTSEL_INV_SHIF) +struct pmu_caps { +u64 perf_cap; +}; + +extern struct pmu_caps pmu; + +void pmu_init (void ); +static inline u8 pmu_version (void ) {return cpuid (10 ).a &0xff ;
@@ -91,10 +99,17 @@ static inline bool pmu_gp_counter_is_available(int i)static inline u64 this_cpu_perf_capabilities (void ) { -if (!this_cpu_has (X86_FEATURE_PDCM )) -return 0 ; +return pmu.perf_cap; +} -return rdmsr (MSR_IA32_PERF_CAPABILITIES ); +static inline u64 pmu_lbr_version (void ) +{ +return this_cpu_perf_capabilities () &PMU_CAP_LBR_FMT ; +} + +static inline bool pmu_has_full_writes (void ) +{ +return this_cpu_perf_capabilities () &PMU_CAP_FW_WRITES ; }#endif /* _X86_PMU_H_ */
lib/x86/setup.c +2 -0
@@ -15,6 +15,7 @@#include "apic-defs.h" #include "asm/setup.h" #include "atomic.h" +#include "pmu.h" #include "processor.h" #include "smp.h"
@@ -398,4 +399,5 @@ void bsp_rest_init(void)bringup_aps ();enable_x2apic ();smp_init (); +pmu_init (); }
x86/pmu.c +1 -1
@@ -669,7 +669,7 @@ int main(int ac, char **av)check_counters (); -if (this_cpu_perf_capabilities () &PMU_CAP_FW_WRITES ) { +if (pmu_has_full_writes ()) { gp_counter_base =MSR_IA32_PMC0 ;report_prefix_push ("full-width writes" );check_counters ();
x86/pmu_lbr.c +2 -5
@@ -43,7 +43,6 @@ static bool test_init_lbr_from_exception(u64 index)int main (int ac,char **av) { -u64 perf_cap;int max, i;setup_vm ();
@@ -63,15 +62,13 @@ int main(int ac, char **av)return report_summary (); } - perf_cap =this_cpu_perf_capabilities (); - -if (!(perf_cap &PMU_CAP_LBR_FMT )) { +if (!pmu_lbr_version ()) {report_skip ("(Architectural) LBR is not supported." );return report_summary (); }printf ("PMU version: %d\n" ,pmu_version ()); -printf ("LBR version: %ld\n" , perf_cap &PMU_CAP_LBR_FMT ); +printf ("LBR version: %ld\n" ,pmu_lbr_version ());/* Look for LBR from and to MSRs */ lbr_from =MSR_LBR_CORE_FROM ;
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 16/27 x86/pmu: Snapshot CPUID.0xA PMU capabilities during BSP initialization
In reply to Sean ChristophersonSnapshot PMU info from CPUID.0xA into "struct pmu_caps pmu" during pmu_init() instead of reading CPUID.0xA every time a test wants to query PMU capabilities. Using pmu_caps to track various properties will also make it easier to hide the differences between AMD and Intel PMUs.
--- lib/x86/pmu.c | 16 ++++++++++++++++ lib/x86/pmu.h | 32 ++++++++++++++------------------ 2 files changed, 30 insertions(+), 18 deletions(-)
lib/x86/pmu.c +16 -0
@@ -4,6 +4,22 @@ struct pmu_caps pmu;void pmu_init (void ) { +struct cpuid cpuid_10 =cpuid (10 ); + + pmu.version = cpuid_10.a &0xff ; + +if (pmu.version >1 ) { + pmu.nr_fixed_counters = cpuid_10.d &0x1f ; + pmu.fixed_counter_width = (cpuid_10.d >>5 ) &0xff ; + } + + pmu.nr_gp_counters = (cpuid_10.a >>8 ) &0xff ; + pmu.gp_counter_width = (cpuid_10.a >>16 ) &0xff ; + pmu.gp_counter_mask_length = (cpuid_10.a >>24 ) &0xff ; + +/* CPUID.0xA.EBX bit is '1' if a counter is NOT available. */ + pmu.gp_counter_available = ~cpuid_10.b; +if (this_cpu_has (X86_FEATURE_PDCM )) pmu.perf_cap =rdmsr (MSR_IA32_PERF_CAPABILITIES ); }
lib/x86/pmu.h +14 -18
@@ -34,6 +34,13 @@#define EVNTSEL_INV (1 << EVNTSEL_INV_SHIF)struct pmu_caps { +u8 version; +u8 nr_fixed_counters; +u8 fixed_counter_width; +u8 nr_gp_counters; +u8 gp_counter_width; +u8 gp_counter_mask_length; +u32 gp_counter_available;u64 perf_cap; };
@@ -43,7 +50,7 @@ void pmu_init(void);static inline u8 pmu_version (void ) { -return cpuid (10 ).a &0xff ; +return pmu.version; }static inline bool this_cpu_has_pmu (void )
@@ -58,43 +65,32 @@ static inline bool this_cpu_has_perf_global_ctrl(void)static inline u8 pmu_nr_gp_counters (void ) { -return (cpuid (10 ).a >>8 ) &0xff ; +return pmu.nr_gp_counters; }static inline u8 pmu_gp_counter_width (void ) { -return (cpuid (10 ).a >>16 ) &0xff ; +return pmu.gp_counter_width; }static inline u8 pmu_gp_counter_mask_length (void ) { -return (cpuid (10 ).a >>24 ) &0xff ; +return pmu.gp_counter_mask_length; }static inline u8 pmu_nr_fixed_counters (void ) { -struct cpuid id =cpuid (10 ); - -if ((id.a &0xff ) >1 ) -return id.d &0x1f ; -else -return 0 ; +return pmu.nr_fixed_counters; }static inline u8 pmu_fixed_counter_width (void ) { -struct cpuid id =cpuid (10 ); - -if ((id.a &0xff ) >1 ) -return (id.d >>5 ) &0xff ; -else -return 0 ; +return pmu.fixed_counter_width; }static inline bool pmu_gp_counter_is_available (int i) { -/* CPUID.0xA.EBX bit is '1 if they counter is NOT available. */ -return !(cpuid (10 ).b &BIT (i)); +return pmu.gp_counter_available &BIT (i); }static inline u64 this_cpu_perf_capabilities (void )
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 17/27 x86/pmu: Drop wrappers that just passthrough pmu_caps fields
In reply to Sean ChristophersonDrop wrappers that are and always will be pure passthroughs of pmu_caps fields, e.g. the number of fixed/general_purpose counters can always be determined during PMU initialization and doesn't need runtime logic.
No functional change intended.
--- lib/x86/pmu.h | 43 ++++-------------------------------- x86/pmu.c | 60 +++++++++++++++++++++------------------------------ x86/pmu_lbr.c | 2 +- 3 files changed, 30 insertions(+), 75 deletions(-)
lib/x86/pmu.h +4 -39
@@ -48,44 +48,14 @@ extern struct pmu_caps pmu;void pmu_init (void ); -static inline u8 pmu_version (void ) -{ -return pmu.version; -} -static inline bool this_cpu_has_pmu (void ) { -return !!pmu_version (); +return !!pmu.version; }static inline bool this_cpu_has_perf_global_ctrl (void ) { -return pmu_version () >1 ; -} - -static inline u8 pmu_nr_gp_counters (void ) -{ -return pmu.nr_gp_counters; -} - -static inline u8 pmu_gp_counter_width (void ) -{ -return pmu.gp_counter_width; -} - -static inline u8 pmu_gp_counter_mask_length (void ) -{ -return pmu.gp_counter_mask_length; -} - -static inline u8 pmu_nr_fixed_counters (void ) -{ -return pmu.nr_fixed_counters; -} - -static inline u8 pmu_fixed_counter_width (void ) -{ -return pmu.fixed_counter_width; +return pmu.version >1 ; }static inline bool pmu_gp_counter_is_available (int i)
@@ -93,19 +63,14 @@ static inline bool pmu_gp_counter_is_available(int i) return pmu.gp_counter_available & BIT(i); } -static inline u64 this_cpu_perf_capabilities (void ) -{ -return pmu.perf_cap; -} -static inline u64 pmu_lbr_version (void ) { -return this_cpu_perf_capabilities () &PMU_CAP_LBR_FMT ; +return pmu.perf_cap &PMU_CAP_LBR_FMT ; }static inline bool pmu_has_full_writes (void ) { -return this_cpu_perf_capabilities () &PMU_CAP_FW_WRITES ; +return pmu.perf_cap &PMU_CAP_FW_WRITES ; }#endif /* _X86_PMU_H_ */
x86/pmu.c +25 -35
188 lines not on this page · raw message
x86/pmu_lbr.c +1 -1
@@ -67,7 +67,7 @@ int main(int ac, char **av)return report_summary (); } -printf ("PMU version: %d\n" ,pmu_version ()); +printf ("PMU version: %d\n" , pmu.version);printf ("LBR version: %ld\n" ,pmu_lbr_version ());/* Look for LBR from and to MSRs */
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 18/27 x86/pmu: Track GP counter and event select base MSRs in pmu_caps
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Snapshot the base MSRs for GP counters and event selects during pmu_init() so that tests don't need to manually compute the bases.
--- lib/x86/pmu.c | 2 ++ lib/x86/pmu.h | 18 +++++++++++++++ x86/pmu.c | 63 ++++++++++++++++++++++++++------------------------- 3 files changed, 52 insertions(+), 31 deletions(-)
lib/x86/pmu.c +2 -0
@@ -22,4 +22,6 @@ void pmu_init(void)if (this_cpu_has (X86_FEATURE_PDCM )) pmu.perf_cap =rdmsr (MSR_IA32_PERF_CAPABILITIES ); + pmu.msr_gp_counter_base =MSR_IA32_PERFCTR0 ; + pmu.msr_gp_event_select_base =MSR_P6_EVNTSEL0 ; }
lib/x86/pmu.h +18 -0
@@ -41,6 +41,9 @@ struct pmu_caps {u8 gp_counter_width;u8 gp_counter_mask_length;u32 gp_counter_available; +u32 msr_gp_counter_base; +u32 msr_gp_event_select_base; +u64 perf_cap; };
@@ -48,6 +51,16 @@ extern struct pmu_caps pmu;void pmu_init (void ); +static inline u32 MSR_GP_COUNTERx (unsigned int i) +{ +return pmu.msr_gp_counter_base + i; +} + +static inline u32 MSR_GP_EVENT_SELECTx (unsigned int i) +{ +return pmu.msr_gp_event_select_base + i; +} +static inline bool this_cpu_has_pmu (void ) {return !!pmu.version;
@@ -73,4 +86,9 @@ static inline bool pmu_has_full_writes(void) return pmu.perf_cap & PMU_CAP_FW_WRITES; } +static inline bool pmu_use_full_writes (void ) +{ +return pmu.msr_gp_counter_base ==MSR_IA32_PMC0 ; +} +#endif /* _X86_PMU_H_ */
x86/pmu.c +32 -31
190 lines not on this page · raw message
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 19/27 x86/pmu: Add helper to get fixed counter MSR index
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Add a helper to get the index of a fixed counter instead of manually calculating the index, a future patch will add more users of the fixed counter MSRs.
No functional change intended.
--- lib/x86/pmu.h | 5 +++++ x86/pmu.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-)
lib/x86/pmu.h +5 -0
@@ -91,4 +91,9 @@ static inline bool pmu_use_full_writes(void) return pmu.msr_gp_counter_base == MSR_IA32_PMC0; } +static inline u32 MSR_PERF_FIXED_CTRx (unsigned int i) +{ +return MSR_CORE_PERF_FIXED_CTR0 + i; +} +#endif /* _X86_PMU_H_ */
x86/pmu.c +1 -1
@@ -390,7 +390,7 @@ static void check_rdpmc(void) .idx = i }; -wrmsr (MSR_CORE_PERF_FIXED_CTR0 + i, x); +wrmsr (MSR_PERF_FIXED_CTRx (i), x);report (rdpmc (i | (1 <<30 )) == x,"fixed cntr-%d" , i); exc =test_for_exception (GP_VECTOR , do_rdpmc_fast, &cnt);
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 20/27 x86/pmu: Reset GP and Fixed counters during pmu_init().
In reply to Sean ChristophersonIn generic PMU testing, it is very common to initialize the test env by resetting counters registers. Add helpers to reset all PMU counters for code reusability, and reset all counters during PMU initialization for good measure.
--- lib/x86/pmu.c | 2 ++ lib/x86/pmu.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+)
lib/x86/pmu.c +2 -0
@@ -24,4 +24,6 @@ void pmu_init(void) pmu.perf_cap =rdmsr (MSR_IA32_PERF_CAPABILITIES ); pmu.msr_gp_counter_base =MSR_IA32_PERFCTR0 ; pmu.msr_gp_event_select_base =MSR_P6_EVNTSEL0 ; + +pmu_reset_all_counters (); }
lib/x86/pmu.h +28 -0
@@ -96,4 +96,32 @@ static inline u32 MSR_PERF_FIXED_CTRx(unsigned int i)return MSR_CORE_PERF_FIXED_CTR0 + i; } +static inline void pmu_reset_all_gp_counters (void ) +{ +unsigned int idx; + +for (idx =0 ; idx < pmu.nr_gp_counters; idx++) { +wrmsr (MSR_GP_EVENT_SELECTx (idx),0 ); +wrmsr (MSR_GP_COUNTERx (idx),0 ); + } +} + +static inline void pmu_reset_all_fixed_counters (void ) +{ +unsigned int idx; + +if (!pmu.nr_fixed_counters) +return ; + +wrmsr (MSR_CORE_PERF_FIXED_CTR_CTRL ,0 ); +for (idx =0 ; idx < pmu.nr_fixed_counters; idx++) +wrmsr (MSR_PERF_FIXED_CTRx (idx),0 ); +} + +static inline void pmu_reset_all_counters (void ) +{ +pmu_reset_all_gp_counters (); +pmu_reset_all_fixed_counters (); +} +#endif /* _X86_PMU_H_ */
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 21/27 x86/pmu: Track global status/control/clear MSRs in pmu_caps
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Track the global PMU MSRs in pmu_caps so that tests don't need to manually differntiate between AMD and Intel. Although AMD and Intel PMUs have the same semantics in terms of global control features (including ctl and status), their MSR indexes are not the same
--- lib/x86/pmu.c | 3 +++ lib/x86/pmu.h | 9 +++++++++ x86/pmu.c | 31 +++++++++++++------------------ 3 files changed, 25 insertions(+), 18 deletions(-)
lib/x86/pmu.c +3 -0
@@ -24,6 +24,9 @@ void pmu_init(void) pmu.perf_cap =rdmsr (MSR_IA32_PERF_CAPABILITIES ); pmu.msr_gp_counter_base =MSR_IA32_PERFCTR0 ; pmu.msr_gp_event_select_base =MSR_P6_EVNTSEL0 ; + pmu.msr_global_status =MSR_CORE_PERF_GLOBAL_STATUS ; + pmu.msr_global_ctl =MSR_CORE_PERF_GLOBAL_CTRL ; + pmu.msr_global_status_clr =MSR_CORE_PERF_GLOBAL_OVF_CTRL ;pmu_reset_all_counters (); }
lib/x86/pmu.h +9 -0
@@ -44,6 +44,10 @@ struct pmu_caps {u32 msr_gp_counter_base;u32 msr_gp_event_select_base; +u32 msr_global_status; +u32 msr_global_ctl; +u32 msr_global_status_clr; +u64 perf_cap; };
@@ -124,4 +128,9 @@ static inline void pmu_reset_all_counters(void) pmu_reset_all_fixed_counters(); } +static inline void pmu_clear_global_status (void ) +{ +wrmsr (pmu.msr_global_status_clr,rdmsr (pmu.msr_global_status)); +} +#endif /* _X86_PMU_H_ */
x86/pmu.c +13 -18
@@ -103,15 +103,12 @@ static struct pmu_event* get_counter_event(pmu_counter_t *cnt)static void global_enable (pmu_counter_t *cnt) { cnt->idx =event_to_global_idx (cnt); - -wrmsr (MSR_CORE_PERF_GLOBAL_CTRL ,rdmsr (MSR_CORE_PERF_GLOBAL_CTRL ) | - (1ull << cnt->idx)); +wrmsr (pmu.msr_global_ctl,rdmsr (pmu.msr_global_ctl) |BIT_ULL (cnt->idx)); }static void global_disable (pmu_counter_t *cnt) { -wrmsr (MSR_CORE_PERF_GLOBAL_CTRL ,rdmsr (MSR_CORE_PERF_GLOBAL_CTRL ) & - ~(1ull << cnt->idx)); +wrmsr (pmu.msr_global_ctl,rdmsr (pmu.msr_global_ctl) & ~BIT_ULL (cnt->idx)); }static void __start_event (pmu_counter_t *evt,uint64_t count)
@@ -286,7 +283,7 @@ static void check_counter_overflow(void) overflow_preset =measure_for_overflow (&cnt);/* clear status before test */ -wrmsr (MSR_CORE_PERF_GLOBAL_OVF_CTRL ,rdmsr (MSR_CORE_PERF_GLOBAL_STATUS )); +pmu_clear_global_status ();report_prefix_push ("overflow" );
@@ -313,10 +310,10 @@ static void check_counter_overflow(void) idx =event_to_global_idx (&cnt);__measure (&cnt, cnt.count);report (cnt.count ==1 ,"cntr-%d" , i); - status =rdmsr (MSR_CORE_PERF_GLOBAL_STATUS ); + status =rdmsr (pmu.msr_global_status);report (status & (1ull << idx),"status-%d" , i); -wrmsr (MSR_CORE_PERF_GLOBAL_OVF_CTRL , status); - status =rdmsr (MSR_CORE_PERF_GLOBAL_STATUS ); +wrmsr (pmu.msr_global_status_clr, status); + status =rdmsr (pmu.msr_global_status);report (!(status & (1ull << idx)),"status clear-%d" , i);report (check_irq () == (i %2 ),"irq-%d" , i); }
@@ -421,8 +418,7 @@ static void check_running_counter_wrmsr(void)report (evt.count < gp_events[1 ].min,"cntr" );/* clear status before overflow test */ -wrmsr (MSR_CORE_PERF_GLOBAL_OVF_CTRL , -rdmsr (MSR_CORE_PERF_GLOBAL_STATUS )); +pmu_clear_global_status ();start_event (&evt);
@@ -434,8 +430,8 @@ static void check_running_counter_wrmsr(void)loop ();stop_event (&evt); - status =rdmsr (MSR_CORE_PERF_GLOBAL_STATUS ); -report (status &1 ,"status" ); + status =rdmsr (pmu.msr_global_status); +report (status &1 ,"status msr bit" );report_prefix_pop (); }
@@ -455,8 +451,7 @@ static void check_emulated_instr(void) };report_prefix_push ("emulated instruction" ); -wrmsr (MSR_CORE_PERF_GLOBAL_OVF_CTRL , -rdmsr (MSR_CORE_PERF_GLOBAL_STATUS )); +pmu_clear_global_status ();start_event (&brnch_cnt);start_event (&instr_cnt);
@@ -490,7 +485,7 @@ static void check_emulated_instr(void) : : "eax", "ebx", "ecx", "edx"); -wrmsr (MSR_CORE_PERF_GLOBAL_CTRL ,0 ); +wrmsr (pmu.msr_global_ctl,0 );stop_event (&brnch_cnt);stop_event (&instr_cnt);
@@ -502,7 +497,7 @@ static void check_emulated_instr(void)report (brnch_cnt.count - brnch_start >=EXPECTED_BRNCH ,"branch count" );// Additionally check that those counters overflowed properly. - status =rdmsr (MSR_CORE_PERF_GLOBAL_STATUS ); + status =rdmsr (pmu.msr_global_status);report (status &1 ,"branch counter overflow" );report (status &2 ,"instruction counter overflow" );
@@ -590,7 +585,7 @@ static void set_ref_cycle_expectations(void)if (!pmu.nr_gp_counters || !pmu_gp_counter_is_available (2 ))return ; -wrmsr (MSR_CORE_PERF_GLOBAL_CTRL ,0 ); +wrmsr (pmu.msr_global_ctl,0 ); t0 =fenced_rdtsc ();start_event (&cnt);
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 22/27 x86: Add tests for Guest Processor Event Based Sampling (PEBS)
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
This unit-test is intended to test the KVM's support for the Processor Event Based Sampling (PEBS) which is another PMU feature on Intel processors (start from Ice Lake Server).
If a bit in PEBS_ENABLE is set to 1, its corresponding counter will write at least one PEBS records (including partial state of the vcpu at the time of the current hardware event) to the guest memory on counter overflow, and trigger an interrupt at a specific DS state. The format of a PEBS record can be configured by another register.
These tests cover most usage scenarios, for example there are some specially constructed scenarios (not a typical behaviour of Linux PEBS driver). It lowers the threshold for others to understand this feature and opens up more exploration of KVM implementation or hw feature itself.
--- lib/x86/msr.h | 1 + lib/x86/pmu.h | 34 ++++ x86/Makefile.x86_64 | 1 + x86/pmu_pebs.c | 433 ++++++++++++++++++++++++++++++++++++++++++++ x86/unittests.cfg | 8 + 5 files changed, 477 insertions(+) create mode 100644 x86/pmu_pebs.c
lib/x86/msr.h +1 -0
@@ -52,6 +52,7 @@#define MSR_IA32_MCG_CTL 0x0000017b#define MSR_IA32_PEBS_ENABLE 0x000003f1 +#define MSR_PEBS_DATA_CFG 0x000003f2#define MSR_IA32_DS_AREA 0x00000600#define MSR_IA32_PERF_CAPABILITIES 0x00000345
lib/x86/pmu.h +34 -0
@@ -14,6 +14,8 @@#define PMU_CAP_LBR_FMT 0x3f#define PMU_CAP_FW_WRITES (1ULL << 13) +#define PMU_CAP_PEBS_BASELINE (1ULL << 14) +#define PERF_CAP_PEBS_FORMAT 0xf00#define EVNSEL_EVENT_SHIFT 0#define EVNTSEL_UMASK_SHIFT 8
@@ -33,6 +35,18 @@#define EVNTSEL_INT (1 << EVNTSEL_INT_SHIFT)#define EVNTSEL_INV (1 << EVNTSEL_INV_SHIF) +#define GLOBAL_STATUS_BUFFER_OVF_BIT 62 +#define GLOBAL_STATUS_BUFFER_OVF BIT_ULL(GLOBAL_STATUS_BUFFER_OVF_BIT) + +#define PEBS_DATACFG_MEMINFO BIT_ULL(0) +#define PEBS_DATACFG_GP BIT_ULL(1) +#define PEBS_DATACFG_XMMS BIT_ULL(2) +#define PEBS_DATACFG_LBRS BIT_ULL(3) + +#define ICL_EVENTSEL_ADAPTIVE (1ULL << 34) +#define PEBS_DATACFG_LBR_SHIFT 24 +#define MAX_NUM_LBR_ENTRY 32 +struct pmu_caps {u8 version;u8 nr_fixed_counters;
@@ -90,6 +104,11 @@ static inline bool pmu_has_full_writes(void)return pmu.perf_cap &PMU_CAP_FW_WRITES ; } +static inline void pmu_activate_full_writes (void ) +{ + pmu.msr_gp_counter_base =MSR_IA32_PMC0 ; +} +static inline bool pmu_use_full_writes (void ) {return pmu.msr_gp_counter_base ==MSR_IA32_PMC0 ;
@@ -133,4 +152,19 @@ static inline void pmu_clear_global_status(void) wrmsr(pmu.msr_global_status_clr, rdmsr(pmu.msr_global_status)); } +static inline bool pmu_has_pebs (void ) +{ +return pmu.version >1 ; +} + +static inline u8 pmu_pebs_format (void ) +{ +return (pmu.perf_cap &PERF_CAP_PEBS_FORMAT ) >>8 ; +} + +static inline bool pmu_has_pebs_baseline (void ) +{ +return pmu.perf_cap &PMU_CAP_PEBS_BASELINE ; +} +#endif /* _X86_PMU_H_ */
x86/Makefile.x86_64 +1 -0
@@ -33,6 +33,7 @@ tests += $(TEST_DIR)/vmware_backdoors.$(exe) tests += $(TEST_DIR)/rdpru.$(exe) tests += $(TEST_DIR)/pks.$(exe) tests += $(TEST_DIR)/pmu_lbr.$(exe) +tests += $(TEST_DIR)/pmu_pebs.$(exe) ifeq ($(CONFIG_EFI),y) tests += $(TEST_DIR)/amd_sev.$(exe)
x86/pmu_pebs.c new file +433 -0
433 lines not on this page · raw message
x86/unittests.cfg +8 -0
@@ -200,6 +200,14 @@ check = /proc/sys/kernel/nmi_watchdog=0 accel = kvm groups = pmu +[pmu_pebs] +arch = x86_64 +file = pmu_pebs.flat +extra_params = -cpu host,migratable=no +check = /proc/sys/kernel/nmi_watchdog=0 +accel = kvm +groups = pmu + [vmware_backdoors] file = vmware_backdoors.flat extra_params = -machine vmport=on -cpu max
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 23/27 x86/pmu: Add global helpers to cover Intel Arch PMU Version 1
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
To test Intel arch pmu version 1, most of the basic framework and use cases which test any PMU counter do not require any changes, except no access to registers introduced only in PMU version 2.
Adding some guardian's checks can seamlessly support version 1, while opening the door for normal AMD PMUs tests.
--- lib/x86/pmu.c | 9 ++++++--- lib/x86/pmu.h | 5 +++++ x86/pmu.c | 47 +++++++++++++++++++++++++++++++---------------- 3 files changed, 42 insertions(+), 19 deletions(-)
lib/x86/pmu.c +6 -3
@@ -24,9 +24,12 @@ void pmu_init(void) pmu.perf_cap =rdmsr (MSR_IA32_PERF_CAPABILITIES ); pmu.msr_gp_counter_base =MSR_IA32_PERFCTR0 ; pmu.msr_gp_event_select_base =MSR_P6_EVNTSEL0 ; - pmu.msr_global_status =MSR_CORE_PERF_GLOBAL_STATUS ; - pmu.msr_global_ctl =MSR_CORE_PERF_GLOBAL_CTRL ; - pmu.msr_global_status_clr =MSR_CORE_PERF_GLOBAL_OVF_CTRL ; + +if (this_cpu_has_perf_global_status ()) { + pmu.msr_global_status =MSR_CORE_PERF_GLOBAL_STATUS ; + pmu.msr_global_ctl =MSR_CORE_PERF_GLOBAL_CTRL ; + pmu.msr_global_status_clr =MSR_CORE_PERF_GLOBAL_OVF_CTRL ; + }pmu_reset_all_counters (); }
lib/x86/pmu.h +5 -0
@@ -89,6 +89,11 @@ static inline bool this_cpu_has_perf_global_ctrl(void)return pmu.version >1 ; } +static inline bool this_cpu_has_perf_global_status (void ) +{ +return pmu.version >1 ; +} +static inline bool pmu_gp_counter_is_available (int i) {return pmu.gp_counter_available &BIT (i);
x86/pmu.c +31 -16
113 lines not on this page · raw message
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 24/27 x86/pmu: Add gp_events pointer to route different event tables
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
AMD and Intel do not share the same set of coding rules for performance events, and code to test the same performance event can be reused by pointing to a different coding table, noting that the table size also needs to be updated.
--- x86/pmu.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
x86/pmu.c +12 -7
@@ -30,7 +30,7 @@ struct pmu_event {uint32_t unit_sel;int min;int max; -} gp_events[] = { +} intel_gp_events[] = { {"core cycles", 0x003c, 1*N, 50*N}, {"instructions", 0x00c0, 10*N, 10.2*N}, {"ref cycles", 0x013c, 1*N, 30*N},
@@ -46,6 +46,9 @@ struct pmu_event {char *buf; +static struct pmu_event *gp_events; +static unsigned int gp_events_size; +static inline void loop (void ) {unsigned long tmp, tmp2, tmp3;
@@ -91,7 +94,7 @@ static struct pmu_event* get_counter_event(pmu_counter_t *cnt)if (is_gp (cnt)) {int i; -for (i =0 ; i <sizeof (gp_events)/sizeof (gp_events[0 ]); i++) +for (i =0 ; i < gp_events_size; i++)if (gp_events[i].unit_sel == (cnt->config &0xffff ))return &gp_events[i]; }else
@@ -213,7 +216,7 @@ static void check_gp_counters(void) {int i; -for (i =0 ; i <sizeof (gp_events)/sizeof (gp_events[0 ]); i++) +for (i =0 ; i < gp_events_size; i++)if (pmu_gp_counter_is_available (i))check_gp_counter (&gp_events[i]);else
@@ -246,7 +249,7 @@ static void check_counters_many(void) cnt[n].ctr =MSR_GP_COUNTERx (n); cnt[n].config =EVNTSEL_OS |EVNTSEL_USR | - gp_events[i %ARRAY_SIZE (gp_events)].unit_sel; + gp_events[i % gp_events_size].unit_sel; n++; }for (i =0 ; i < pmu.nr_fixed_counters; i++) {
@@ -595,7 +598,7 @@ static void set_ref_cycle_expectations(void) {pmu_counter_t cnt = { .ctr =MSR_IA32_PERFCTR0 , - .config =EVNTSEL_OS |EVNTSEL_USR | gp_events[2 ].unit_sel, + .config =EVNTSEL_OS |EVNTSEL_USR | intel_gp_events[2 ].unit_sel, };uint64_t tsc_delta;uint64_t t0, t1, t2, t3;
@@ -631,8 +634,8 @@ static void set_ref_cycle_expectations(void)if (!tsc_delta)return ; - gp_events[2 ].min = (gp_events[2 ].min * cnt.count) / tsc_delta; - gp_events[2 ].max = (gp_events[2 ].max * cnt.count) / tsc_delta; + intel_gp_events[2 ].min = (intel_gp_events[2 ].min * cnt.count) / tsc_delta; + intel_gp_events[2 ].max = (intel_gp_events[2 ].max * cnt.count) / tsc_delta; }static void check_invalid_rdpmc_gp (void )
@@ -656,6 +659,8 @@ int main(int ac, char **av)return report_summary (); } + gp_events = (struct pmu_event *)intel_gp_events; + gp_events_size =sizeof (intel_gp_events)/sizeof (intel_gp_events[0 ]);set_ref_cycle_expectations ();printf ("PMU version: %d\n" , pmu.version);
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 25/27 x86/pmu: Add pmu_caps flag to track if CPU is Intel (versus AMD)
In reply to Sean ChristophersonAdd a flag to track whether the PMU is backed by an Intel CPU. Future support for AMD will sadly need to constantly check whether the PMU is Intel or AMD, and invoking is_intel() every time is rather expensive due to it requiring CPUID (VM-Exit) and a string comparison.
--- lib/x86/pmu.c | 5 +++++ lib/x86/pmu.h | 1 + x86/pmu_lbr.c | 2 +- x86/pmu_pebs.c | 2 +- 4 files changed, 8 insertions(+), 2 deletions(-)
lib/x86/pmu.c +5 -0
@@ -6,6 +6,11 @@ void pmu_init(void) {struct cpuid cpuid_10 =cpuid (10 ); + pmu.is_intel =is_intel (); + +if (!pmu.is_intel) +return ; + pmu.version = cpuid_10.a &0xff ;if (pmu.version >1 ) {
lib/x86/pmu.h +1 -0
@@ -48,6 +48,7 @@#define MAX_NUM_LBR_ENTRY 32struct pmu_caps { +bool is_intel;u8 version;u8 nr_fixed_counters;u8 fixed_counter_width;
x86/pmu_lbr.c +1 -1
@@ -47,7 +47,7 @@ int main(int ac, char **av)setup_vm (); -if (!is_intel ()) { +if (!pmu.is_intel) {report_skip ("PMU_LBR test is for intel CPU's only" );return report_summary (); }
x86/pmu_pebs.c +1 -1
@@ -392,7 +392,7 @@ int main(int ac, char **av)if (pmu_has_full_writes ())pmu_activate_full_writes (); -if (!is_intel ()) { +if (!pmu.is_intel) {report_skip ("PEBS requires Intel ICX or later, non-Intel detected" );return report_summary (); }else if (!pmu_has_pebs ()) {
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Sean Christopherson
reviewerPATCH v5 26/27 x86/pmu: Update testcases to cover AMD PMU
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
AMD core PMU before Zen4 did not have version numbers, there were no fixed counters, it had a hard-coded number of generic counters, bit-width, and only hardware events common across amd generations (starting with K7) were added to amd_gp_events[] table.
All above differences are instantiated at the detection step, and it also covers the K7 PMU registers, which is consistent with bare-metal.
--- lib/x86/msr.h | 17 +++++++++++++ lib/x86/pmu.c | 59 ++++++++++++++++++++++++++++----------------- lib/x86/pmu.h | 13 +++++++++- lib/x86/processor.h | 1 + x86/pmu.c | 58 +++++++++++++++++++++++++++++++++++--------- 5 files changed, 114 insertions(+), 34 deletions(-)
lib/x86/msr.h +17 -0
@@ -146,6 +146,23 @@#define FAM10H_MMIO_CONF_BASE_SHIFT 20#define MSR_FAM10H_NODE_ID 0xc001100c +/* Fam 15h MSRs */ +#define MSR_F15H_PERF_CTL 0xc0010200 +#define MSR_F15H_PERF_CTL0 MSR_F15H_PERF_CTL +#define MSR_F15H_PERF_CTL1 (MSR_F15H_PERF_CTL + 2) +#define MSR_F15H_PERF_CTL2 (MSR_F15H_PERF_CTL + 4) +#define MSR_F15H_PERF_CTL3 (MSR_F15H_PERF_CTL + 6) +#define MSR_F15H_PERF_CTL4 (MSR_F15H_PERF_CTL + 8) +#define MSR_F15H_PERF_CTL5 (MSR_F15H_PERF_CTL + 10) + +#define MSR_F15H_PERF_CTR 0xc0010201 +#define MSR_F15H_PERF_CTR0 MSR_F15H_PERF_CTR +#define MSR_F15H_PERF_CTR1 (MSR_F15H_PERF_CTR + 2) +#define MSR_F15H_PERF_CTR2 (MSR_F15H_PERF_CTR + 4) +#define MSR_F15H_PERF_CTR3 (MSR_F15H_PERF_CTR + 6) +#define MSR_F15H_PERF_CTR4 (MSR_F15H_PERF_CTR + 8) +#define MSR_F15H_PERF_CTR5 (MSR_F15H_PERF_CTR + 10) +/* K8 MSRs */ #define MSR_K8_TOP_MEM1 0xc001001a#define MSR_K8_TOP_MEM2 0xc001001d
lib/x86/pmu.c +37 -22
73 lines not on this page · raw message
lib/x86/pmu.h +12 -1
@@ -10,6 +10,11 @@/* Performance Counter Vector for the LVT PC Register */ #define PMI_VECTOR 32 +#define AMD64_NUM_COUNTERS 4 +#define AMD64_NUM_COUNTERS_CORE 6 + +#define PMC_DEFAULT_WIDTH 48 +#define DEBUGCTLMSR_LBR (1UL << 0)#define PMU_CAP_LBR_FMT 0x3f
@@ -72,17 +77,23 @@ void pmu_init(void);static inline u32 MSR_GP_COUNTERx (unsigned int i) { +if (pmu.msr_gp_counter_base ==MSR_F15H_PERF_CTR0 ) +return pmu.msr_gp_counter_base +2 * i; +return pmu.msr_gp_counter_base + i; }static inline u32 MSR_GP_EVENT_SELECTx (unsigned int i) { +if (pmu.msr_gp_event_select_base ==MSR_F15H_PERF_CTL0 ) +return pmu.msr_gp_event_select_base +2 * i; +return pmu.msr_gp_event_select_base + i; }static inline bool this_cpu_has_pmu (void ) { -return !!pmu.version; +return !pmu.is_intel || !!pmu.version; }static inline bool this_cpu_has_perf_global_ctrl (void )
lib/x86/processor.h +1 -0
@@ -252,6 +252,7 @@ static inline bool is_intel(void)* Extended Leafs, a.k.a. AMD defined */ #define X86_FEATURE_SVM (CPUID(0x80000001, 0, ECX, 2)) +#define X86_FEATURE_PERFCTR_CORE (CPUID(0x80000001, 0, ECX, 23))#define X86_FEATURE_NX (CPUID(0x80000001, 0, EDX, 20))#define X86_FEATURE_GBPAGES (CPUID(0x80000001, 0, EDX, 26))#define X86_FEATURE_RDTSCP (CPUID(0x80000001, 0, EDX, 27))
x86/pmu.c +47 -11
110 lines not on this page · raw message
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Paolo Bonzini
maintainerRe: [kvm-unit-tests PATCH v5 26/27] x86/pmu: Update testcases to cover AMD PMU
Review
Quoted 7 lines · Sean Christopherson
(quoted diff) +7
-0+ pmu.msr_gp_counter_base = MSR_F15H_PERF_CTR0; + pmu.msr_gp_event_select_base = MSR_F15H_PERF_CTL0; + if (!this_cpu_has(X86_FEATURE_PERFCTR_CORE)) + pmu.nr_gp_counters = AMD64_NUM_COUNTERS; + else + pmu.nr_gp_counters = AMD64_NUM_COUNTERS_CORE; +
If X86_FEATURE_PERFCTR_CORE is not set, pmu.msr_gp_*_base should point to MSR_K7_PERFCTR0/MSR_K7_EVNTSEL0:
lib/x86/pmu.c +6 -3
@@ -47,10 +47,13 @@ void pmu_init(void) pmu.msr_gp_event_select_base =MSR_F15H_PERF_CTL0 ;if (this_cpu_has (X86_FEATURE_AMD_PMU_V2 )) pmu.nr_gp_counters =cpuid (0x80000022 ).b &0xf ; -else if (!this_cpu_has (X86_FEATURE_PERFCTR_CORE )) - pmu.nr_gp_counters =AMD64_NUM_COUNTERS ; -else +else if (this_cpu_has (X86_FEATURE_PERFCTR_CORE )) pmu.nr_gp_counters =AMD64_NUM_COUNTERS_CORE ; +else { + pmu.nr_gp_counters =AMD64_NUM_COUNTERS ; + pmu.msr_gp_counter_base =MSR_K7_PERFCTR0 ; + pmu.msr_gp_event_select_base =MSR_K7_EVNTSEL0 ; + } pmu.gp_counter_width =PMC_DEFAULT_WIDTH ; pmu.gp_counter_mask_length = pmu.nr_gp_counters;
Paolo
Sean Christopherson
reviewerRe: [kvm-unit-tests PATCH v5 26/27] x86/pmu: Update testcases to cover AMD PMU
Quoted 11 lines · Paolo Bonzini
Review
Quoted 7 lines · Sean Christopherson
(quoted diff) +7
-0+ pmu.msr_gp_counter_base = MSR_F15H_PERF_CTR0; + pmu.msr_gp_event_select_base = MSR_F15H_PERF_CTL0; + if (!this_cpu_has(X86_FEATURE_PERFCTR_CORE)) + pmu.nr_gp_counters = AMD64_NUM_COUNTERS; + else + pmu.nr_gp_counters = AMD64_NUM_COUNTERS_CORE; +If
X86_FEATURE_PERFCTR_COREis not set, pmu.msr_gp_*_base should point to MSR_K7_PERFCTR0/MSR_K7_EVNTSEL0:
/facepalm
I only ran the PMU tests, which all passthrough the relevant host CPUID.
Glad you debugged this, all tests were failing on Milan due to this. I shudder to think about how long it would have taken me to figure this out.
Thanks!
lib/x86/pmu.c void pmu_init(void) · line 50
Quoted 12 lines
lib/x86/pmu.c +1
-3@@ -47,10 +47,13 @@ void pmu_init(void) pmu.msr_gp_event_select_base =MSR_F15H_PERF_CTL0 ;if (this_cpu_has (X86_FEATURE_AMD_PMU_V2 )) pmu.nr_gp_counters =cpuid (0x80000022 ).b &0xf ; -else if (!this_cpu_has (X86_FEATURE_PERFCTR_CORE )) - pmu.nr_gp_counters =AMD64_NUM_COUNTERS ; -else +else if (this_cpu_has (X86_FEATURE_PERFCTR_CORE ))
Nit, the if and else-if statements should also have braces.
Quoted 12 lines
lib/x86/pmu.c +5
-0pmu.nr_gp_counters =AMD64_NUM_COUNTERS_CORE ; + else { + pmu.nr_gp_counters =AMD64_NUM_COUNTERS ; + pmu.msr_gp_counter_base =MSR_K7_PERFCTR0 ; + pmu.msr_gp_event_select_base =MSR_K7_EVNTSEL0 ; + } pmu.gp_counter_width =PMC_DEFAULT_WIDTH ; pmu.gp_counter_mask_length = pmu.nr_gp_counters;Paolo
Sean Christopherson
reviewerPATCH v5 27/27 x86/pmu: Add AMD Guest PerfMonV2 testcases
In reply to Sean ChristophersonFrom: Like Xu <[email protected]>
Updated test cases to cover KVM enabling code for AMD Guest PerfMonV2.
The Intel-specific PMU helpers were added to check for AMD cpuid, and some of the same semantics of MSRs were assigned during the initialization phase. The vast majority of pmu test cases are reused seamlessly.
On some x86 machines (AMD only), even with retired events, the same workload is measured repeatedly and the number of events collected is erratic, which essentially reflects the details of hardware implementation, and from a software perspective, the type of event is an unprecise event, which brings a tolerance check in the counter overflow testcases.
--- lib/x86/msr.h | 5 +++++ lib/x86/pmu.c | 14 +++++++++++++- lib/x86/processor.h | 2 +- 3 files changed, 19 insertions(+), 2 deletions(-)
lib/x86/msr.h +5 -0
@@ -426,6 +426,11 @@#define MSR_CORE_PERF_GLOBAL_CTRL 0x0000038f#define MSR_CORE_PERF_GLOBAL_OVF_CTRL 0x00000390 +/* AMD Performance Counter Global Status and Control MSRs */ +#define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS 0xc0000300 +#define MSR_AMD64_PERF_CNTR_GLOBAL_CTL 0xc0000301 +#define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR 0xc0000302 +/* Geode defined MSRs */ #define MSR_GEODE_BUSCONT_CONF0 0x00001900
lib/x86/pmu.c +13 -1
@@ -39,9 +39,15 @@ void pmu_init(void) pmu.msr_global_status_clr =MSR_CORE_PERF_GLOBAL_OVF_CTRL ; } }else { +/* Performance Monitoring Version 2 Supported */ +if (this_cpu_has (X86_FEATURE_AMD_PMU_V2 )) + pmu.version =2 ; + pmu.msr_gp_counter_base =MSR_F15H_PERF_CTR0 ; pmu.msr_gp_event_select_base =MSR_F15H_PERF_CTL0 ; -if (!this_cpu_has (X86_FEATURE_PERFCTR_CORE )) +if (this_cpu_has (X86_FEATURE_AMD_PMU_V2 )) + pmu.nr_gp_counters =cpuid (0x80000022 ).b &0xf ; +else if (!this_cpu_has (X86_FEATURE_PERFCTR_CORE )) pmu.nr_gp_counters =AMD64_NUM_COUNTERS ;else pmu.nr_gp_counters =AMD64_NUM_COUNTERS_CORE ;
@@ -49,6 +55,12 @@ void pmu_init(void) pmu.gp_counter_width =PMC_DEFAULT_WIDTH ; pmu.gp_counter_mask_length = pmu.nr_gp_counters; pmu.gp_counter_available = (1u << pmu.nr_gp_counters) -1 ; + +if (this_cpu_has_perf_global_status ()) { + pmu.msr_global_status =MSR_AMD64_PERF_CNTR_GLOBAL_STATUS ; + pmu.msr_global_ctl =MSR_AMD64_PERF_CNTR_GLOBAL_CTL ; + pmu.msr_global_status_clr =MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR ; + } }pmu_reset_all_counters ();
lib/x86/processor.h +1 -1
@@ -266,7 +266,7 @@ static inline bool is_intel(void)#define X86_FEATURE_PAUSEFILTER (CPUID(0x8000000A, 0, EDX, 10))#define X86_FEATURE_PFTHRESHOLD (CPUID(0x8000000A, 0, EDX, 12))#define X86_FEATURE_VGIF (CPUID(0x8000000A, 0, EDX, 16)) - +#define X86_FEATURE_AMD_PMU_V2 (CPUID(0x80000022, 0, EAX, 0))static inline bool this_cpu_has (u64 feature) {
Signature · 1 line
2.38.1.431.g37b22c650d-goog
Quoted 3 lines · Sean Christopherson
There are no major changes in the test logic. The big cleanups are to add lib/x86/pmu.[c,h] and a global PMU capabilities struct to improve readability of the code and to hide some AMD vs. Intel details.
The extra patch helps. Thank you.
Quoted 5 lines
Like's v4 was tested on AMD Zen3/4 and Intel ICX/SPR machines, but this version has only been tested on AMD Zen3 (Milan) and Intel ICX and HSW, i.e. I haven't tested AMD PMU v2 or anything new in SPR (if there is anything in SPR?).
V5 tests passed on AMD Zen 4 (AMD PMU v2) and Intel SPR (for PEBS). Please move forward.
Quoted 64 lines · Sean Christopherson
This series is a big pile of PMU cleanups and enhancements from Like.
The changes are roughly divided into three parts: (1) fixes (2) cleanups, and (3) new test cases. The changes are bundled in a mega-series as the original, separate series was difficult to review/manage due to a number of dependencies.
There are no major changes in the test logic. The big cleanups are to add lib/x86/pmu.[c,h] and a global PMU capabilities struct to improve readability of the code and to hide some AMD vs. Intel details.
Like's v4 was tested on AMD Zen3/4 and Intel ICX/SPR machines, but this version has only been tested on AMD Zen3 (Milan) and Intel ICX and HSW, i.e. I haven't tested AMD PMU v2 or anything new in SPR (if there is anything in SPR?).
Like Xu (22):
x86/pmu: Add PDCM check before accessingPERF_CAPregister x86/pmu: Test emulation instructions on full-width counters x86/pmu: Pop up FW prefix to avoid out-of-context propagation x86/pmu: Report SKIP when testing Intel LBR on AMD platforms x86/pmu: Fix printed messages for emulated instruction test x86/pmu: Introduce__start_event()to drop all of the manual zeroing x86/pmu: Introduce multiple_{one, many}() to improve readability x86/pmu: Reset the expected count of the fixed counter 0 when i386
x86: create pmu group for quick pmu-scope testing x86/pmu: Refine info to clarify the current support x86/pmu: Update rdpmc testcase to cover #GP path
x86/pmu: RenamePC_VECTORtoPMI_VECTORfor better readability x86/pmu: Add lib/x86/pmu.[c.h] and move common code to header files x86/pmu: Snapshot PMUperf_capabilitiesduring BSP initialization x86/pmu: Track GP counter and event select base MSRs inpmu_capsx86/pmu: Add helper to get fixed counter MSR index x86/pmu: Track global status/control/clear MSRs inpmu_caps
x86: Add tests for Guest Processor Event Based Sampling (PEBS) x86/pmu: Add global helpers to cover Intel Arch PMU Version 1 x86/pmu: Addgp_eventspointer to route different event tables x86/pmu: Update testcases to cover AMD PMU
x86/pmu: Add AMD Guest PerfMonV2 testcasesSean Christopherson (5):
x86: Add a helper for the BSP's final init sequence common to all
flavors
x86/pmu: Snapshot CPUID.0xA PMU capabilities during BSP initialization x86/pmu: Drop wrappers that just passthroughpmu_capsfields x86/pmu: Reset GP and Fixed counters duringpmu_init(). x86/pmu: Addpmu_capsflag to track if CPU is Intel (versus AMD)lib/x86/asm/setup.h | 1 + lib/x86/msr.h | 30 +++ lib/x86/pmu.c | 67 +++++++ lib/x86/pmu.h | 187 +++++++++++++++++++ lib/x86/processor.h | 80 ++------ lib/x86/setup.c | 13 +- x86/Makefile.common | 1 + x86/Makefile.x86_64 | 1 + x86/cstart.S | 4 +- x86/cstart64.S | 4 +- x86/pmu.c | 360 ++++++++++++++++++++---------------- x86/pmu_lbr.c | 24 +-- x86/pmu_pebs.c | 433 ++++++++++++++++++++++++++++++++++++++++++++ x86/unittests.cfg | 10 + x86/vmx_tests.c | 1 + 15 files changed, 975 insertions(+), 241 deletions(-)
Quoted 3 lines
create mode 100644
lib/x86/pmu.c
create mode 100644lib/x86/pmu.h
create mode 100644x86/pmu_pebs.c
Applied, thanks.
Paolo