x86/pmu: Treat all AMD CPUs as having instruction/branch overcount errata
Sean Christopherson
reviewerPATCH 1/1 x86/pmu: Treat all AMD CPUs as having instruction/branch overcount errata
Apply the instructions and branches overcount errata to all AMD CPUs, which count VMRUN as a retired branch instruction in guest context. I.e. any asynchronous #VMEXITs during any measurement will result in an overcount.
--- lib/x86/pmu.c | 7 +++++++ x86/pmu.c | 7 +------ 2 files changed, 8 insertions(+), 6 deletions(-)
lib/x86/pmu.c +7 -0
@@ -72,6 +72,13 @@ void pmu_init(void) pmu.msr_global_status_clr =MSR_CORE_PERF_GLOBAL_OVF_CTRL ; } }else { +/* +* All AMD CPUs overcount instructions and branches retired, as +* they count VMRUN as a branch instruction in guest context. +*/ + pmu.errata.instructions_retired_overcount = true; + pmu.errata.branches_retired_overcount = true; +if (this_cpu_has (X86_FEATURE_PERFCTR_CORE )) {/* Performance Monitoring Version 2 Supported */ if (this_cpu_has (X86_FEATURE_AMD_PMU_V2 )) {
x86/pmu.c +1 -6
@@ -222,13 +222,8 @@ static void adjust_events_range(struct pmu_event *gp_events,* If HW supports GLOBAL_CTRL MSR, enabling and disabling PMCs are * moved in __precise_loop(). Thus, instructions and branches events * can be verified against a precise count instead of a rough range. -* -* Skip the precise checks on AMD, as AMD CPUs count VMRUN as a branch -* instruction in guest context, which* leads to intermittent failures -* as the counts will vary depending on how many asynchronous VM-Exits -* occur while running the measured code, e.g. if the host takes IRQs. */ -if (pmu.is_intel &&this_cpu_has_perf_global_ctrl ()) { +if (this_cpu_has_perf_global_ctrl ()) {if (!pmu.errata.instructions_retired_overcount) { gp_events[instruction_idx].min =LOOP_INSNS ; gp_events[instruction_idx].max =LOOP_INSNS ;
base-commit: 5a221342a025948fa7d84f0c404748f6ba57f3ad
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
Quoted 39 lines · Sean Christopherson
Apply the instructions and branches overcount errata to all AMD CPUs, which count VMRUN as a retired branch instruction in guest context. I.e. any asynchronous #VMEXITs during any measurement will result in an overcount.
Reported-by Sandipan Das <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- lib/x86/pmu.c | 7 +++++++ x86/pmu.c | 7 +------ 2 files changed, 8 insertions(+), 6 deletions(-)lib/x86/pmu.c +7
-0@@ -72,6 +72,13 @@ void pmu_init(void) pmu.msr_global_status_clr =MSR_CORE_PERF_GLOBAL_OVF_CTRL ; } }else { +/* +* All AMD CPUs overcount instructions and branches retired, as +* they count VMRUN as a branch instruction in guest context. +*/ + pmu.errata.instructions_retired_overcount = true; + pmu.errata.branches_retired_overcount = true; +if (this_cpu_has (X86_FEATURE_PERFCTR_CORE )) {/* Performance Monitoring Version 2 Supported */ if (this_cpu_has (X86_FEATURE_AMD_PMU_V2 )) {x86/pmu.c +0
-1@@ -222,13 +222,8 @@ static void adjust_events_range(struct pmu_event *gp_events,* If HW supports GLOBAL_CTRL MSR, enabling and disabling PMCs are * moved in __precise_loop(). Thus, instructions and branches events * can be verified against a precise count instead of a rough range. -*
x86/pmu.c static void adjust_events_range(struct pmu_event *gp_events, · line 229
Quoted 10 lines
x86/pmu.c +1
-5-* Skip the precise checks on AMD, as AMD CPUs count VMRUN as a branch -* instruction in guest context, which* leads to intermittent failures -* as the counts will vary depending on how many asynchronous VM-Exits -* occur while running the measured code, e.g. if the host takes IRQs. */ -if (pmu.is_intel &&this_cpu_has_perf_global_ctrl ()) { +if (this_cpu_has_perf_global_ctrl ()) {if (!pmu.errata.instructions_retired_overcount) { gp_events[instruction_idx].min =LOOP_INSNS ; gp_events[instruction_idx].max =LOOP_INSNS ;
Commit 61a7ee521d ("x86/pmu: Handle instruction overcount issue in overflow test") makes measure_for_overflow() always return 1 - LOOP_INSNS (-10000016) when pmu.errata.instructions_retired_overcount is true. For runs without "perfmon-v2" on a Zen 5 system, I see that the overshoot from this preset value is between 125 and 135, which makes the following condition in check_counter_overflow() fail.
report(cnt.count == 0xffffffffffff || cnt.count < 7, "cntr-%d", i);
How should we handle this?
Restrict the condition in measure_for_overflow() to return 1 - LOOP_INSNS if (pmu.is_intel && pmu.errata.instructions_retired_overcount)
or, accept a larger overshoot.
x86/pmu.c +2 -0
@@ -584,6 +584,8 @@ static void check_counter_overflow(void)else report (cnt.count ==1 ,"cntr-%d" , i); } +else if (pmu.errata.instructions_retired_overcount) +report (cnt.count ==0xffffffffffff || cnt.count <150 ,"cntr-%d" , i);else report (cnt.count ==0xffffffffffff || cnt.count <7 ,"cntr-%d" , i);
Sean Christopherson
reviewerQuoted 19 lines · Sandipan Das
Quoted 18 lines · Sean Christopherson
x86/pmu.c +1
-6@@ -222,13 +222,8 @@ static void adjust_events_range(struct pmu_event *gp_events,* If HW supports GLOBAL_CTRL MSR, enabling and disabling PMCs are * moved in __precise_loop(). Thus, instructions and branches events * can be verified against a precise count instead of a rough range. -* -* Skip the precise checks on AMD, as AMD CPUs count VMRUN as a branch -* instruction in guest context, which* leads to intermittent failures -* as the counts will vary depending on how many asynchronous VM-Exits -* occur while running the measured code, e.g. if the host takes IRQs. */ -if (pmu.is_intel &&this_cpu_has_perf_global_ctrl ()) { +if (this_cpu_has_perf_global_ctrl ()) {if (!pmu.errata.instructions_retired_overcount) { gp_events[instruction_idx].min =LOOP_INSNS ;
Quoted 8 lines
x86/pmu.c static void adjust_events_range(struct pmu_event *gp_events, · line 229
Quoted 2 lines
x86/pmu.c +0
-0gp_events[instruction_idx].max =LOOP_INSNS ;Commit
61a7ee521d("x86/pmu: Handle instruction overcount issue in overflow test") makesmeasure_for_overflow()always return 1 -LOOP_INSNS(-10000016) when pmu.errata.instructions_retired_overcount is true. For runs without "perfmon-v2" on a Zen 5 system, I see that the overshoot from this preset value is between 125 and 135,
Heh, I was going to say "woah, that's a lot of exits!", then I realized the loop does a million runs...
Quoted 10 lines
which makes the following condition in
check_counter_overflow()fail.report(cnt.count == 0xffffffffffff || cnt.count < 7, "cntr-%d", i);How should we handle this?
Restrict the condition in
measure_for_overflow()to return 1 -LOOP_INSNSif (pmu.is_intel && pmu.errata.instructions_retired_overcount)or, accept a larger overshoot.
Doesn't it have to be the latter? Because won't the test fail if the first run of __measure() from measure_for_overflow() takes more exits than the second run of __measure()?
x86/pmu.c static void check_counter_overflow(void) · line 588
Quoted 10 lines
x86/pmu.c +2
-0@@ -584,6 +584,8 @@ static void check_counter_overflow(void)else report (cnt.count ==1 ,"cntr-%d" , i); } +else if (pmu.errata.instructions_retired_overcount) +report (cnt.count ==0xffffffffffff || cnt.count <150 ,"cntr-%d" , i);
Do you have any idea where the 0xffffffffffff check comes from? Commit b883751a ("x86/pmu: Update testcases to cover AMD PMU") doesn't explain it at all.
Depending on what's up with the 0xffffffffffff thing, my vote would be to go big hammer, and express the overflow as a fraction of loop instructions, e.g. allow for an extra 200 exits:
if (pmu.errata.instructions_retired_overcount) report(cnt.count < N / 5000, "cntr-%d", i); else report(cnt.count == 1, "cntr-%d", i);
Quoted 3 lines
x86/pmu.c +0
-0else report (cnt.count ==0xffffffffffff || cnt.count <7 ,"cntr-%d" , i);
Quoted 3 lines · Sean Christopherson
Apply the instructions and branches overcount errata to all AMD CPUs, which count VMRUN as a retired branch instruction in guest context. I.e. any asynchronous #VMEXITs during any measurement will result in an overcount.
Applied to kvm-x86 next, thanks!
[1/1] x86/pmu: Treat all AMD CPUs as having instruction/branch overcount errata
https://github.com/kvm-x86/kvm-unit-tests/commit/7999ff1d28b0