Skip to contentSkip to conversation

perf/x86/intel: Avoid unnecessary PEBS_ENABLE MSR access in PMI

PATCH RESEND Superseded [email protected] 2 participants 3 messages Jan 16, 2020 – Jan 17, 2020

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH RESEND v2 1/1 perf/x86/intel: Avoid unnecessary PEBS_ENABLE MSR access in PMI

From: Kan Liang <[email protected]>

The perf PMI handler, intel_pmu_handle_irq(), currently does unnecessary MSR accesses for PEBS_ENABLE MSR in __intel_pmu_enable/disable_all() when PEBS is enabled.

When entering the handler, global ctrl is explicitly disabled. All counters do not count anymore. It doesn't matter if PEBS is enabled or not in a PMI handler.
Furthermore, for most cases, the cpuc->pebs_enabled is not changed in PMI. The PEBS status doesn't change. The PEBS_ENABLE MSR doesn't need to be changed either when exiting the handler.

PMI throttle may change the PEBS status during PMI handler. The x86_pmu_stop() ends up in intel_pmu_pebs_disable() which can update cpuc->pebs_enabled. But the PEBS_ENABLE MSR will be updated as well when cpuc->enabled == 1. No need to access PEBS_ENABLE MSR again for this case when exiting the handler.

A PMI may land after cpuc->enabled=0 in x86_pmu_disable() and PMI throttle may be triggered for the PMI. For this rare case, intel_pmu_pebs_disable() will not touch PEBS_ENABLE MSR. The patch explicitly disable the PEBS for this case.

Use ftrace to measure the duration of intel_pmu_handle_irq() on BDX.
#perf record -e cycles:P -- ./tchain_edit

The average duration of intel_pmu_handle_irq()
Without the patch 1.144 us
With the patch 1.025 us

---

Changes since V1:
- Update description and comments
- Handle a rare case. The PMI may land after cpuc->enabled=0 in x86_pmu_disable() and PMI throttle may be triggered for the PMI.

 arch/x86/events/intel/core.c | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)
arch/x86/events/intel/core.c +19 -3
index bc6468329c52..18e1132c0fd7 100644
@@ -1963,6 +1963,14 @@ static __initconst const u64 knl_hw_cache_extra_regs
  * intel_bts events don't coexist with intel PMU's BTS events because of
  * x86_add_exclusive(x86_lbr_exclusive_lbr); there's no need to keep them
  * disabled around intel PMU's event batching etc, only inside the PMI handler.
+ *
+ * Avoid PEBS_ENABLE MSR access in PMIs.
+ * The GLOBAL_CTRL has been disabled. All the counters do not count anymore.
+ * It doesn't matter if the PEBS is enabled or not.
+ * Usually, the PEBS status are not changed in PMIs. It's unnecessary to
+ * access PEBS_ENABLE MSR in disable_all()/enable_all().
+ * However, there are some cases which may change PEBS status, e.g. PMI
+ * throttle. The PEBS_ENABLE should be updated where the status changes.
  */
 static void __intel_pmu_disable_all(void)
 {
@@ -1972,13 +1980,12 @@ static void __intel_pmu_disable_all(void)
 
 	if (test_bit(INTEL_PMC_IDX_FIXED_BTS, cpuc->active_mask))
 		intel_pmu_disable_bts();
-
-	intel_pmu_pebs_disable_all();
 }
 
 static void intel_pmu_disable_all(void)
 {
 	__intel_pmu_disable_all();
+	intel_pmu_pebs_disable_all();
 	intel_pmu_lbr_disable_all();
 }
 
@@ -1986,7 +1993,6 @@ static void __intel_pmu_enable_all(int added, bool pmi)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
 
-	intel_pmu_pebs_enable_all();
 	intel_pmu_lbr_enable_all(pmi);
 	wrmsrl(MSR_CORE_PERF_GLOBAL_CTRL,
 			x86_pmu.intel_ctrl & ~cpuc->intel_ctrl_guest_mask);
@@ -2004,6 +2010,7 @@ static void __intel_pmu_enable_all(int added, bool pmi)
 
 static void intel_pmu_enable_all(int added)
 {
+	intel_pmu_pebs_enable_all();
 	__intel_pmu_enable_all(added, false);
 }
 
@@ -2620,6 +2627,15 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 		handled++;
 		x86_pmu.drain_pebs(regs);
 		status &= x86_pmu.intel_ctrl | GLOBAL_STATUS_TRACE_TOPAPMI;
+
+		/*
+		 * PMI may land after cpuc->enabled=0 in x86_pmu_disable() and
+		 * PMI throttle may be triggered for the PMI.
+		 * For this rare case, intel_pmu_pebs_disable() will not touch
+		 * MSR_IA32_PEBS_ENABLE. Explicitly disable the PEBS here.
+		 */
+		if (unlikely(!cpuc->enabled && !cpuc->pebs_enabled))
+			wrmsrl(MSR_IA32_PEBS_ENABLE, 0);
 	}
 
 	/*
Signature · 1 line
2.17.1
maintainer
#1 lore raw
Quoted 4 lines · On Thu, Jan 16, 2020 at 10:21:12AM -0800, [email protected] wrote:

A PMI may land after cpuc->enabled=0 in x86_pmu_disable() and PMI throttle may be triggered for the PMI. For this rare case, intel_pmu_pebs_disable() will not touch PEBS_ENABLE MSR. The patch explicitly disable the PEBS for this case.

intel_pmu_handle_irq()
pmu_enabled = cpuc->enabled;
cpuc->enabled = 0;
__intel_pmu_disable_all();

...

    x86_pmu_stop()
      intel_pmu_disable_event()
        intel_pmu_pebs_disable()
	  if (cpuc->enabled) // FALSE!!!
  cpuc->enabled = pmu_enabled;
  if (pmu_enabled)
    __intel_pmu_enable_all();
Quoted 4 lines
arch/x86/events/intel/core.c +0 -0
@@ -2620,6 +2627,15 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 		handled++;
 		x86_pmu.drain_pebs(regs);
 		status &= x86_pmu.intel_ctrl | GLOBAL_STATUS_TRACE_TOPAPMI;

arch/x86/events/intel/core.c static int handle_pmi_common(struct pt_regs *regs, u64 status) · line 2639

Quoted 10 lines
arch/x86/events/intel/core.c +9 -0
+
+		/*
+		 * PMI may land after cpuc->enabled=0 in x86_pmu_disable() and
+		 * PMI throttle may be triggered for the PMI.
+		 * For this rare case, intel_pmu_pebs_disable() will not touch
+		 * MSR_IA32_PEBS_ENABLE. Explicitly disable the PEBS here.
+		 */
+		if (unlikely(!cpuc->enabled && !cpuc->pebs_enabled))
+			wrmsrl(MSR_IA32_PEBS_ENABLE, 0);
 	}

How does that make sense? AFAICT this is all still completely broken.

Please be more careful.

author
#2 lore raw
Quoted 8 lines · Peter Zijlstra
Quoted 4 lines · On Thu, Jan 16, 2020 at 10:21:12AM -0800, [email protected] wrote:

A PMI may land after cpuc->enabled=0 in x86_pmu_disable() and PMI throttle may be triggered for the PMI. For this rare case, intel_pmu_pebs_disable() will not touch PEBS_ENABLE MSR. The patch explicitly disable the PEBS for this case.

intel_pmu_handle_irq()

Quoted 9 lines
   pmu_enabled = cpuc->enabled;
   cpuc->enabled = 0;
   __intel_pmu_disable_all();

   ...
     x86_pmu_stop()
       intel_pmu_disable_event()
         intel_pmu_pebs_disable()
	  if (cpuc->enabled) // FALSE!!!

Right, it always be false in PMI.
We force the 'enabled' to 0 when entering the PMI.

For this case, I think we may add a variable to save the pebs_enabled when entering PMI. If it's changed, we have to write the new value. I will prepare the V3.

Thanks,
Kan

Quoted 24 lines
   cpuc->enabled = pmu_enabled;
   if (pmu_enabled)
     __intel_pmu_enable_all();
Quoted 4 lines
arch/x86/events/intel/core.c +0 -0
@@ -2620,6 +2627,15 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
  		handled++;
  		x86_pmu.drain_pebs(regs);
  		status &= x86_pmu.intel_ctrl | GLOBAL_STATUS_TRACE_TOPAPMI;

arch/x86/events/intel/core.c static int handle_pmi_common(struct pt_regs *regs, u64 status) · line 2639

Quoted 10 lines
arch/x86/events/intel/core.c +9 -0
+
+		/*
+		 * PMI may land after cpuc->enabled=0 in x86_pmu_disable() and
+		 * PMI throttle may be triggered for the PMI.
+		 * For this rare case, intel_pmu_pebs_disable() will not touch
+		 * MSR_IA32_PEBS_ENABLE. Explicitly disable the PEBS here.
+		 */
+		if (unlikely(!cpuc->enabled && !cpuc->pebs_enabled))
+			wrmsrl(MSR_IA32_PEBS_ENABLE, 0);
  	}

How does that make sense? AFAICT this is all still completely broken.

Please be more careful.