Skip to contentSkip to conversation

perf/x86: Package residency counter improvements

PATCH Superseded Tero Kristo 1 participant 3 messages Aug 23, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover perf/x86: Package residency counter improvements

Hello,

Following two patches address a couple of different issues with Intel C-state package residency counters, which are used to track power saving state usage on Intel chips. The patches don't have any dependencies to each other and can be applied separately.

1) The residency counters are always read from the first CPU of the package via an SMP call, even if they are available from any CPU. This causes extra latency. Patch #1 fixes this issue by flagging the perf event properly and allowing to execute it on any CPU of the package.
2) The residency counters are completely impossible to read from a BPF program running on any other than the first CPU of the package, as the SMP call is not available in this context. Patch #2 addresses this issue by allowing the read of the perf event from the local CPU, similar to what is done in perf_read_event(). This patch also allows reading any other package scope perf events (RAPL/UNCORE) from arbitrary CPUs on a package via BPF.

-Tero

#1 lore raw

PATCH 1/2 perf/x86/cstate: Allow reading the package statistics from local CPU

The MSR registers for reading the package residency counters are available on every CPU of the package. To avoid doing unnecessary SMP calls to read the values for these from the various CPUs inside a package, allow reading them from any CPU of the package.

---
 arch/x86/events/intel/cstate.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/events/intel/cstate.c +3 -0
index 835862c548cc..5cfe021edc65 100644
@@ -336,6 +336,9 @@ static int cstate_pmu_event_init(struct perf_event *event)
 		cfg = array_index_nospec((unsigned long)cfg, PERF_CSTATE_PKG_EVENT_MAX);
 		if (!(pkg_msr_mask & (1 << cfg)))
 			return -EINVAL;
+
+		event->event_caps |= PERF_EV_CAP_READ_ACTIVE_PKG;
+
 		event->hw.event_base = pkg_msr[cfg].msr;
 		cpu = cpumask_any_and(&cstate_pkg_cpu_mask,
 				      topology_die_cpumask(event->cpu));
Signature · 1 line
2.25.1
#2 lore raw

PATCH 2/2 perf/core: Allow reading package events from perf_event_read_local

In reply to Tero Kristo

Per-package perf events are typically registered with a single CPU only, however they can be read across all the CPUs within the package. Currently perf_event_read maps the event CPU according to the topology information to avoid an unnecessary SMP call, however perf_event_read_local deals with hard values and rejects a read with a failure if the CPU is not the one exactly registered. Allow similar mapping within the perf_event_read_local if the perf event in question can support this.

This allows users like BPF code to read the package perf events properly across different CPUs within a package.

---
 kernel/events/core.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
kernel/events/core.c +8 -3
index 78ae7b6f90fd..37db7c003b79 100644
@@ -4528,6 +4528,7 @@ int perf_event_read_local(struct perf_event *event, u64 *value,
 {
 	unsigned long flags;
 	int ret = 0;
+	int event_cpu;
 
 	/*
 	 * Disabling interrupts avoids all counter scheduling (context
@@ -4551,15 +4552,19 @@ int perf_event_read_local(struct perf_event *event, u64 *value,
 		goto out;
 	}
 
+	/* Allow reading a per-package perf-event from local CPU also */
+	event_cpu = READ_ONCE(event->oncpu);
+	event_cpu = __perf_event_read_cpu(event, event_cpu);
+
 	/* If this is a per-CPU event, it must be for this CPU */
 	if (!(event->attach_state & PERF_ATTACH_TASK) &&
-	    event->cpu != smp_processor_id()) {
+	    event_cpu != smp_processor_id()) {
 		ret = -EINVAL;
 		goto out;
 	}
 
 	/* If this is a pinned event it must be running on this CPU */
-	if (event->attr.pinned && event->oncpu != smp_processor_id()) {
+	if (event->attr.pinned && event_cpu != smp_processor_id()) {
 		ret = -EBUSY;
 		goto out;
 	}
@@ -4569,7 +4574,7 @@ int perf_event_read_local(struct perf_event *event, u64 *value,
 	 * or local to this CPU. Furthermore it means its ACTIVE (otherwise
 	 * oncpu == -1).
 	 */
-	if (event->oncpu == smp_processor_id())
+	if (event_cpu == smp_processor_id())
 		event->pmu->read(event);
 
 	*value = local64_read(&event->count);
Signature · 1 line
2.25.1