Skip to contentSkip to conversation

Update perf tools topdown documentation

PATCH In review Andi Kleen 5 participants 8 messages Dec 10, 2024 – Dec 11, 2024 perf events

lore
Share & tools
lore

Andi Kleen

author
#0 lore raw

PATCH 1/1 Update perf tools topdown documentation

- Document and give examples for the Lunar Lake perf metrics reset mode.
- Fix the error handling for mmap (it returns -1 on error, not 0)
- Clarify the slots placement documentation. It is not Icelake specific and also applies for non sampling.

---
 tools/perf/Documentation/topdown.txt | 33 +++++++++++++++++++++++-----
 1 file changed, 28 insertions(+), 5 deletions(-)
tools/perf/Documentation/topdown.txt +28 -5
index 5c17fff694ee..3e2340f3f78e 100644
@@ -87,7 +87,7 @@ if (slots_fd < 0)
 
 /* Memory mapping the fd permits _rdpmc calls from userspace */
 void *slots_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, slots_fd, 0);
-if (!slot_p)
+if (slot_p == (void*)-1L)
 	.... error ...
 
 /*
@@ -107,7 +107,7 @@ if (metrics_fd < 0)
 
 /* Memory mapping the fd permits _rdpmc calls from userspace */
 void *metrics_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, metrics_fd, 0);
-if (!metrics_p)
+if (metrics_p == (void*)-1L)
 	... error ...
 
 Note: the file descriptors returned by the perf_event_open calls must be memory
@@ -290,15 +290,38 @@ This "opens" a new measurement period.
 A program using RDPMC for TopDown should schedule such a reset
 regularly, as in every few seconds.
 
-Limits on Intel Ice Lake
-========================
+Newer Intel CPUs (Lunar Lake, Arrow Lake+) support resetting the perf
+metrics automatically RDPMC, which can avoid a system call or needing
+to schedule special resets.
+
+This is available if /sys/devices/cpu*/cap/metrics_clear
+exists, and requires setting an opt-in bit when opening the 
+slots counter:
+
+if (access("/sys/devices/cpu/cap/metrics_clear") &&
+    access("/sys/devices/cpu_core/cap/metrics_clear"))
+     ... functionality not supported ...
+
+#define INTEL_TD_CFG_METRIC_CLEAR 1ULL
+
+struct perf_event_attr slots_event = {
+	... same as slots example above ...
+	.config1 = INTEL_TD_CFG_METRIC_CLEAR,
+};
+
+... open and map counter same as example above ...
+
+Then any metric read will reset the metrics and slots.
+
+Using perf metrics with perf stat
+=================================
 
 Four pseudo TopDown metric events are exposed for the end-users,
 topdown-retiring, topdown-bad-spec, topdown-fe-bound and topdown-be-bound.
 They can be used to collect the TopDown value under the following
 rules:
 - All the TopDown metric events must be in a group with the SLOTS event.
-- The SLOTS event must be the leader of the group.
+- The SLOTS event must be the first entry of the group.
 - The PERF_FORMAT_GROUP flag must be applied for each TopDown metric
   events
 
Signature · 1 line
2.47.1
reviewer Reviewed-by on v1 1/1
#1 lore raw
Quoted 33 lines · Andi Kleen

- Document and give examples for the Lunar Lake perf metrics reset mode.
- Fix the error handling for mmap (it returns -1 on error, not 0)
- Clarify the slots placement documentation. It is not Icelake specific and also applies for non sampling.

---
  tools/perf/Documentation/topdown.txt | 33 +++++++++++++++++++++++-----
  1 file changed, 28 insertions(+), 5 deletions(-)
tools/perf/Documentation/topdown.txt +2 -2
index 5c17fff694ee..3e2340f3f78e 100644
@@ -87,7 +87,7 @@ if (slots_fd < 0)
 
  /* Memory mapping the fd permits _rdpmc calls from userspace */
  void *slots_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, slots_fd, 0);
-if (!slot_p)
+if (slot_p == (void*)-1L)
  	.... error ...
 
  /*
@@ -107,7 +107,7 @@ if (metrics_fd < 0)
 
  /* Memory mapping the fd permits _rdpmc calls from userspace */
  void *metrics_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, metrics_fd, 0);
-if (!metrics_p)
+if (metrics_p == (void*)-1L)
  	... error ...
 
  Note: the file descriptors returned by the perf_event_open calls must be memory

tools/perf/Documentation/topdown.txt This "opens" a new measurement period. · line 294

Quoted 7 lines
tools/perf/Documentation/topdown.txt +2 -2
@@ -290,15 +290,38 @@ This "opens" a new measurement period.
  A program using RDPMC for TopDown should schedule such a reset
  regularly, as in every few seconds.
 
-Limits on Intel Ice Lake
-========================
+Newer Intel CPUs (Lunar Lake, Arrow Lake+) support resetting the perf
+metrics automatically RDPMC, which can avoid a system call or needing

Minor nit: "automatically using RDPMC"?

Quoted 34 lines
tools/perf/Documentation/topdown.txt +24 -1
+to schedule special resets.
+
+This is available if /sys/devices/cpu*/cap/metrics_clear
+exists, and requires setting an opt-in bit when opening the
+slots counter:
+
+if (access("/sys/devices/cpu/cap/metrics_clear") &&
+    access("/sys/devices/cpu_core/cap/metrics_clear"))
+     ... functionality not supported ...
+
+#define INTEL_TD_CFG_METRIC_CLEAR 1ULL
+
+struct perf_event_attr slots_event = {
+	... same as slots example above ...
+	.config1 = INTEL_TD_CFG_METRIC_CLEAR,
+};
+
+... open and map counter same as example above ...
+
+Then any metric read will reset the metrics and slots.
+
+Using perf metrics with perf stat
+=================================
 
  Four pseudo TopDown metric events are exposed for the end-users,
  topdown-retiring, topdown-bad-spec, topdown-fe-bound and topdown-be-bound.
  They can be used to collect the TopDown value under the following
  rules:
  - All the TopDown metric events must be in a group with the SLOTS event.
-- The SLOTS event must be the leader of the group.
+- The SLOTS event must be the first entry of the group.
  - The PERF_FORMAT_GROUP flag must be applied for each TopDown metric
    events
 
#6 lore raw
In reply to Andi Kleen
Quoted 33 lines · Andi Kleen

- Document and give examples for the Lunar Lake perf metrics reset mode.
- Fix the error handling for mmap (it returns -1 on error, not 0)
- Clarify the slots placement documentation. It is not Icelake specific and also applies for non sampling.

---
 tools/perf/Documentation/topdown.txt | 33 +++++++++++++++++++++++-----
 1 file changed, 28 insertions(+), 5 deletions(-)
tools/perf/Documentation/topdown.txt +2 -2
index 5c17fff694ee..3e2340f3f78e 100644
@@ -87,7 +87,7 @@ if (slots_fd < 0)
 
 /* Memory mapping the fd permits _rdpmc calls from userspace */
 void *slots_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, slots_fd, 0);
-if (!slot_p)
+if (slot_p == (void*)-1L)
 	.... error ...
 
 /*
@@ -107,7 +107,7 @@ if (metrics_fd < 0)
 
 /* Memory mapping the fd permits _rdpmc calls from userspace */
 void *metrics_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, metrics_fd, 0);
-if (!metrics_p)
+if (metrics_p == (void*)-1L)
 	... error ...
 
 Note: the file descriptors returned by the perf_event_open calls must be memory

tools/perf/Documentation/topdown.txt This "opens" a new measurement period. · line 297

Quoted 10 lines
tools/perf/Documentation/topdown.txt +5 -2
@@ -290,15 +290,38 @@ This "opens" a new measurement period.
 A program using RDPMC for TopDown should schedule such a reset
 regularly, as in every few seconds.
 
-Limits on Intel Ice Lake
-========================
+Newer Intel CPUs (Lunar Lake, Arrow Lake+) support resetting the perf
+metrics automatically RDPMC, which can avoid a system call or needing
+to schedule special resets.
+
+This is available if /sys/devices/cpu*/cap/metrics_clear

It's under the "format/metrics_clear", not "cap/metrics_clear"

https://lore.kernel.org/lkml/[email protected]/

Since the kernel patch hasn't been merged yet, I will add this document patch to the kernel patch series. I will re-post the whole series soon.

Thanks,
Kan

Quoted 31 lines
tools/perf/Documentation/topdown.txt +21 -1
+exists, and requires setting an opt-in bit when opening the 
+slots counter:
+
+if (access("/sys/devices/cpu/cap/metrics_clear") &&
+    access("/sys/devices/cpu_core/cap/metrics_clear"))
+     ... functionality not supported ...
+
+#define INTEL_TD_CFG_METRIC_CLEAR 1ULL
+
+struct perf_event_attr slots_event = {
+	... same as slots example above ...
+	.config1 = INTEL_TD_CFG_METRIC_CLEAR,
+};
+
+... open and map counter same as example above ...
+
+Then any metric read will reset the metrics and slots.
+
+Using perf metrics with perf stat
+=================================
 
 Four pseudo TopDown metric events are exposed for the end-users,
 topdown-retiring, topdown-bad-spec, topdown-fe-bound and topdown-be-bound.
 They can be used to collect the TopDown value under the following
 rules:
 - All the TopDown metric events must be in a group with the SLOTS event.
-- The SLOTS event must be the leader of the group.
+- The SLOTS event must be the first entry of the group.
 - The PERF_FORMAT_GROUP flag must be applied for each TopDown metric
   events
 
#2 lore raw
In reply to James Clark
Quoted 36 lines · James Clark
Quoted 33 lines · Andi Kleen

- Document and give examples for the Lunar Lake perf metrics reset mode.
- Fix the error handling for mmap (it returns -1 on error, not 0)
- Clarify the slots placement documentation. It is not Icelake specific and also applies for non sampling.

---
  tools/perf/Documentation/topdown.txt | 33 +++++++++++++++++++++++-----

  1 file changed, 28 insertions(+), 5 deletions(-)
tools/perf/ renamed from tools/perf/Documentation/topdown.txt +0 -0

Documentation/topdown.txt
index 5c17fff694ee..3e2340f3f78e 100644

tools/perf/Documentation/topdown.txt +1 -1
@@ -87,7 +87,7 @@ if (slots_fd < 0)
     /* Memory mapping the fd permits _rdpmc calls from userspace */
   void *slots_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED,
 slots_fd, 0);
-if (!slot_p)
+if (slot_p == (void*)-1L)

      .... error ...
    /*

tools/perf/Documentation/topdown.txt +1 -1
@@ -107,7 +107,7 @@ if (metrics_fd < 0)
     /* Memory mapping the fd permits _rdpmc calls from userspace */
   void *metrics_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED,
 metrics_fd, 0);
-if (!metrics_p)
+if (metrics_p == (void*)-1L)

      ... error ...
    Note: the file descriptors returned by the perf_event_open calls must be memory

tools/perf/Documentation/topdown.txt +0 -0
@@ -290,15 +290,38 @@ This "opens" a new measurement period.
Quoted 8 lines

tools/perf/Documentation/topdown.txt This "opens" a new measurement period. · line 294

Quoted 6 lines

  A program using RDPMC for TopDown should schedule such a reset regularly, as in every few seconds.
  -Limits on Intel Ice Lake
-========================

tools/perf/Documentation/topdown.txt +2 -0
+Newer Intel CPUs (Lunar Lake, Arrow Lake+) support resetting the perf
+metrics automatically RDPMC, which can avoid a system call or needing

Minor nit: "automatically using RDPMC"?

The RDPMC can automatically reset the perf metrics. Maybe change it to "automatically resetting the perf metrics RDPMC"

Thanks,
Kan

Quoted 39 lines
Quoted 35 lines
Quoted 1 line

+to schedule special resets.

tools/perf/Documentation/topdown.txt +22 -0
+
+This is available if /sys/devices/cpu*/cap/metrics_clear
+exists, and requires setting an opt-in bit when opening the
+slots counter:
+
+if (access("/sys/devices/cpu/cap/metrics_clear") &&
+    access("/sys/devices/cpu_core/cap/metrics_clear"))
+     ... functionality not supported ...
+
+#define INTEL_TD_CFG_METRIC_CLEAR 1ULL
+
+struct perf_event_attr slots_event = {
+    ... same as slots example above ...
+    .config1 = INTEL_TD_CFG_METRIC_CLEAR,
+};
+
+... open and map counter same as example above ...
+
+Then any metric read will reset the metrics and slots.
+
+Using perf metrics with perf stat
+=================================

    Four pseudo TopDown metric events are exposed for the end-users, topdown-retiring, topdown-bad-spec, topdown-fe-bound and topdown-be- bound.
  They can be used to collect the TopDown value under the following rules:
  - All the TopDown metric events must be in a group with the SLOTS event.
-- The SLOTS event must be the leader of the group.
+- The SLOTS event must be the first entry of the group.
  - The PERF_FORMAT_GROUP flag must be applied for each TopDown metric events

#3 lore raw
Quoted 48 lines · Kan
Quoted 36 lines · James Clark
Quoted 33 lines · Andi Kleen

- Document and give examples for the Lunar Lake perf metrics reset mode.
- Fix the error handling for mmap (it returns -1 on error, not 0)
- Clarify the slots placement documentation. It is not Icelake specific and also applies for non sampling.

---
  tools/perf/Documentation/topdown.txt | 33 +++++++++++++++++++++++-----

  1 file changed, 28 insertions(+), 5 deletions(-)
tools/perf/ renamed from tools/perf/Documentation/topdown.txt +0 -0

Documentation/topdown.txt
index 5c17fff694ee..3e2340f3f78e 100644

tools/perf/Documentation/topdown.txt +1 -1
@@ -87,7 +87,7 @@ if (slots_fd < 0)
     /* Memory mapping the fd permits _rdpmc calls from userspace */
   void *slots_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED,
 slots_fd, 0);
-if (!slot_p)
+if (slot_p == (void*)-1L)

      .... error ...
    /*

tools/perf/Documentation/topdown.txt +1 -1
@@ -107,7 +107,7 @@ if (metrics_fd < 0)
     /* Memory mapping the fd permits _rdpmc calls from userspace */
   void *metrics_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED,
 metrics_fd, 0);
-if (!metrics_p)
+if (metrics_p == (void*)-1L)

      ... error ...
    Note: the file descriptors returned by the perf_event_open calls must be memory

tools/perf/Documentation/topdown.txt +0 -0
@@ -290,15 +290,38 @@ This "opens" a new measurement period.
Quoted 8 lines

tools/perf/Documentation/topdown.txt This "opens" a new measurement period. · line 294

Quoted 6 lines

  A program using RDPMC for TopDown should schedule such a reset regularly, as in every few seconds.
  -Limits on Intel Ice Lake
-========================

tools/perf/Documentation/topdown.txt +2 -0
+Newer Intel CPUs (Lunar Lake, Arrow Lake+) support resetting the perf
+metrics automatically RDPMC, which can avoid a system call or needing

Minor nit: "automatically using RDPMC"?

Quoted 3 lines

The RDPMC can automatically reset the perf metrics. Maybe change it to "automatically resetting the perf metrics RDPMC"

The new post can be found at
https://lore.kernel.org/lkml/[email protected]/

I also added the reviewed-by in the new patch.

Please let me know if there are more comments.

Thanks,
Kan

Quoted 43 lines

Thanks,
Kan

Quoted 39 lines
Quoted 35 lines
Quoted 1 line

+to schedule special resets.

tools/perf/Documentation/topdown.txt +22 -0
+
+This is available if /sys/devices/cpu*/cap/metrics_clear
+exists, and requires setting an opt-in bit when opening the
+slots counter:
+
+if (access("/sys/devices/cpu/cap/metrics_clear") &&
+    access("/sys/devices/cpu_core/cap/metrics_clear"))
+     ... functionality not supported ...
+
+#define INTEL_TD_CFG_METRIC_CLEAR 1ULL
+
+struct perf_event_attr slots_event = {
+    ... same as slots example above ...
+    .config1 = INTEL_TD_CFG_METRIC_CLEAR,
+};
+
+... open and map counter same as example above ...
+
+Then any metric read will reset the metrics and slots.
+
+Using perf metrics with perf stat
+=================================

    Four pseudo TopDown metric events are exposed for the end-users, topdown-retiring, topdown-bad-spec, topdown-fe-bound and topdown-be- bound.
  They can be used to collect the TopDown value under the following rules:
  - All the TopDown metric events must be in a group with the SLOTS event.
-- The SLOTS event must be the leader of the group.
+- The SLOTS event must be the first entry of the group.
  - The PERF_FORMAT_GROUP flag must be applied for each TopDown metric events

maintainer
#7 lore raw
In reply to Andi Kleen

Hello,

Quoted 15 lines · Andi Kleen

- Document and give examples for the Lunar Lake perf metrics reset mode.
- Fix the error handling for mmap (it returns -1 on error, not 0)
- Clarify the slots placement documentation. It is not Icelake specific and also applies for non sampling.

---
 tools/perf/Documentation/topdown.txt | 33 +++++++++++++++++++++++-----
 1 file changed, 28 insertions(+), 5 deletions(-)
tools/perf/Documentation/topdown.txt +0 -0
index 5c17fff694ee..3e2340f3f78e 100644

tools/perf/Documentation/topdown.txt if (slots_fd < 0) · line 90

Quoted 5 lines
tools/perf/Documentation/topdown.txt +1 -1
@@ -87,7 +87,7 @@ if (slots_fd < 0)
 
 /* Memory mapping the fd permits _rdpmc calls from userspace */
 void *slots_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, slots_fd, 0);
-if (!slot_p)
+if (slot_p == (void*)-1L)

We can check with MAP_FAILED.

Thanks,
Namhyung

Quoted 57 lines
tools/perf/Documentation/topdown.txt +27 -4
 	.... error ...
 
 /*
@@ -107,7 +107,7 @@ if (metrics_fd < 0)
 
 /* Memory mapping the fd permits _rdpmc calls from userspace */
 void *metrics_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, metrics_fd, 0);
-if (!metrics_p)
+if (metrics_p == (void*)-1L)
 	... error ...
 
 Note: the file descriptors returned by the perf_event_open calls must be memory
@@ -290,15 +290,38 @@ This "opens" a new measurement period.
 A program using RDPMC for TopDown should schedule such a reset
 regularly, as in every few seconds.
 
-Limits on Intel Ice Lake
-========================
+Newer Intel CPUs (Lunar Lake, Arrow Lake+) support resetting the perf
+metrics automatically RDPMC, which can avoid a system call or needing
+to schedule special resets.
+
+This is available if /sys/devices/cpu*/cap/metrics_clear
+exists, and requires setting an opt-in bit when opening the 
+slots counter:
+
+if (access("/sys/devices/cpu/cap/metrics_clear") &&
+    access("/sys/devices/cpu_core/cap/metrics_clear"))
+     ... functionality not supported ...
+
+#define INTEL_TD_CFG_METRIC_CLEAR 1ULL
+
+struct perf_event_attr slots_event = {
+	... same as slots example above ...
+	.config1 = INTEL_TD_CFG_METRIC_CLEAR,
+};
+
+... open and map counter same as example above ...
+
+Then any metric read will reset the metrics and slots.
+
+Using perf metrics with perf stat
+=================================
 
 Four pseudo TopDown metric events are exposed for the end-users,
 topdown-retiring, topdown-bad-spec, topdown-fe-bound and topdown-be-bound.
 They can be used to collect the TopDown value under the following
 rules:
 - All the TopDown metric events must be in a group with the SLOTS event.
-- The SLOTS event must be the leader of the group.
+- The SLOTS event must be the first entry of the group.
 - The PERF_FORMAT_GROUP flag must be applied for each TopDown metric
   events
 
Signature · 1 line
2.47.1
reviewer
#4 lore raw
In reply to Liang, Kan
Quoted 10 lines · Kan
Quoted 3 lines

The RDPMC can automatically reset the perf metrics. Maybe change it to "automatically resetting the perf metrics RDPMC"

The new post can be found at
https://lore.kernel.org/lkml/[email protected]/

I also added the reviewed-by in the new patch.

Please let me know if there are more comments.

Hi Kan,

Vince Weaver posted to the list a related question on topdown + rdpmc + hybrid:
https://lore.kernel.org/linux-perf-users/[email protected]/
The topic of being rescheduled isn't part of the rdpmc documentation:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/include/uapi/linux/perf_event.h?h=perf-tools-next#n610
In the testing we're only testing per-task mode and not per-CPU:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/lib/perf/tests/test-evsel.c?h=perf-tools-next#n145
It isn't clear to me how per-CPU would work given rescheduling, you could read a counter that doesn't match the mmap page unless you pin everything to just 1 CPU. Given the issues with rdpmc, perhaps it is safer in the documentation to cover cases without it.

Thanks,
Ian

#5 lore raw
Quoted 10 lines · Ian Rogers
Quoted 9 lines · Kan
Quoted 3 lines

The RDPMC can automatically reset the perf metrics. Maybe change it to "automatically resetting the perf metrics RDPMC"

The new post can be found at
https://lore.kernel.org/lkml/[email protected]/

I also added the reviewed-by in the new patch.

Quoted 8 lines
Quoted 1 line

Please let me know if there are more comments.

Hi Kan,

Vince Weaver posted to the list a related question on topdown + rdpmc + hybrid:
https://lore.kernel.org/linux-perf-users/[email protected]/
The topic of being rescheduled isn't part of the rdpmc documentation:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/include/uapi/linux/perf_event.h?h=perf-tools-next#n610

Maybe we should emphasis that the example in the documentation is only for p-core.

(quoted diff) +0 -0

b/tools/perf/Documentation/topdown.txt
index 44644e917ffd..a229d0043399 100644

tools/perf/Documentation/topdown.txt +2 -2
@@ -21,14 +21,14 @@ per architecture.
      5.704818971                 14.0                 27.5
    51.3                    7.3
 ...
 
-New Topdown features in Intel Ice Lake
+New Topdown features in Intel Ice Lake and later p-core
 ======================================
 
 With Ice Lake CPUs the TopDown metrics are directly available as
 fixed counters and do not require generic counters. This allows
 to collect TopDown always in addition to other events.
 
-Using TopDown through RDPMC in applications on Intel Ice Lake
+Using TopDown through RDPMC in applications on Intel Ice Lake and later

p-core
=============================================================

For more fine grained measurements it can be useful to

Quoted 2 lines

In the testing we're only testing per-task mode and not per-CPU:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/lib/perf/tests/test-evsel.c?h=perf-tools-next#n145

The event type is hard code to PERF_TYPE_HARDWARE, which is available for both core. It doesn't matter which core the event is scheduled to.

If you plan to add a topdown event, the PERF_TYPE_RAW should be used, which is the type for the p-core. If the app is scheduled on an e-core, the event is disabled.

Quoted 4 lines

It isn't clear to me how per-CPU would work given rescheduling, you could read a counter that doesn't match the mmap page unless you pin everything to just 1 CPU. Given the issues with rdpmc, perhaps it is safer in the documentation to cover cases without it.

As long as the scope of the RDPMC is clearly defined, I think it's OK to add the usage in the documentation.

Thanks,
Kan