Skip to contentSkip to conversation

perf off-cpu: Use the CPU map index for the bpf-output file descriptor

PATCH Applied Ian Rogers 4 participants 13 messages Oct 27 – Nov 3 perf events

lore
Share & tools
lore

Ian Rogers

author reviewer
#0 lore raw

PATCH 1/4 perf off-cpu: Use the CPU map index for the bpf-output file descriptor

When populating the offcpu_output BPF perf event array, the file descriptor of the bpf-output event was looked up using the CPU number rather than the index within the evsel's CPU map. These differ when CPUs are offline or a subset of CPUs is used, leading to the wrong, or an out-of-bounds, file descriptor being used. Index the file descriptor array using the CPU map index.

Fixes 282c195906c7 ("perf record --off-cpu: Preparation of off-cpu BPF program")
---
 tools/perf/util/bpf_off_cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/bpf_off_cpu.c +1 -1
index c4639f6a5776..ff8935688b43 100644
@@ -95,7 +95,7 @@ static void off_cpu_start(void *arg)
 		int cpu_nr = pcpu.cpu;
 
 		err = bpf_map__update_elem(skel->maps.offcpu_output, &cpu_nr, sizeof(int),
-					   xyarray__entry(evsel->core.fd, cpu_nr, 0),
+					   xyarray__entry(evsel->core.fd, i, 0),
 					   sizeof(int), BPF_ANY);
 		if (err) {
 			pr_err("Failed to update perf event map for direct off-cpu dumping\n");
Signature · 1 line
2.56.0.313.g2479322e68-goog
author reviewer
#1 lore raw

PATCH 2/4 perf off-cpu: Write the sample ID for PERF_SAMPLE_ID

OFFCPU_SAMPLE_TYPES allows PERF_SAMPLE_ID, and the sample ID is computed, but off_cpu_write() only wrote it for PERF_SAMPLE_IDENTIFIER. When PERF_SAMPLE_ID was set the synthesized samples were one u64 short, so the remaining fields were misparsed. Write the ID after the time as the sample format requires.

Fixes edc41a1099c2 ("perf record: Enable off-cpu analysis with BPF")
---
 tools/perf/util/bpf_off_cpu.c | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/util/bpf_off_cpu.c +2 -0
index ff8935688b43..16948761cc45 100644
@@ -373,6 +373,8 @@ int off_cpu_write(struct perf_session *session)
 			data.array[n++] = 0;
 		if (sample_type & PERF_SAMPLE_TIME)
 			data.array[n++] = tstamp;
+		if (sample_type & PERF_SAMPLE_ID)
+			data.array[n++] = sid;
 		if (sample_type & PERF_SAMPLE_CPU)
 			data.array[n++] = 0;
 		if (sample_type & PERF_SAMPLE_PERIOD)
Signature · 1 line
2.56.0.313.g2479322e68-goog
#2 lore raw

Re: [PATCH v1 2/4] perf off-cpu: Write the sample ID for PERF_SAMPLE_ID

Sashiko has reviewed this patch and found no issues.
Quoted 8 lines

OFFCPU_SAMPLE_TYPES allows PERF_SAMPLE_ID, and the sample ID is computed, but off_cpu_write() only wrote it for PERF_SAMPLE_IDENTIFIER. When PERF_SAMPLE_ID was set the synthesized samples were one u64 short, so the remaining fields were misparsed. Write the ID after the time as the sample format requires.

Fixes edc41a1099c2 ("perf record: Enable off-cpu analysis with BPF")

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
reviewer Reviewed-by on v1 2/4
#3 lore raw

Re: [PATCH v1 2/4] perf off-cpu: Write the sample ID for PERF_SAMPLE_ID

In reply to Ian Rogers
Quoted 17 lines · Ian Rogers

OFFCPU_SAMPLE_TYPES allows PERF_SAMPLE_ID, and the sample ID is computed, but off_cpu_write() only wrote it for PERF_SAMPLE_IDENTIFIER. When PERF_SAMPLE_ID was set the synthesized samples were one u64 short, so the remaining fields were misparsed. Write the ID after the time as the sample format requires.

Fixes edc41a1099c2 ("perf record: Enable off-cpu analysis with BPF")
---
  tools/perf/util/bpf_off_cpu.c | 2 ++
  1 file changed, 2 insertions(+)
tools/perf/util/bpf_off_cpu.c +0 -0
index ff8935688b43..16948761cc45 100644

tools/perf/util/bpf_off_cpu.c int off_cpu_write(struct perf_session *session) · line 380

Quoted 8 lines
tools/perf/util/bpf_off_cpu.c +2 -0
@@ -373,6 +373,8 @@ int off_cpu_write(struct perf_session *session)
  			data.array[n++] = 0;
  		if (sample_type & PERF_SAMPLE_TIME)
  			data.array[n++] = tstamp;
+		if (sample_type & PERF_SAMPLE_ID)
+			data.array[n++] = sid;
  		if (sample_type & PERF_SAMPLE_CPU)
  			data.array[n++] = 0;
  		if (sample_type & PERF_SAMPLE_PERIOD)
author reviewer
#4 lore raw

PATCH 3/4 perf off-cpu: Fix cgroup_filter map sizing

In reply to Ian Rogers

off_cpu_prepare() assumed that if evlist__first(evlist)->cgrp is set, the number of cgroups is evlist__nr_entries(evlist) - 1 (excluding the dummy bpf-output event). This can under-size cgroup_filter when non-cgroup events precede cgroup events or when no dummy bpf-output event is added.

Count the number of evsels in evlist that have a cgroup set and use that count to size and populate cgroup_filter.

Fixes 685439a7a037 ("perf record: Add cgroup support for off-cpu profiling")
---
 tools/perf/util/bpf_off_cpu.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
tools/perf/util/bpf_off_cpu.c +9 -5
index 16948761cc45..71933a71c8d1 100644
@@ -151,9 +151,10 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target,
 		    struct record_opts *opts)
 {
 	int err, fd, i;
-	int ncpus = 1, ntasks = 1, ncgrps = 1;
+	int ncpus = 1, ntasks = 1, ncgrps = 0;
 	struct strlist *pid_slist = NULL;
 	struct str_node *pos;
+	struct evsel *evsel;
 
 	if (off_cpu_config(evlist) < 0) {
 		pr_err("Failed to config off-cpu BPF event\n");
@@ -208,8 +209,12 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target,
 		skel->rodata->uses_tgid = 1;
 	}
 
-	if (evlist__first(evlist)->cgrp) {
-		ncgrps = evlist__nr_entries(evlist) - 1; /* excluding a dummy */
+	evlist__for_each_entry(evlist, evsel) {
+		if (evsel->cgrp)
+			ncgrps++;
+	}
+
+	if (ncgrps > 0) {
 		bpf_map__set_max_entries(skel->maps.cgroup_filter, ncgrps);
 
 		if (!cgroup_is_v2("perf_event"))
@@ -274,8 +279,7 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target,
 		}
 	}
 
-	if (evlist__first(evlist)->cgrp) {
-		struct evsel *evsel;
+	if (ncgrps > 0) {
 		u8 val = 1;
 
 		fd = bpf_map__fd(skel->maps.cgroup_filter);
Signature · 1 line
2.56.0.313.g2479322e68-goog
#5 lore raw

Re: [PATCH v1 3/4] perf off-cpu: Fix cgroup_filter map sizing

Sashiko has reviewed this patch and found no issues.
Quoted 11 lines

off_cpu_prepare() assumed that if evlist__first(evlist)->cgrp is set, the number of cgroups is evlist__nr_entries(evlist) - 1 (excluding the dummy bpf-output event). This can under-size cgroup_filter when non-cgroup events precede cgroup events or when no dummy bpf-output event is added.

Count the number of evsels in evlist that have a cgroup set and use that count to size and populate cgroup_filter.

Fixes 685439a7a037 ("perf record: Add cgroup support for off-cpu profiling")

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
reviewer Reviewed-by on v1 3/4
#6 lore raw

Re: [PATCH v1 3/4] perf off-cpu: Fix cgroup_filter map sizing

In reply to Ian Rogers
Quoted 47 lines · Ian Rogers

off_cpu_prepare() assumed that if evlist__first(evlist)->cgrp is set, the number of cgroups is evlist__nr_entries(evlist) - 1 (excluding the dummy bpf-output event). This can under-size cgroup_filter when non-cgroup events precede cgroup events or when no dummy bpf-output event is added.

Count the number of evsels in evlist that have a cgroup set and use that count to size and populate cgroup_filter.

Fixes 685439a7a037 ("perf record: Add cgroup support for off-cpu profiling")
---
  tools/perf/util/bpf_off_cpu.c | 14 +++++++++-----
  1 file changed, 9 insertions(+), 5 deletions(-)
tools/perf/util/bpf_off_cpu.c +8 -3
index 16948761cc45..71933a71c8d1 100644
@@ -151,9 +151,10 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target,
  		    struct record_opts *opts)
  {
  	int err, fd, i;
-	int ncpus = 1, ntasks = 1, ncgrps = 1;
+	int ncpus = 1, ntasks = 1, ncgrps = 0;
  	struct strlist *pid_slist = NULL;
  	struct str_node *pos;
+	struct evsel *evsel;
 
  	if (off_cpu_config(evlist) < 0) {
  		pr_err("Failed to config off-cpu BPF event\n");
@@ -208,8 +209,12 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target,
  		skel->rodata->uses_tgid = 1;
  	}
 
-	if (evlist__first(evlist)->cgrp) {
-		ncgrps = evlist__nr_entries(evlist) - 1; /* excluding a dummy */
+	evlist__for_each_entry(evlist, evsel) {
+		if (evsel->cgrp)
+			ncgrps++;
+	}
+
+	if (ncgrps > 0) {
  		bpf_map__set_max_entries(skel->maps.cgroup_filter, ncgrps);
 
  		if (!cgroup_is_v2("perf_event"))

tools/perf/util/bpf_off_cpu.c int off_cpu_prepare(struct evlist *evlist, struct target *target, · line 285

Quoted 9 lines
tools/perf/util/bpf_off_cpu.c +1 -2
@@ -274,8 +279,7 @@ int off_cpu_prepare(struct evlist *evlist, struct target *target,
  		}
  	}
 
-	if (evlist__first(evlist)->cgrp) {
-		struct evsel *evsel;
+	if (ncgrps > 0) {
  		u8 val = 1;
 
  		fd = bpf_map__fd(skel->maps.cgroup_filter);
author reviewer
#7 lore raw

PATCH 4/4 libperf evlist: Recompute CPU map when an evsel goes system wide

In reply to Ian Rogers

perf_evlist__go_system_wide() marks the evsel as system wide and then propagates the evlist's maps to it. __perf_evlist__propagate_maps() only assigns the PMU CPUs to evsel->cpus if the map is empty, but an evsel added after the evlist's maps were created already has the user requested CPUs propagated (e.g. the "any CPU" map of -1 in per-thread or workload mode). Going system wide then leaves the evsel opened on the wrong CPUs.

Drop the existing CPU map before propagating so the evsel is given all of its PMU's CPUs, as is expected for a system wide event.

Fixes 811082e4b668 ("perf parse-events: Support user CPUs mixed with threads/processes")
---
 tools/lib/perf/evlist.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/lib/perf/evlist.c +4 -1
index c1c5b0684eed..46de69aa5a86 100644
@@ -835,7 +835,10 @@ void perf_evlist__go_system_wide(struct perf_evlist *evlist, struct perf_evsel *
 {
 	if (!evsel->system_wide) {
 		evsel->system_wide = true;
-		if (evlist->needs_map_propagation)
+		if (evlist->needs_map_propagation) {
+			perf_cpu_map__put(evsel->cpus);
+			evsel->cpus = NULL;
 			__perf_evlist__propagate_maps(evlist, evsel);
+		}
 	}
 }
Signature · 1 line
2.56.0.313.g2479322e68-goog
#8 lore raw

Re: [PATCH v1 4/4] libperf evlist: Recompute CPU map when an evsel goes system wide

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] libperf evlist: stale CPUs left in evlist->all_cpus by perf_evlist__go_system_wide() --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] libperf evlist: stale CPUs left in evlist->all_cpus by perf_evlist__go_system_wide() --

commit 03b9925431ed73d3588b37d9c3ef20e7a5b73d9c
Author: Ian Rogers <[email protected]>

libperf evlist: Recompute CPU map when an evsel goes system wide

This patch drops the existing evsel->cpus map in perf_evlist__go_system_wide() before calling __perf_evlist__propagate_maps() so that an evsel added after map creation gets its PMU's CPUs instead of retaining the user-requested CPUs.

Quoted 5 lines
tools/lib/perf/evlist.c +0 -0
index 1f210dadd666f..c48425d4933dc 100644

tools/lib/perf/evlist.c void perf_evlist__go_system_wide(struct perf_evlist *evlist, struct perf_evsel * · line 841

Quoted 9 lines
tools/lib/perf/evlist.c +4 -1
@@ -834,7 +834,10 @@ void perf_evlist__go_system_wide(struct perf_evlist *evlist, struct perf_evsel *
 {
 	if (!evsel->system_wide) {
 		evsel->system_wide = true;
-		if (evlist->needs_map_propagation)
+		if (evlist->needs_map_propagation) {
+			perf_cpu_map__put(evsel->cpus);
+			evsel->cpus = NULL;
 			__perf_evlist__propagate_maps(evlist, evsel);
+		}

[Severity: Medium]
Does calling __perf_evlist__propagate_maps() on only this evsel leave stale CPUs in evlist->all_cpus?

When an evsel is added via perf_evlist__add() after the evlist's maps were created in per-thread or workload mode, __perf_evlist__propagate_maps() sets evsel->cpus to the user-requested {-1} map and merges {-1} into evlist->all_cpus:

tools/lib/perf/evlist.c:__perf_evlist__propagate_maps() {
	...
	perf_cpu_map__merge(&evlist->all_cpus, evsel->cpus);
}

When perf_evlist__go_system_wide() subsequently drops evsel->cpus and calls __perf_evlist__propagate_maps(evlist, evsel) again, the PMU CPUs are merged into evlist->all_cpus, but the old -1 entry is never removed. In contrast, perf_evlist__propagate_maps() clears and rebuilds evlist->all_cpus across all evsels:

tools/lib/perf/evlist.c:perf_evlist__propagate_maps() {
	evlist->needs_map_propagation = true;

	/* Clear the all_cpus set which will be merged into during propagation. */
	perf_cpu_map__put(evlist->all_cpus);
	evlist->all_cpus = NULL;
	...
}

If no remaining evsel in the evlist uses CPU -1 (for instance, this evsel is the only event, or all other events are system-wide or have requires_cpu set), evlist->all_cpus retains the stale -1 entry. When perf_evlist__mmap_ops() runs, perf_cpu_map__has_any_cpu_or_is_empty(cpus) then evaluates to true and selects mmap_per_thread() instead of mmap_per_cpu():

tools/lib/perf/evlist.c:perf_evlist__mmap_ops() {
	...
	if (perf_cpu_map__has_any_cpu_or_is_empty(cpus))
		return mmap_per_thread(evlist, ops, mp);

	return mmap_per_cpu(evlist, ops, mp);
}

In mmap_per_thread(), cpu_idx 0 (for CPU -1 in evlist->all_cpus) matches no evsel->cpus, so idx 0 is never mmapped, perf_evlist__set_mmap_first() is skipped, and nr_mmaps is miscounted.

Should perf_evlist__go_system_wide() call perf_evlist__propagate_maps(evlist) to rebuild evlist->all_cpus instead of only re-propagating the single evsel?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
maintainer Applied
#9 lore raw

Re: [PATCH v1 4/4] libperf evlist: Recompute CPU map when an evsel goes system wide

Quoted 72 lines · On Tue, Oct 27, 2026 at 05:15:19PM +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] libperf evlist: stale CPUs left in evlist->all_cpus by perf_evlist__go_system_wide() --

commit 03b9925431ed73d3588b37d9c3ef20e7a5b73d9c
Author: Ian Rogers <[email protected]>

libperf evlist: Recompute CPU map when an evsel goes system wide

This patch drops the existing evsel->cpus map in perf_evlist__go_system_wide() before calling __perf_evlist__propagate_maps() so that an evsel added after map creation gets its PMU's CPUs instead of retaining the user-requested CPUs.

Quoted 5 lines
tools/lib/perf/evlist.c +0 -0
index 1f210dadd666f..c48425d4933dc 100644

tools/lib/perf/evlist.c void perf_evlist__go_system_wide(struct perf_evlist *evlist, struct perf_evsel * · line 841

Quoted 9 lines
tools/lib/perf/evlist.c +4 -1
@@ -834,7 +834,10 @@ void perf_evlist__go_system_wide(struct perf_evlist *evlist, struct perf_evsel *
 {
 	if (!evsel->system_wide) {
 		evsel->system_wide = true;
-		if (evlist->needs_map_propagation)
+		if (evlist->needs_map_propagation) {
+			perf_cpu_map__put(evsel->cpus);
+			evsel->cpus = NULL;
 			__perf_evlist__propagate_maps(evlist, evsel);
+		}

[Severity: Medium]
Does calling __perf_evlist__propagate_maps() on only this evsel leave stale CPUs in evlist->all_cpus?

When an evsel is added via perf_evlist__add() after the evlist's maps were created in per-thread or workload mode, __perf_evlist__propagate_maps() sets evsel->cpus to the user-requested {-1} map and merges {-1} into evlist->all_cpus:

tools/lib/perf/evlist.c:__perf_evlist__propagate_maps() {
	...
	perf_cpu_map__merge(&evlist->all_cpus, evsel->cpus);
}

When perf_evlist__go_system_wide() subsequently drops evsel->cpus and calls __perf_evlist__propagate_maps(evlist, evsel) again, the PMU CPUs are merged into evlist->all_cpus, but the old -1 entry is never removed. In contrast, perf_evlist__propagate_maps() clears and rebuilds evlist->all_cpus across all evsels:

tools/lib/perf/evlist.c:perf_evlist__propagate_maps() {
	evlist->needs_map_propagation = true;

	/* Clear the all_cpus set which will be merged into during propagation. */
	perf_cpu_map__put(evlist->all_cpus);
	evlist->all_cpus = NULL;
	...
}

If no remaining evsel in the evlist uses CPU -1 (for instance, this evsel is the only event, or all other events are system-wide or have requires_cpu set), evlist->all_cpus retains the stale -1 entry. When perf_evlist__mmap_ops() runs, perf_cpu_map__has_any_cpu_or_is_empty(cpus) then evaluates to true and selects mmap_per_thread() instead of mmap_per_cpu():

tools/lib/perf/evlist.c:perf_evlist__mmap_ops() {
	...
	if (perf_cpu_map__has_any_cpu_or_is_empty(cpus))
		return mmap_per_thread(evlist, ops, mp);

	return mmap_per_cpu(evlist, ops, mp);
}
Quoted 6 lines

In mmap_per_thread(), cpu_idx 0 (for CPU -1 in evlist->all_cpus) matches no evsel->cpus, so idx 0 is never mmapped, perf_evlist__set_mmap_first() is skipped, and nr_mmaps is miscounted.

Should perf_evlist__go_system_wide() call perf_evlist__propagate_maps(evlist) to rebuild evlist->all_cpus instead of only re-propagating the single evsel?

Hi Ian, what do you think about this?

I've applied the first 3 patches to perf-tools-next already.

Thanks,
Namhyung

reviewer Reviewed-by on v1 4/4
#10 lore raw

Re: [PATCH v1 4/4] libperf evlist: Recompute CPU map when an evsel goes system wide

In reply to Ian Rogers
Quoted 23 lines · Ian Rogers

perf_evlist__go_system_wide() marks the evsel as system wide and then propagates the evlist's maps to it. __perf_evlist__propagate_maps() only assigns the PMU CPUs to evsel->cpus if the map is empty, but an evsel added after the evlist's maps were created already has the user requested CPUs propagated (e.g. the "any CPU" map of -1 in per-thread or workload mode). Going system wide then leaves the evsel opened on the wrong CPUs.

Drop the existing CPU map before propagating so the evsel is given all of its PMU's CPUs, as is expected for a system wide event.

Fixes 811082e4b668 ("perf parse-events: Support user CPUs mixed with threads/processes")
---
  tools/lib/perf/evlist.c | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
tools/lib/perf/evlist.c +0 -0
index c1c5b0684eed..46de69aa5a86 100644
@@ -835,7 +835,10 @@ void perf_evlist__go_system_wide(struct perf_evlist *evlist, struct perf_evsel *
  {

tools/lib/perf/evlist.c void perf_evlist__go_system_wide(struct perf_evlist *evlist, struct perf_evsel * · line 844

Quoted 10 lines
tools/lib/perf/evlist.c +4 -1
  	if (!evsel->system_wide) {
  		evsel->system_wide = true;
-		if (evlist->needs_map_propagation)
+		if (evlist->needs_map_propagation) {
+			perf_cpu_map__put(evsel->cpus);
+			evsel->cpus = NULL;
  			__perf_evlist__propagate_maps(evlist, evsel);
+		}
  	}
  }
#11 lore raw
In reply to Ian Rogers
Sashiko has reviewed this patch and found no issues.
Quoted 9 lines

When populating the offcpu_output BPF perf event array, the file descriptor of the bpf-output event was looked up using the CPU number rather than the index within the evsel's CPU map. These differ when CPUs are offline or a subset of CPUs is used, leading to the wrong, or an out-of-bounds, file descriptor being used. Index the file descriptor array using the CPU map index.

Fixes 282c195906c7 ("perf record --off-cpu: Preparation of off-cpu BPF program")

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
reviewer Reviewed-by on v1 1/4
#12 lore raw
In reply to Ian Rogers
Quoted 18 lines · Ian Rogers

When populating the offcpu_output BPF perf event array, the file descriptor of the bpf-output event was looked up using the CPU number rather than the index within the evsel's CPU map. These differ when CPUs are offline or a subset of CPUs is used, leading to the wrong, or an out-of-bounds, file descriptor being used. Index the file descriptor array using the CPU map index.

Fixes 282c195906c7 ("perf record --off-cpu: Preparation of off-cpu BPF program")
---
  tools/perf/util/bpf_off_cpu.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/bpf_off_cpu.c +0 -0
index c4639f6a5776..ff8935688b43 100644

tools/perf/util/bpf_off_cpu.c static void off_cpu_start(void *arg) · line 101

Quoted 8 lines
tools/perf/util/bpf_off_cpu.c +1 -1
@@ -95,7 +95,7 @@ static void off_cpu_start(void *arg)
  		int cpu_nr = pcpu.cpu;
 
  		err = bpf_map__update_elem(skel->maps.offcpu_output, &cpu_nr, sizeof(int),
-					   xyarray__entry(evsel->core.fd, cpu_nr, 0),
+					   xyarray__entry(evsel->core.fd, i, 0),
  					   sizeof(int), BPF_ANY);
  		if (err) {
  			pr_err("Failed to update perf event map for direct off-cpu dumping\n");