Skip to contentSkip to conversation

perf record: Use a pinned BPF program for filter

PATCH In review Namhyung Kim 2 participants 21 messages Jul 3, 2024 – Aug 2, 2024

Subsystems (2)
lore
Share & tools
lore

Namhyung Kim

author maintainer
#0 lore raw

v3 cover perf record: Use a pinned BPF program for filter

Hello,

This is to support the unprivileged BPF filter for profiling per-task events. Until now only root (or any user with CAP_BPF) can use the filter and we cannot add a new unprivileged BPF program types. After talking with the BPF folks at LSF/MM/BPF 2024, I was told that this is the way to go. Finally I managed to make it working with pinned BPF objects. :)

v3 changes)
* rebased onto latest perf-tools-next

v2 changes)
* rebased onto Ian's UID/GID (non-sample data based) filter term change
* support separate lost counts for each use case
* update the test case to allow normal users (if supported)

This only supports the per-task mode for normal users and root still uses its own instance of the same BPF program - not shared with other users. But it requires the one-time setup (by root) before using it by normal users like below.

  $ sudo perf record --setup-filter pin

This will load the BPF program and maps and pin them in the BPF-fs. Then normal users can use the filter.

  $ perf record -o- -e cycles:u --filter 'period < 10000' perf test -w noploop | perf script -i-
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.011 MB - ]
        perf  759982 448227.214189:          1 cycles:u:      7f153719f4d0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
        perf  759982 448227.214195:          1 cycles:u:      7f153719f4d0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
        perf  759982 448227.214196:          7 cycles:u:      7f153719f4d0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
        perf  759982 448227.214196:        223 cycles:u:      7f153719f4d0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
        perf  759982 448227.214198:       9475 cycles:u:  ffffffff8ee012a0 [unknown] ([unknown])
        perf  759982 448227.548608:          1 cycles:u:      559a9f03c81c noploop+0x5c (/home/namhyung/linux/tools/perf/perf)
        perf  759982 448227.548611:          1 cycles:u:      559a9f03c81c noploop+0x5c (/home/namhyung/linux/tools/perf/perf)
        perf  759982 448227.548612:         12 cycles:u:      559a9f03c81c noploop+0x5c (/home/namhyung/linux/tools/perf/perf)
        perf  759982 448227.548613:        466 cycles:u:      559a9f03c81c noploop+0x5c (/home/namhyung/linux/tools/perf/perf)

It's also possible to unload (and unpin, of course) using this command:

  $ sudo perf record --setup-filter unpin

The code is avaiable in 'perf/pinned-filter-v3' branch at

git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git

Thanks,
Namhyung

Namhyung Kim (8):
perf bpf-filter: Make filters map a single entry hashmap perf bpf-filter: Pass 'target' to perf_bpf_filter__prepare() perf bpf-filter: Split per-task filter use case perf bpf-filter: Support pin/unpin BPF object
perf bpf-filter: Support separate lost counts for each filter perf record: Fix a potential error handling issue perf record: Add --setup-filter option
perf test: Update sample filtering test

 tools/perf/Documentation/perf-record.txt     |   5 +
 tools/perf/builtin-record.c                  |  23 +-
 tools/perf/builtin-stat.c                    |   2 +-
 tools/perf/builtin-top.c                     |   2 +-
 tools/perf/builtin-trace.c                   |   2 +-
 tools/perf/tests/shell/record_bpf_filter.sh  |  13 +-
 tools/perf/util/bpf-filter.c                 | 406 +++++++++++++++++--
 tools/perf/util/bpf-filter.h                 |  19 +-
 tools/perf/util/bpf_skel/sample-filter.h     |   2 +
 tools/perf/util/bpf_skel/sample_filter.bpf.c |  75 +++-
 tools/perf/util/evlist.c                     |   5 +-
 tools/perf/util/evlist.h                     |   4 +-
 12 files changed, 483 insertions(+), 75 deletions(-)
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
author maintainer
#1 lore raw

PATCH v3 1/8 perf bpf-filter: Make filters map a single entry hashmap

And the value is now an array. This is to support multiple filter entries in the map later.

No functional changes intended.

---
 tools/perf/util/bpf-filter.c                 | 81 ++++++++++++++------
 tools/perf/util/bpf_skel/sample-filter.h     |  1 +
 tools/perf/util/bpf_skel/sample_filter.bpf.c | 39 +++++-----
 3 files changed, 78 insertions(+), 43 deletions(-)
tools/perf/util/bpf-filter.c +56 -25
index 04f98b6bb291..2510832d83f9 100644
@@ -93,71 +93,102 @@ static int check_sample_flags(struct evsel *evsel, struct perf_bpf_filter_expr *
 
 int perf_bpf_filter__prepare(struct evsel *evsel)
 {
-	int i, x, y, fd;
+	int i, x, y, fd, ret;
 	struct sample_filter_bpf *skel;
 	struct bpf_program *prog;
 	struct bpf_link *link;
 	struct perf_bpf_filter_expr *expr;
+	struct perf_bpf_filter_entry *entry;
+
+	entry = calloc(MAX_FILTERS, sizeof(*entry));
+	if (entry == NULL)
+		return -1;
 
 	skel = sample_filter_bpf__open_and_load();
 	if (!skel) {
 		pr_err("Failed to load perf sample-filter BPF skeleton\n");
-		return -1;
+		ret = -EPERM;
+		goto err;
 	}
 
 	i = 0;
 	fd = bpf_map__fd(skel->maps.filters);
 	list_for_each_entry(expr, &evsel->bpf_filters, list) {
-		struct perf_bpf_filter_entry entry = {
-			.op = expr->op,
-			.part = expr->part,
-			.term = expr->term,
-			.value = expr->val,
-		};
+		if (check_sample_flags(evsel, expr) < 0) {
+			ret = -EINVAL;
+			goto err;
+		}
 
-		if (check_sample_flags(evsel, expr) < 0)
-			return -1;
+		if (i == MAX_FILTERS) {
+			ret = -E2BIG;
+			goto err;
+		}
 
-		bpf_map_update_elem(fd, &i, &entry, BPF_ANY);
+		entry[i].op = expr->op;
+		entry[i].part = expr->part;
+		entry[i].term = expr->term;
+		entry[i].value = expr->val;
 		i++;
 
 		if (expr->op == PBF_OP_GROUP_BEGIN) {
 			struct perf_bpf_filter_expr *group;
 
 			list_for_each_entry(group, &expr->groups, list) {
-				struct perf_bpf_filter_entry group_entry = {
-					.op = group->op,
-					.part = group->part,
-					.term = group->term,
-					.value = group->val,
-				};
-				bpf_map_update_elem(fd, &i, &group_entry, BPF_ANY);
+				if (i == MAX_FILTERS) {
+					ret = -E2BIG;
+					goto err;
+				}
+
+				entry[i].op = group->op;
+				entry[i].part = group->part;
+				entry[i].term = group->term;
+				entry[i].value = group->val;
 				i++;
 			}
 
-			memset(&entry, 0, sizeof(entry));
-			entry.op = PBF_OP_GROUP_END;
-			bpf_map_update_elem(fd, &i, &entry, BPF_ANY);
+			if (i == MAX_FILTERS) {
+				ret = -E2BIG;
+				goto err;
+			}
+
+			entry[i].op = PBF_OP_GROUP_END;
 			i++;
 		}
 	}
 
-	if (i > MAX_FILTERS) {
-		pr_err("Too many filters: %d (max = %d)\n", i, MAX_FILTERS);
-		return -1;
+	if (i < MAX_FILTERS) {
+		/* to terminate the loop early */
+		entry[i].op = PBF_OP_DONE;
+		i++;
+	}
+
+	/* The filters map has only one entry for now */
+	i = 0;
+	if (bpf_map_update_elem(fd, &i, entry, BPF_ANY) < 0) {
+		ret = -errno;
+		pr_err("Failed to update the filter map\n");
+		goto err;
 	}
+
 	prog = skel->progs.perf_sample_filter;
 	for (x = 0; x < xyarray__max_x(evsel->core.fd); x++) {
 		for (y = 0; y < xyarray__max_y(evsel->core.fd); y++) {
 			link = bpf_program__attach_perf_event(prog, FD(evsel, x, y));
 			if (IS_ERR(link)) {
 				pr_err("Failed to attach perf sample-filter program\n");
-				return PTR_ERR(link);
+				ret = PTR_ERR(link);
+				goto err;
 			}
 		}
 	}
+	free(entry);
 	evsel->bpf_skel = skel;
 	return 0;
+
+err:
+	free(entry);
+	sample_filter_bpf__destroy(skel);
+	return ret;
 }
 
 int perf_bpf_filter__destroy(struct evsel *evsel)
tools/perf/util/bpf_skel/sample-filter.h +1 -0
index 350efa121026..bb6a1b91f1df 100644
@@ -14,6 +14,7 @@ enum perf_bpf_filter_op {
 	PBF_OP_AND,
 	PBF_OP_GROUP_BEGIN,
 	PBF_OP_GROUP_END,
+	PBF_OP_DONE,
 };
 
 enum perf_bpf_filter_term {
tools/perf/util/bpf_skel/sample_filter.bpf.c +21 -18
index f59985101973..0d56e52b922c 100644
@@ -9,10 +9,10 @@
 
 /* BPF map that will be filled by user space */
 struct filters {
-	__uint(type, BPF_MAP_TYPE_ARRAY);
+	__uint(type, BPF_MAP_TYPE_HASH);
 	__type(key, int);
-	__type(value, struct perf_bpf_filter_entry);
-	__uint(max_entries, MAX_FILTERS);
+	__type(value, struct perf_bpf_filter_entry[MAX_FILTERS]);
+	__uint(max_entries, 1);
 } filters SEC(".maps");
 
 int dropped;
@@ -179,39 +179,39 @@ int perf_sample_filter(void *ctx)
 	__u64 sample_data;
 	int in_group = 0;
 	int group_result = 0;
-	int i;
+	int i, k;
 
 	kctx = bpf_cast_to_kern_ctx(ctx);
 
-	for (i = 0; i < MAX_FILTERS; i++) {
-		int key = i; /* needed for verifier :( */
+	k = 0;
+	entry = bpf_map_lookup_elem(&filters, &k);
+	if (entry == NULL)
+		goto drop;
 
-		entry = bpf_map_lookup_elem(&filters, &key);
-		if (entry == NULL)
-			break;
-		sample_data = perf_get_sample(kctx, entry);
+	for (i = 0; i < MAX_FILTERS; i++) {
+		sample_data = perf_get_sample(kctx, &entry[i]);
 
-		switch (entry->op) {
+		switch (entry[i].op) {
 		case PBF_OP_EQ:
-			CHECK_RESULT(sample_data, ==, entry->value)
+			CHECK_RESULT(sample_data, ==, entry[i].value)
 			break;
 		case PBF_OP_NEQ:
-			CHECK_RESULT(sample_data, !=, entry->value)
+			CHECK_RESULT(sample_data, !=, entry[i].value)
 			break;
 		case PBF_OP_GT:
-			CHECK_RESULT(sample_data, >, entry->value)
+			CHECK_RESULT(sample_data, >, entry[i].value)
 			break;
 		case PBF_OP_GE:
-			CHECK_RESULT(sample_data, >=, entry->value)
+			CHECK_RESULT(sample_data, >=, entry[i].value)
 			break;
 		case PBF_OP_LT:
-			CHECK_RESULT(sample_data, <, entry->value)
+			CHECK_RESULT(sample_data, <, entry[i].value)
 			break;
 		case PBF_OP_LE:
-			CHECK_RESULT(sample_data, <=, entry->value)
+			CHECK_RESULT(sample_data, <=, entry[i].value)
 			break;
 		case PBF_OP_AND:
-			CHECK_RESULT(sample_data, &, entry->value)
+			CHECK_RESULT(sample_data, &, entry[i].value)
 			break;
 		case PBF_OP_GROUP_BEGIN:
 			in_group = 1;
@@ -222,6 +222,9 @@ int perf_sample_filter(void *ctx)
 				goto drop;
 			in_group = 0;
 			break;
+		case PBF_OP_DONE:
+			/* no failures so far, accept it */
+			return 1;
 		}
 	}
 	/* generate sample data */
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
#2 lore raw

Re: [PATCH v3 1/8] perf bpf-filter: Make filters map a single entry hashmap

Quoted 11 lines · Namhyung Kim

And the value is now an array. This is to support multiple filter entries in the map later.

No functional changes intended.

+++ b/tools/perf/util/bpf-filter.c

(quoted diff) +0 -1
@@ -93,71 +93,102 @@ static int check_sample_flags(struct evsel *evsel, struct perf_bpf_filter_expr *
 
 int perf_bpf_filter__prepare(struct evsel *evsel)
 {
-	int i, x, y, fd;

Review static int check_sample_flags(struct evsel *evsel, struct perf_bpf_filter_expr * · line 105

Quoted 10 lines
(quoted diff) +6 -0
+	int i, x, y, fd, ret;
 	struct sample_filter_bpf *skel;
 	struct bpf_program *prog;
 	struct bpf_link *link;
 	struct perf_bpf_filter_expr *expr;
+	struct perf_bpf_filter_entry *entry;
+
+	entry = calloc(MAX_FILTERS, sizeof(*entry));
+	if (entry == NULL)
+		return -1;

I'm changing this to -ENOMEM since you use errno values in the other failure cases, ok?

This:

tools/perf/util/bpf-filter.c +1 -1
index 2510832d83f95e03..e98bacf41a248ced 100644
@@ -102,7 +102,7 @@ int perf_bpf_filter__prepare(struct evsel *evsel)
 
 	entry = calloc(MAX_FILTERS, sizeof(*entry));
 	if (entry == NULL)
-		return -1;
+		return -ENOMEM;
 
 	skel = sample_filter_bpf__open_and_load();
 	if (!skel) {
author maintainer
#3 lore raw

Re: [PATCH v3 1/8] perf bpf-filter: Make filters map a single entry hashmap

Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 8 lines · Namhyung Kim

And the value is now an array. This is to support multiple filter entries in the map later.

No functional changes intended.

+++ b/tools/perf/util/bpf-filter.c

(quoted diff) +0 -0
@@ -93,71 +93,102 @@ static int check_sample_flags(struct evsel *evsel, struct perf_bpf_filter_expr *
 
Quoted 16 lines

Review static int check_sample_flags(struct evsel *evsel, struct perf_bpf_filter_expr * · line 105

Quoted 13 lines
(quoted diff) +6 -1
 int perf_bpf_filter__prepare(struct evsel *evsel)
 {
-	int i, x, y, fd;
+	int i, x, y, fd, ret;
 	struct sample_filter_bpf *skel;
 	struct bpf_program *prog;
 	struct bpf_link *link;
 	struct perf_bpf_filter_expr *expr;
+	struct perf_bpf_filter_entry *entry;
+
+	entry = calloc(MAX_FILTERS, sizeof(*entry));
+	if (entry == NULL)
+		return -1;

I'm changing this to -ENOMEM since you use errno values in the other failure cases, ok?

Sure thing!

Thanks,
Namhyung

Quoted 16 lines

This:

tools/perf/util/bpf-filter.c +1 -1
index 2510832d83f95e03..e98bacf41a248ced 100644
@@ -102,7 +102,7 @@ int perf_bpf_filter__prepare(struct evsel *evsel)
 
 	entry = calloc(MAX_FILTERS, sizeof(*entry));
 	if (entry == NULL)
-		return -1;
+		return -ENOMEM;
 
 	skel = sample_filter_bpf__open_and_load();
 	if (!skel) {
#4 lore raw

Re: [PATCH v3 1/8] perf bpf-filter: Make filters map a single entry hashmap

In reply to Namhyung Kim
Quoted 4 lines · Namhyung Kim

And the value is now an array. This is to support multiple filter entries in the map later.

No functional changes intended.

Hey how can we test this feature these days?

With this first patch applied:

root@number:~# perf record -a -W -e cycles:p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 95 (Operation not supported)
root@number:~# perf record -a -W -e cpu_core/cycles/p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_core/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_core/cycles/p with 95 (Operation not supported)
root@number:~# perf record -a -W -e cpu_atom/cycles/p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 95 (Operation not supported)
root@number:~#

Interesting, it is taking a long time on the BPF prog load:

bpf(BPF_MAP_UPDATE_ELEM, {map_fd=49, key=0x7ffcc85a545c, value=0x7fee34bc2000, flags=BPF_ANY}, 32) = 0
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_PERF_EVENT, insn_cnt=335, insns=0xd1e6480, license="Dual BSD/GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(6, 9, 9), prog_flags=0, prog_name="perf_sample_fil", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=50, func_info_rec_size=8, func_info=0xd1b9c80, func_info_cnt=1, line_info_rec_size=16, line_info=0xd1e5300, line_info_cnt=135, attach_btf_id=0, attach_prog_fd=0, fd_array=NULL}, 148^Cstrace: Process 2110180 detached
<detached ...>

<HERE it takes an unusual time, even returning after I cancelled the strace session>

root@number:~#

root@number:~# Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 11 (Resource temporarily unavailable)

root@number:~#

root@number:~# uname -a
Linux number 6.9.9-100.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jul 11 19:26:10 UTC 2024 x86_64 GNU/Linux
root@number:~#
root@number:~# perf -v
perf version 6.10.g5510fb5c79e9
root@number:~# 

⬢[acme@toolbox perf-tools-next]$ git log --oneline -10
5510fb5c79e9f500 (HEAD -> perf-tools-next) perf annotate: Set instruction name to be used with insn-stat when using raw instruction b35a86e53eb496ea perf annotate: Add support to use libcapstone in powerpc
f2dc60d11290d53e perf annotate: Use capstone_init and remove open_capstone_handle from disasm.c c5bcba602eeee554 perf annotate: Make capstone_init non-static so that it can be used during symbol disassemble eef369c562510092 perf annotate: Update instruction tracking for powerpc
282701f1d77a3bdb perf annotate: Add more instructions for instruction tracking
758ee468ce5721e4 perf annotate: Add some of the arithmetic instructions to support instruction tracking in powerpc e8e7c1b6a9572bab perf annotate: Add support to identify memory instructions of opcode 31 in powerpc 3b3a0f04c1c6cd10 perf annotate: Add parse function for memory instructions in powerpc
a159d2acd44e707f perf annotate: Update parameters for reg extract functions to use raw instruction on powerpc ⬢[acme@toolbox perf-tools-next]$

Ideas?

- Arnaldo

author maintainer
#5 lore raw

Re: [PATCH v3 1/8] perf bpf-filter: Make filters map a single entry hashmap

Quoted 7 lines · Arnaldo Carvalho de Melo
Quoted 4 lines · Namhyung Kim

And the value is now an array. This is to support multiple filter entries in the map later.

No functional changes intended.

Hey how can we test this feature these days?

There's a 'perf record sample filtering (by BPF) tests'.

  $ ./perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 1042594
  Checking BPF-filter privilege
  try 'sudo perf record --setup-filter pin' first.
  bpf-filter test [Skipped permission]
  ---- end(-2) ----
   95: perf record sample filtering (by BPF) tests                     : Skip
Quoted 16 lines

With this first patch applied:

root@number:~# perf record -a -W -e cycles:p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 95 (Operation not supported)
root@number:~# perf record -a -W -e cpu_core/cycles/p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_core/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_core/cycles/p with 95 (Operation not supported)
root@number:~# perf record -a -W -e cpu_atom/cycles/p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 95 (Operation not supported)
root@number:~#

Do you say it's failing after the first patch? It looks like the atom CPU doesn't support PERF_SAMPLE_WEIGHT and should fail already.

The above test doesn't check the weight field FYI.

Quoted 38 lines

Interesting, it is taking a long time on the BPF prog load:

bpf(BPF_MAP_UPDATE_ELEM, {map_fd=49, key=0x7ffcc85a545c, value=0x7fee34bc2000, flags=BPF_ANY}, 32) = 0
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_PERF_EVENT, insn_cnt=335, insns=0xd1e6480, license="Dual BSD/GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(6, 9, 9), prog_flags=0, prog_name="perf_sample_fil", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=50, func_info_rec_size=8, func_info=0xd1b9c80, func_info_cnt=1, line_info_rec_size=16, line_info=0xd1e5300, line_info_cnt=135, attach_btf_id=0, attach_prog_fd=0, fd_array=NULL}, 148^Cstrace: Process 2110180 detached
<detached ...>

<HERE it takes an unusual time, even returning after I cancelled the strace session>

root@number:~#

root@number:~# Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 11 (Resource temporarily unavailable)

root@number:~#

root@number:~# uname -a
Linux number 6.9.9-100.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jul 11 19:26:10 UTC 2024 x86_64 GNU/Linux
root@number:~#
root@number:~# perf -v
perf version 6.10.g5510fb5c79e9
root@number:~# 

⬢[acme@toolbox perf-tools-next]$ git log --oneline -10
5510fb5c79e9f500 (HEAD -> perf-tools-next) perf annotate: Set instruction name to be used with insn-stat when using raw instruction b35a86e53eb496ea perf annotate: Add support to use libcapstone in powerpc
f2dc60d11290d53e perf annotate: Use capstone_init and remove open_capstone_handle from disasm.c c5bcba602eeee554 perf annotate: Make capstone_init non-static so that it can be used during symbol disassemble eef369c562510092 perf annotate: Update instruction tracking for powerpc
282701f1d77a3bdb perf annotate: Add more instructions for instruction tracking
758ee468ce5721e4 perf annotate: Add some of the arithmetic instructions to support instruction tracking in powerpc e8e7c1b6a9572bab perf annotate: Add support to identify memory instructions of opcode 31 in powerpc 3b3a0f04c1c6cd10 perf annotate: Add parse function for memory instructions in powerpc
a159d2acd44e707f perf annotate: Update parameters for reg extract functions to use raw instruction on powerpc ⬢[acme@toolbox perf-tools-next]$

Quoted 1 line

Ideas?

I don't know.. is it changed with this patch?

Thanks,
Namhyung

#6 lore raw

Re: [PATCH v3 1/8] perf bpf-filter: Make filters map a single entry hashmap

Quoted 38 lines · Namhyung Kim
Quoted 7 lines · Arnaldo Carvalho de Melo
Quoted 4 lines · Namhyung Kim

And the value is now an array. This is to support multiple filter entries in the map later.

No functional changes intended.

Hey how can we test this feature these days?

There's a 'perf record sample filtering (by BPF) tests'.

  $ ./perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 1042594
  Checking BPF-filter privilege
  try 'sudo perf record --setup-filter pin' first.
  bpf-filter test [Skipped permission]
  ---- end(-2) ----
   95: perf record sample filtering (by BPF) tests                     : Skip
Quoted 16 lines

With this first patch applied:

root@number:~# perf record -a -W -e cycles:p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 95 (Operation not supported)
root@number:~# perf record -a -W -e cpu_core/cycles/p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_core/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_core/cycles/p with 95 (Operation not supported)
root@number:~# perf record -a -W -e cpu_atom/cycles/p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 95 (Operation not supported)
root@number:~#
Quoted 1 line

Do you say it's failing after the first patch? It looks like the atom

yes

Quoted 1 line

CPU doesn't support PERF_SAMPLE_WEIGHT and should fail already.

I tried with 'cycles:p', 'cpu_atom/cycles/p' and with 'cpu_core/cycles/p', with and without -W (to use the warning advice) will try again tomorrow.

- Arnaldo

Quoted 46 lines

The above test doesn't check the weight field FYI.

Quoted 38 lines

Interesting, it is taking a long time on the BPF prog load:

bpf(BPF_MAP_UPDATE_ELEM, {map_fd=49, key=0x7ffcc85a545c, value=0x7fee34bc2000, flags=BPF_ANY}, 32) = 0
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_PERF_EVENT, insn_cnt=335, insns=0xd1e6480, license="Dual BSD/GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(6, 9, 9), prog_flags=0, prog_name="perf_sample_fil", prog_ifindex=0, expected_attach_type=BPF_CGROUP_INET_INGRESS, prog_btf_fd=50, func_info_rec_size=8, func_info=0xd1b9c80, func_info_cnt=1, line_info_rec_size=16, line_info=0xd1e5300, line_info_cnt=135, attach_btf_id=0, attach_prog_fd=0, fd_array=NULL}, 148^Cstrace: Process 2110180 detached
<detached ...>

<HERE it takes an unusual time, even returning after I cancelled the strace session>

root@number:~#

root@number:~# Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 11 (Resource temporarily unavailable)

root@number:~#

root@number:~# uname -a
Linux number 6.9.9-100.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jul 11 19:26:10 UTC 2024 x86_64 GNU/Linux
root@number:~#
root@number:~# perf -v
perf version 6.10.g5510fb5c79e9
root@number:~# 

⬢[acme@toolbox perf-tools-next]$ git log --oneline -10
5510fb5c79e9f500 (HEAD -> perf-tools-next) perf annotate: Set instruction name to be used with insn-stat when using raw instruction b35a86e53eb496ea perf annotate: Add support to use libcapstone in powerpc
f2dc60d11290d53e perf annotate: Use capstone_init and remove open_capstone_handle from disasm.c c5bcba602eeee554 perf annotate: Make capstone_init non-static so that it can be used during symbol disassemble eef369c562510092 perf annotate: Update instruction tracking for powerpc
282701f1d77a3bdb perf annotate: Add more instructions for instruction tracking
758ee468ce5721e4 perf annotate: Add some of the arithmetic instructions to support instruction tracking in powerpc e8e7c1b6a9572bab perf annotate: Add support to identify memory instructions of opcode 31 in powerpc 3b3a0f04c1c6cd10 perf annotate: Add parse function for memory instructions in powerpc
a159d2acd44e707f perf annotate: Update parameters for reg extract functions to use raw instruction on powerpc ⬢[acme@toolbox perf-tools-next]$

Quoted 1 line

Ideas?

I don't know.. is it changed with this patch?

Thanks,
Namhyung

author maintainer
#7 lore raw

Re: [PATCH v3 1/8] perf bpf-filter: Make filters map a single entry hashmap

Quoted 41 lines · Arnaldo Carvalho de Melo
Quoted 38 lines · Namhyung Kim
Quoted 7 lines · Arnaldo Carvalho de Melo
Quoted 4 lines · Namhyung Kim

And the value is now an array. This is to support multiple filter entries in the map later.

No functional changes intended.

Hey how can we test this feature these days?

There's a 'perf record sample filtering (by BPF) tests'.

  $ ./perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 1042594
  Checking BPF-filter privilege
  try 'sudo perf record --setup-filter pin' first.
  bpf-filter test [Skipped permission]
  ---- end(-2) ----
   95: perf record sample filtering (by BPF) tests                     : Skip
Quoted 16 lines

With this first patch applied:

root@number:~# perf record -a -W -e cycles:p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 95 (Operation not supported)
root@number:~# perf record -a -W -e cpu_core/cycles/p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_core/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_core/cycles/p with 95 (Operation not supported)
root@number:~# perf record -a -W -e cpu_atom/cycles/p --filter 'period > 100 || weight > 0' sleep 1
Error: cpu_atom/cycles/p event does not have PERF_SAMPLE_WEIGHT
 Hint: please add -W option to perf record
failed to set filter "BPF" on event cpu_atom/cycles/p with 95 (Operation not supported)
root@number:~#
Quoted 1 line

Do you say it's failing after the first patch? It looks like the atom

Quoted 7 lines

yes

Quoted 1 line

CPU doesn't support PERF_SAMPLE_WEIGHT and should fail already.

I tried with 'cycles:p', 'cpu_atom/cycles/p' and with 'cpu_core/cycles/p', with and without -W (to use the warning advice) will try again tomorrow.

Let me know if you find anything. Maybe it didn't set the flag in the attr. Can you run perf record -W true && perf evlist -v ?

Thanks,
Namhyung

author maintainer
#8 lore raw

PATCH v3 2/8 perf bpf-filter: Pass 'target' to perf_bpf_filter__prepare()

In reply to Namhyung Kim

This is needed to prepare target-specific actions in the later patch. We want to reuse the pinned BPF program and map for regular users to profile their own processes.

---
 tools/perf/builtin-record.c  | 2 +-
 tools/perf/builtin-stat.c    | 2 +-
 tools/perf/builtin-top.c     | 2 +-
 tools/perf/builtin-trace.c   | 2 +-
 tools/perf/util/bpf-filter.c | 2 +-
 tools/perf/util/bpf-filter.h | 6 ++++--
 tools/perf/util/evlist.c     | 5 +++--
 tools/perf/util/evlist.h     | 4 +++-
 8 files changed, 15 insertions(+), 10 deletions(-)
tools/perf/builtin-record.c +1 -1
index 019305b94e5f..e855a7688008 100644
@@ -1389,7 +1389,7 @@ static int record__open(struct record *rec)
 "even with a suitable vmlinux or kallsyms file.\n\n");
 	}
 
-	if (evlist__apply_filters(evlist, &pos)) {
+	if (evlist__apply_filters(evlist, &pos, &opts->target)) {
 		pr_err("failed to set filter \"%s\" on event %s with %d (%s)\n",
 			pos->filter ?: "BPF", evsel__name(pos), errno,
 			str_error_r(errno, msg, sizeof(msg)));
tools/perf/builtin-stat.c +1 -1
index 661832756a24..1f92445f7480 100644
@@ -833,7 +833,7 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx)
 			return -1;
 	}
 
-	if (evlist__apply_filters(evsel_list, &counter)) {
+	if (evlist__apply_filters(evsel_list, &counter, &target)) {
 		pr_err("failed to set filter \"%s\" on event %s with %d (%s)\n",
 			counter->filter, evsel__name(counter), errno,
 			str_error_r(errno, msg, sizeof(msg)));
tools/perf/builtin-top.c +1 -1
index e8cbbf10d361..d1a06a88d693 100644
@@ -1055,7 +1055,7 @@ static int perf_top__start_counters(struct perf_top *top)
 		}
 	}
 
-	if (evlist__apply_filters(evlist, &counter)) {
+	if (evlist__apply_filters(evlist, &counter, &opts->target)) {
 		pr_err("failed to set filter \"%s\" on event %s with %d (%s)\n",
 			counter->filter ?: "BPF", evsel__name(counter), errno,
 			str_error_r(errno, msg, sizeof(msg)));
tools/perf/builtin-trace.c +1 -1
index c42bc608954e..0f93ba83717d 100644
@@ -3959,7 +3959,7 @@ static int trace__run(struct trace *trace, int argc, const char **argv)
 	err = trace__expand_filters(trace, &evsel);
 	if (err)
 		goto out_delete_evlist;
-	err = evlist__apply_filters(evlist, &evsel);
+	err = evlist__apply_filters(evlist, &evsel, &trace->opts.target);
 	if (err < 0)
 		goto out_error_apply_filters;
 
tools/perf/util/bpf-filter.c +1 -1
index 2510832d83f9..0b2eca56aa10 100644
@@ -91,7 +91,7 @@ static int check_sample_flags(struct evsel *evsel, struct perf_bpf_filter_expr *
 	return -1;
 }
 
-int perf_bpf_filter__prepare(struct evsel *evsel)
+int perf_bpf_filter__prepare(struct evsel *evsel, struct target *target __maybe_unused)
 {
 	int i, x, y, fd, ret;
 	struct sample_filter_bpf *skel;
tools/perf/util/bpf-filter.h +4 -2
index cd6764442c16..605a3d0226e0 100644
@@ -16,6 +16,7 @@ struct perf_bpf_filter_expr {
 };
 
 struct evsel;
+struct target;
 
 #ifdef HAVE_BPF_SKEL
 struct perf_bpf_filter_expr *perf_bpf_filter_expr__new(enum perf_bpf_filter_term term,
@@ -23,7 +24,7 @@ struct perf_bpf_filter_expr *perf_bpf_filter_expr__new(enum perf_bpf_filter_term
 						       enum perf_bpf_filter_op op,
 						       unsigned long val);
 int perf_bpf_filter__parse(struct list_head *expr_head, const char *str);
-int perf_bpf_filter__prepare(struct evsel *evsel);
+int perf_bpf_filter__prepare(struct evsel *evsel, struct target *target);
 int perf_bpf_filter__destroy(struct evsel *evsel);
 u64 perf_bpf_filter__lost_count(struct evsel *evsel);
 
@@ -34,7 +35,8 @@ static inline int perf_bpf_filter__parse(struct list_head *expr_head __maybe_unu
 {
 	return -EOPNOTSUPP;
 }
-static inline int perf_bpf_filter__prepare(struct evsel *evsel __maybe_unused)
+static inline int perf_bpf_filter__prepare(struct evsel *evsel __maybe_unused,
+					   struct target *target __maybe_unused)
 {
 	return -EOPNOTSUPP;
 }
tools/perf/util/evlist.c +3 -2
index 3a719edafc7a..1417f9a23083 100644
@@ -1086,7 +1086,8 @@ int evlist__create_maps(struct evlist *evlist, struct target *target)
 	return -1;
 }
 
-int evlist__apply_filters(struct evlist *evlist, struct evsel **err_evsel)
+int evlist__apply_filters(struct evlist *evlist, struct evsel **err_evsel,
+			  struct target *target)
 {
 	struct evsel *evsel;
 	int err = 0;
@@ -1108,7 +1109,7 @@ int evlist__apply_filters(struct evlist *evlist, struct evsel **err_evsel)
 		 * non-tracepoint events can have BPF filters.
 		 */
 		if (!list_empty(&evsel->bpf_filters)) {
-			err = perf_bpf_filter__prepare(evsel);
+			err = perf_bpf_filter__prepare(evsel, target);
 			if (err) {
 				*err_evsel = evsel;
 				break;
tools/perf/util/evlist.h +3 -1
index cb91dc9117a2..cccc34da5a02 100644
@@ -20,6 +20,7 @@ struct pollfd;
 struct thread_map;
 struct perf_cpu_map;
 struct record_opts;
+struct target;
 
 /*
  * State machine of bkw_mmap_state:
@@ -212,7 +213,8 @@ void evlist__enable_non_dummy(struct evlist *evlist);
 void evlist__set_selected(struct evlist *evlist, struct evsel *evsel);
 
 int evlist__create_maps(struct evlist *evlist, struct target *target);
-int evlist__apply_filters(struct evlist *evlist, struct evsel **err_evsel);
+int evlist__apply_filters(struct evlist *evlist, struct evsel **err_evsel,
+			  struct target *target);
 
 u64 __evlist__combined_sample_type(struct evlist *evlist);
 u64 evlist__combined_sample_type(struct evlist *evlist);
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
author maintainer
#9 lore raw

PATCH v3 3/8 perf bpf-filter: Split per-task filter use case

In reply to Namhyung Kim

If the target is a list of tasks, it can use a shared hash map for filter expressions. The key of the filter map is an integer index like in an array. A separate pid_hash map is added to get the index for the filter map using the tgid.

For system-wide mode including per-cpu or per-user targets are handled by the single entry map like before.

---
 tools/perf/util/bpf-filter.c                 | 186 +++++++++++++++----
 tools/perf/util/bpf_skel/sample-filter.h     |   1 +
 tools/perf/util/bpf_skel/sample_filter.bpf.c |  21 +++
 3 files changed, 168 insertions(+), 40 deletions(-)
tools/perf/util/bpf-filter.c +146 -40
index 0b2eca56aa10..5ec0e0955ec4 100644
@@ -3,10 +3,13 @@
 
 #include <bpf/bpf.h>
 #include <linux/err.h>
+#include <api/fs/fs.h>
 #include <internal/xyarray.h>
+#include <perf/threadmap.h>
 
 #include "util/debug.h"
 #include "util/evsel.h"
+#include "util/target.h"
 
 #include "util/bpf-filter.h"
 #include <util/bpf-filter-flex.h>
@@ -91,38 +94,17 @@ static int check_sample_flags(struct evsel *evsel, struct perf_bpf_filter_expr *
 	return -1;
 }
 
-int perf_bpf_filter__prepare(struct evsel *evsel, struct target *target __maybe_unused)
+static int get_filter_entries(struct evsel *evsel, struct perf_bpf_filter_entry *entry)
 {
-	int i, x, y, fd, ret;
-	struct sample_filter_bpf *skel;
-	struct bpf_program *prog;
-	struct bpf_link *link;
+	int i = 0;
 	struct perf_bpf_filter_expr *expr;
-	struct perf_bpf_filter_entry *entry;
-
-	entry = calloc(MAX_FILTERS, sizeof(*entry));
-	if (entry == NULL)
-		return -1;
-
-	skel = sample_filter_bpf__open_and_load();
-	if (!skel) {
-		pr_err("Failed to load perf sample-filter BPF skeleton\n");
-		ret = -EPERM;
-		goto err;
-	}
 
-	i = 0;
-	fd = bpf_map__fd(skel->maps.filters);
 	list_for_each_entry(expr, &evsel->bpf_filters, list) {
-		if (check_sample_flags(evsel, expr) < 0) {
-			ret = -EINVAL;
-			goto err;
-		}
+		if (check_sample_flags(evsel, expr) < 0)
+			return -EINVAL;
 
-		if (i == MAX_FILTERS) {
-			ret = -E2BIG;
-			goto err;
-		}
+		if (i == MAX_FILTERS)
+			return -E2BIG;
 
 		entry[i].op = expr->op;
 		entry[i].part = expr->part;
@@ -134,10 +116,8 @@ int perf_bpf_filter__prepare(struct evsel *evsel, struct target *target __maybe_
 			struct perf_bpf_filter_expr *group;
 
 			list_for_each_entry(group, &expr->groups, list) {
-				if (i == MAX_FILTERS) {
-					ret = -E2BIG;
-					goto err;
-				}
+				if (i == MAX_FILTERS)
+					return -E2BIG;
 
 				entry[i].op = group->op;
 				entry[i].part = group->part;
@@ -146,10 +126,8 @@ int perf_bpf_filter__prepare(struct evsel *evsel, struct target *target __maybe_
 				i++;
 			}
 
-			if (i == MAX_FILTERS) {
-				ret = -E2BIG;
-				goto err;
-			}
+			if (i == MAX_FILTERS)
+				return -E2BIG;
 
 			entry[i].op = PBF_OP_GROUP_END;
 			i++;
@@ -161,15 +139,143 @@ int perf_bpf_filter__prepare(struct evsel *evsel, struct target *target __maybe_
 		entry[i].op = PBF_OP_DONE;
 		i++;
 	}
+	return 0;
+}
+
+static int convert_to_tgid(int tid)
+{
+	char path[128];
+	char *buf, *p, *q;
+	int tgid;
+	size_t len;
+
+	scnprintf(path, sizeof(path), "%d/status", tid);
+	if (procfs__read_str(path, &buf, &len) < 0)
+		return -1;
 
-	/* The filters map has only one entry for now */
-	i = 0;
-	if (bpf_map_update_elem(fd, &i, entry, BPF_ANY) < 0) {
-		ret = -errno;
-		pr_err("Failed to update the filter map\n");
+	p = strstr(buf, "Tgid:");
+	if (p == NULL) {
+		free(buf);
+		return -1;
+	}
+
+	tgid = strtol(p + 6, &q, 0);
+	free(buf);
+	if (*q != '\n')
+		return -1;
+
+	return tgid;
+}
+
+static int update_pid_hash(struct sample_filter_bpf *skel, struct evsel *evsel,
+			   struct perf_bpf_filter_entry *entry)
+{
+	int filter_idx;
+	int nr, last;
+	int fd = bpf_map__fd(skel->maps.filters);
+	struct perf_thread_map *threads;
+
+	/* Find the first available entry in the filters map */
+	for (filter_idx = 0; filter_idx < MAX_FILTERS; filter_idx++) {
+		if (bpf_map_update_elem(fd, &filter_idx, entry, BPF_NOEXIST) == 0)
+			break;
+	}
+
+	if (filter_idx == MAX_FILTERS) {
+		pr_err("Too many users for the filter map\n");
+		return -EBUSY;
+	}
+
+	threads = perf_evsel__threads(&evsel->core);
+	if (threads == NULL) {
+		pr_err("Cannot get the thread list of the event\n");
+		return -EINVAL;
+	}
+
+	/* save the index to a hash map */
+	fd = bpf_map__fd(skel->maps.pid_hash);
+
+	last = -1;
+	nr = perf_thread_map__nr(threads);
+	for (int i = 0; i < nr; i++) {
+		int pid = perf_thread_map__pid(threads, i);
+		int tgid;
+
+		/* it actually needs tgid, let's get tgid from /proc. */
+		tgid = convert_to_tgid(pid);
+		if (tgid < 0) {
+			/* the thread may be dead, ignore. */
+			continue;
+		}
+
+		if (tgid == last)
+			continue;
+		last = tgid;
+
+		if (bpf_map_update_elem(fd, &tgid, &filter_idx, BPF_ANY) < 0) {
+			pr_err("Failed to update the pid hash\n");
+			return -errno;
+		}
+		pr_debug("pid hash: %d -> %d\n", tgid, filter_idx);
+	}
+	return 0;
+}
+
+int perf_bpf_filter__prepare(struct evsel *evsel, struct target *target)
+{
+	int i, x, y, fd, ret;
+	struct sample_filter_bpf *skel = NULL;
+	struct bpf_program *prog;
+	struct bpf_link *link;
+	struct perf_bpf_filter_entry *entry;
+	bool needs_pid_hash = !target__has_cpu(target) && !target->uid_str;
+
+	entry = calloc(MAX_FILTERS, sizeof(*entry));
+	if (entry == NULL)
+		return -1;
+
+	ret = get_filter_entries(evsel, entry);
+	if (ret < 0) {
+		pr_err("Failed to process filter entries\n");
+		goto err;
+	}
+
+	skel = sample_filter_bpf__open();
+	if (!skel) {
+		pr_err("Failed to open perf sample-filter BPF skeleton\n");
+		ret = -EPERM;
 		goto err;
 	}
 
+	if (needs_pid_hash) {
+		bpf_map__set_max_entries(skel->maps.filters, MAX_FILTERS);
+		bpf_map__set_max_entries(skel->maps.pid_hash, MAX_PIDS);
+		skel->rodata->use_pid_hash = 1;
+	}
+
+	if (sample_filter_bpf__load(skel) < 0) {
+		pr_err("Failed to load perf sample-filter BPF skeleton\n");
+		ret = -EPERM;
+		goto err;
+	}
+
+	if (needs_pid_hash) {
+		/* The filters map is shared among other processes  */
+		ret = update_pid_hash(skel, evsel, entry);
+		if (ret < 0)
+			goto err;
+	} else {
+		i = 0;
+		fd = bpf_map__fd(skel->maps.filters);
+
+		/* The filters map has only one entry in this case */
+		if (bpf_map_update_elem(fd, &i, entry, BPF_ANY) < 0) {
+			ret = -errno;
+			pr_err("Failed to update the filter map\n");
+			goto err;
+		}
+	}
+
 	prog = skel->progs.perf_sample_filter;
 	for (x = 0; x < xyarray__max_x(evsel->core.fd); x++) {
 		for (y = 0; y < xyarray__max_y(evsel->core.fd); y++) {
tools/perf/util/bpf_skel/sample-filter.h +1 -0
index bb6a1b91f1df..e666bfd5fbdd 100644
@@ -2,6 +2,7 @@
 #define PERF_UTIL_BPF_SKEL_SAMPLE_FILTER_H
 
 #define MAX_FILTERS  64
+#define MAX_PIDS     (16 * 1024)
 
 /* supported filter operations */
 enum perf_bpf_filter_op {
tools/perf/util/bpf_skel/sample_filter.bpf.c +21 -0
index 0d56e52b922c..c5273f06fa45 100644
@@ -15,7 +15,16 @@ struct filters {
 	__uint(max_entries, 1);
 } filters SEC(".maps");
 
+/* tgid to filter index */
+struct pid_hash {
+	__uint(type, BPF_MAP_TYPE_HASH);
+	__type(key, int);
+	__type(value, int);
+	__uint(max_entries, 1);
+} pid_hash SEC(".maps");
+
 int dropped;
+volatile const int use_pid_hash;
 
 void *bpf_cast_to_kern_ctx(void *) __ksym;
 
@@ -184,6 +193,18 @@ int perf_sample_filter(void *ctx)
 	kctx = bpf_cast_to_kern_ctx(ctx);
 
 	k = 0;
+
+	if (use_pid_hash) {
+		int tgid = bpf_get_current_pid_tgid() >> 32;
+		int *idx;
+
+		idx = bpf_map_lookup_elem(&pid_hash, &tgid);
+		if (idx)
+			k = *idx;
+		else
+			goto drop;
+	}
+
 	entry = bpf_map_lookup_elem(&filters, &k);
 	if (entry == NULL)
 		goto drop;
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
author maintainer
#10 lore raw

PATCH v3 4/8 perf bpf-filter: Support pin/unpin BPF object

In reply to Namhyung Kim

And use the pinned objects for unprivileged users to profile their own tasks. The BPF objects need to be pinned in the BPF-fs by root first and it'll be handled in the later patch.

---
 tools/perf/util/bpf-filter.c | 230 +++++++++++++++++++++++++++++------
 tools/perf/util/bpf-filter.h |  13 ++
 2 files changed, 209 insertions(+), 34 deletions(-)
tools/perf/util/bpf-filter.c +196 -34
index 5ec0e0955ec4..37ed6c48debf 100644
@@ -1,5 +1,8 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 #include <stdlib.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#include <sys/stat.h>
 
 #include <bpf/bpf.h>
 #include <linux/err.h>
@@ -23,6 +26,9 @@
 #define __PERF_SAMPLE_TYPE(tt, st, opt)	{ tt, #st, opt }
 #define PERF_SAMPLE_TYPE(_st, opt)	__PERF_SAMPLE_TYPE(PBF_TERM_##_st, PERF_SAMPLE_##_st, opt)
 
+/* Index in the pinned 'filters' map.  Should be released after use. */
+static int pinned_filter_idx = -1;
+
 static const struct perf_sample_info {
 	enum perf_bpf_filter_term type;
 	const char *name;
@@ -47,6 +53,8 @@ static const struct perf_sample_info {
 	PERF_SAMPLE_TYPE(DATA_PAGE_SIZE, "--data-page-size"),
 };
 
+static int get_pinned_fd(const char *name);
+
 static const struct perf_sample_info *get_sample_info(enum perf_bpf_filter_term type)
 {
 	size_t i;
@@ -167,19 +175,26 @@ static int convert_to_tgid(int tid)
 	return tgid;
 }
 
-static int update_pid_hash(struct sample_filter_bpf *skel, struct evsel *evsel,
-			   struct perf_bpf_filter_entry *entry)
+static int update_pid_hash(struct evsel *evsel, struct perf_bpf_filter_entry *entry)
 {
 	int filter_idx;
-	int nr, last;
-	int fd = bpf_map__fd(skel->maps.filters);
+	int fd, nr, last;
 	struct perf_thread_map *threads;
 
+	fd = get_pinned_fd("filters");
+	if (fd < 0) {
+		pr_debug("cannot get fd for 'filters' map\n");
+		return fd;
+	}
+
 	/* Find the first available entry in the filters map */
 	for (filter_idx = 0; filter_idx < MAX_FILTERS; filter_idx++) {
-		if (bpf_map_update_elem(fd, &filter_idx, entry, BPF_NOEXIST) == 0)
+		if (bpf_map_update_elem(fd, &filter_idx, entry, BPF_NOEXIST) == 0) {
+			pinned_filter_idx = filter_idx;
 			break;
+		}
 	}
+	close(fd);
 
 	if (filter_idx == MAX_FILTERS) {
 		pr_err("Too many users for the filter map\n");
@@ -193,7 +208,9 @@ static int update_pid_hash(struct sample_filter_bpf *skel, struct evsel *evsel,
 	}
 
 	/* save the index to a hash map */
-	fd = bpf_map__fd(skel->maps.pid_hash);
+	fd = get_pinned_fd("pid_hash");
+	if (fd < 0)
+		return fd;
 
 	last = -1;
 	nr = perf_thread_map__nr(threads);
@@ -214,10 +231,12 @@ static int update_pid_hash(struct sample_filter_bpf *skel, struct evsel *evsel,
 
 		if (bpf_map_update_elem(fd, &tgid, &filter_idx, BPF_ANY) < 0) {
 			pr_err("Failed to update the pid hash\n");
-			return -errno;
+			close(fd);
+			return -1;
 		}
 		pr_debug("pid hash: %d -> %d\n", tgid, filter_idx);
 	}
+	close(fd);
 	return 0;
 }
 
@@ -240,40 +259,48 @@ int perf_bpf_filter__prepare(struct evsel *evsel, struct target *target)
 		goto err;
 	}
 
-	skel = sample_filter_bpf__open();
-	if (!skel) {
-		pr_err("Failed to open perf sample-filter BPF skeleton\n");
-		ret = -EPERM;
-		goto err;
-	}
+	if (needs_pid_hash && geteuid() != 0) {
+		/* The filters map is shared among other processes */
+		ret = update_pid_hash(evsel, entry);
+		if (ret < 0)
+			goto err;
 
-	if (needs_pid_hash) {
-		bpf_map__set_max_entries(skel->maps.filters, MAX_FILTERS);
-		bpf_map__set_max_entries(skel->maps.pid_hash, MAX_PIDS);
-		skel->rodata->use_pid_hash = 1;
+		fd = get_pinned_fd("perf_sample_filter");
+		if (fd < 0) {
+			ret = fd;
+			goto err;
+		}
+
+		for (x = 0; x < xyarray__max_x(evsel->core.fd); x++) {
+			for (y = 0; y < xyarray__max_y(evsel->core.fd); y++) {
+				ret = ioctl(FD(evsel, x, y), PERF_EVENT_IOC_SET_BPF, fd);
+				if (ret < 0) {
+					pr_err("Failed to attach perf sample-filter\n");
+					goto err;
+				}
+			}
+		}
+
+		close(fd);
+		free(entry);
+		return 0;
 	}
 
-	if (sample_filter_bpf__load(skel) < 0) {
+	skel = sample_filter_bpf__open_and_load();
+	if (!skel) {
+		ret = -errno;
 		pr_err("Failed to load perf sample-filter BPF skeleton\n");
-		ret = -EPERM;
 		goto err;
 	}
 
-	if (needs_pid_hash) {
-		/* The filters map is shared among other processes  */
-		ret = update_pid_hash(skel, evsel, entry);
-		if (ret < 0)
-			goto err;
-	} else {
-		i = 0;
-		fd = bpf_map__fd(skel->maps.filters);
-
-		/* The filters map has only one entry in this case */
-		if (bpf_map_update_elem(fd, &i, entry, BPF_ANY) < 0) {
-			ret = -errno;
-			pr_err("Failed to update the filter map\n");
-			goto err;
-		}
+	i = 0;
+	fd = bpf_map__fd(skel->maps.filters);
+
+	/* The filters map has only one entry in this case */
+	if (bpf_map_update_elem(fd, &i, entry, BPF_ANY) < 0) {
+		ret = -errno;
+		pr_err("Failed to update the filter map\n");
+		goto err;
 	}
 
 	prog = skel->progs.perf_sample_filter;
@@ -306,6 +333,15 @@ int perf_bpf_filter__destroy(struct evsel *evsel)
 		free(expr);
 	}
 	sample_filter_bpf__destroy(evsel->bpf_skel);
+
+	if (pinned_filter_idx >= 0) {
+		int fd = get_pinned_fd("filters");
+
+		bpf_map_delete_elem(fd, &pinned_filter_idx);
+		pinned_filter_idx = -1;
+		close(fd);
+	}
+
 	return 0;
 }
 
@@ -349,3 +385,129 @@ int perf_bpf_filter__parse(struct list_head *expr_head, const char *str)
 
 	return ret;
 }
+
+int perf_bpf_filter__pin(void)
+{
+	struct sample_filter_bpf *skel;
+	char *path = NULL;
+	int dir_fd, ret = -1;
+
+	skel = sample_filter_bpf__open();
+	if (!skel) {
+		ret = -errno;
+		pr_err("Failed to open perf sample-filter BPF skeleton\n");
+		goto err;
+	}
+
+	/* pinned program will use pid-hash */
+	bpf_map__set_max_entries(skel->maps.filters, MAX_FILTERS);
+	bpf_map__set_max_entries(skel->maps.pid_hash, MAX_PIDS);
+	skel->rodata->use_pid_hash = 1;
+
+	if (sample_filter_bpf__load(skel) < 0) {
+		ret = -errno;
+		pr_err("Failed to load perf sample-filter BPF skeleton\n");
+		goto err;
+	}
+
+	if (asprintf(&path, "%s/fs/bpf/%s", sysfs__mountpoint(),
+		     PERF_BPF_FILTER_PIN_PATH) < 0) {
+		ret = -errno;
+		pr_err("Failed to allocate pathname in the BPF-fs\n");
+		goto err;
+	}
+
+	ret = bpf_object__pin(skel->obj, path);
+	if (ret < 0) {
+		pr_err("Failed to pin BPF filter objects\n");
+		goto err;
+	}
+
+	/* setup access permissions for the pinned objects */
+	dir_fd = open(path, O_PATH);
+	if (dir_fd < 0) {
+		bpf_object__unpin(skel->obj, path);
+		ret = dir_fd;
+		goto err;
+	}
+
+	/* BPF-fs root has the sticky bit */
+	if (fchmodat(dir_fd, "..", 01755, 0) < 0) {
+		pr_debug("chmod for BPF-fs failed\n");
+		ret = -errno;
+		goto err_close;
+	}
+
+	/* perf_filter directory */
+	if (fchmodat(dir_fd, ".", 0755, 0) < 0) {
+		pr_debug("chmod for perf_filter directory failed?\n");
+		ret = -errno;
+		goto err_close;
+	}
+
+	/* programs need write permission for some reason */
+	if (fchmodat(dir_fd, "perf_sample_filter", 0777, 0) < 0) {
+		pr_debug("chmod for perf_sample_filter failed\n");
+		ret = -errno;
+	}
+	/* maps */
+	if (fchmodat(dir_fd, "filters", 0666, 0) < 0) {
+		pr_debug("chmod for filters failed\n");
+		ret = -errno;
+	}
+	if (fchmodat(dir_fd, "pid_hash", 0666, 0) < 0) {
+		pr_debug("chmod for pid_hash failed\n");
+		ret = -errno;
+	}
+
+err_close:
+	close(dir_fd);
+
+err:
+	free(path);
+	sample_filter_bpf__destroy(skel);
+	return ret;
+}
+
+int perf_bpf_filter__unpin(void)
+{
+	struct sample_filter_bpf *skel;
+	char *path = NULL;
+	int ret = -1;
+
+	skel = sample_filter_bpf__open_and_load();
+	if (!skel) {
+		ret = -errno;
+		pr_err("Failed to open perf sample-filter BPF skeleton\n");
+		goto err;
+	}
+
+	if (asprintf(&path, "%s/fs/bpf/%s", sysfs__mountpoint(),
+		     PERF_BPF_FILTER_PIN_PATH) < 0) {
+		ret = -errno;
+		pr_err("Failed to allocate pathname in the BPF-fs\n");
+		goto err;
+	}
+
+	ret = bpf_object__unpin(skel->obj, path);
+
+err:
+	free(path);
+	sample_filter_bpf__destroy(skel);
+	return ret;
+}
+
+static int get_pinned_fd(const char *name)
+{
+	char *path = NULL;
+	int fd;
+
+	if (asprintf(&path, "%s/fs/bpf/%s/%s", sysfs__mountpoint(),
+		     PERF_BPF_FILTER_PIN_PATH, name) < 0)
+		return -1;
+
+	fd = bpf_obj_get(path);
+
+	free(path);
+	return fd;
+}
tools/perf/util/bpf-filter.h +13 -0
index 605a3d0226e0..916ed7770b73 100644
@@ -18,6 +18,9 @@ struct perf_bpf_filter_expr {
 struct evsel;
 struct target;
 
+/* path in BPF-fs for the pinned program and maps */
+#define PERF_BPF_FILTER_PIN_PATH  "perf_filter"
+
 #ifdef HAVE_BPF_SKEL
 struct perf_bpf_filter_expr *perf_bpf_filter_expr__new(enum perf_bpf_filter_term term,
 						       int part,
@@ -27,6 +30,8 @@ int perf_bpf_filter__parse(struct list_head *expr_head, const char *str);
 int perf_bpf_filter__prepare(struct evsel *evsel, struct target *target);
 int perf_bpf_filter__destroy(struct evsel *evsel);
 u64 perf_bpf_filter__lost_count(struct evsel *evsel);
+int perf_bpf_filter__pin(void);
+int perf_bpf_filter__unpin(void);
 
 #else /* !HAVE_BPF_SKEL */
 
@@ -48,5 +53,13 @@ static inline u64 perf_bpf_filter__lost_count(struct evsel *evsel __maybe_unused
 {
 	return 0;
 }
+static inline int perf_bpf_filter__pin(void)
+{
+	return -EOPNOTSUPP;
+}
+static inline int perf_bpf_filter__unpin(void)
+{
+	return -EOPNOTSUPP;
+}
 #endif /* HAVE_BPF_SKEL*/
 #endif /* PERF_UTIL_BPF_FILTER_H */
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
author maintainer
#11 lore raw

PATCH v3 5/8 perf bpf-filter: Support separate lost counts for each filter

In reply to Namhyung Kim

As the BPF filter is shared between other processes, it should have its own counter for each invocation. Add a new array map (lost_count) to save the count using the same index as the filter. It should clear the count before running the filter.

---
 tools/perf/util/bpf-filter.c                 | 37 ++++++++++++++++++--
 tools/perf/util/bpf_skel/sample_filter.bpf.c | 15 ++++++--
 2 files changed, 48 insertions(+), 4 deletions(-)
tools/perf/util/bpf-filter.c +35 -2
index 37ed6c48debf..c5eb0b7eec19 100644
@@ -260,11 +260,23 @@ int perf_bpf_filter__prepare(struct evsel *evsel, struct target *target)
 	}
 
 	if (needs_pid_hash && geteuid() != 0) {
+		int zero = 0;
+
 		/* The filters map is shared among other processes */
 		ret = update_pid_hash(evsel, entry);
 		if (ret < 0)
 			goto err;
 
+		fd = get_pinned_fd("dropped");
+		if (fd < 0) {
+			ret = fd;
+			goto err;
+		}
+
+		/* Reset the lost count */
+		bpf_map_update_elem(fd, &pinned_filter_idx, &zero, BPF_ANY);
+		close(fd);
+
 		fd = get_pinned_fd("perf_sample_filter");
 		if (fd < 0) {
 			ret = fd;
@@ -347,9 +359,25 @@ int perf_bpf_filter__destroy(struct evsel *evsel)
 
 u64 perf_bpf_filter__lost_count(struct evsel *evsel)
 {
-	struct sample_filter_bpf *skel = evsel->bpf_skel;
+	int count = 0;
+
+	if (list_empty(&evsel->bpf_filters))
+		return 0;
+
+	if (pinned_filter_idx >= 0) {
+		int fd = get_pinned_fd("dropped");
+
+		bpf_map_lookup_elem(fd, &pinned_filter_idx, &count);
+		close(fd);
+	} else if (evsel->bpf_skel) {
+		struct sample_filter_bpf *skel = evsel->bpf_skel;
+		int fd = bpf_map__fd(skel->maps.dropped);
+		int idx = 0;
 
-	return skel ? skel->bss->dropped : 0;
+		bpf_map_lookup_elem(fd, &idx, &count);
+	}
+
+	return count;
 }
 
 struct perf_bpf_filter_expr *perf_bpf_filter_expr__new(enum perf_bpf_filter_term term,
@@ -402,6 +430,7 @@ int perf_bpf_filter__pin(void)
 	/* pinned program will use pid-hash */
 	bpf_map__set_max_entries(skel->maps.filters, MAX_FILTERS);
 	bpf_map__set_max_entries(skel->maps.pid_hash, MAX_PIDS);
+	bpf_map__set_max_entries(skel->maps.dropped, MAX_FILTERS);
 	skel->rodata->use_pid_hash = 1;
 
 	if (sample_filter_bpf__load(skel) < 0) {
@@ -459,6 +488,10 @@ int perf_bpf_filter__pin(void)
 		pr_debug("chmod for pid_hash failed\n");
 		ret = -errno;
 	}
+	if (fchmodat(dir_fd, "dropped", 0666, 0) < 0) {
+		pr_debug("chmod for dropped failed\n");
+		ret = -errno;
+	}
 
 err_close:
 	close(dir_fd);
tools/perf/util/bpf_skel/sample_filter.bpf.c +13 -2
index c5273f06fa45..4c75354b84fd 100644
@@ -23,7 +23,14 @@ struct pid_hash {
 	__uint(max_entries, 1);
 } pid_hash SEC(".maps");
 
-int dropped;
+/* tgid to filter index */
+struct lost_count {
+	__uint(type, BPF_MAP_TYPE_ARRAY);
+	__type(key, int);
+	__type(value, int);
+	__uint(max_entries, 1);
+} dropped SEC(".maps");
+
 volatile const int use_pid_hash;
 
 void *bpf_cast_to_kern_ctx(void *) __ksym;
@@ -189,6 +196,7 @@ int perf_sample_filter(void *ctx)
 	int in_group = 0;
 	int group_result = 0;
 	int i, k;
+	int *losts;
 
 	kctx = bpf_cast_to_kern_ctx(ctx);
 
@@ -252,7 +260,10 @@ int perf_sample_filter(void *ctx)
 	return 1;
 
 drop:
-	__sync_fetch_and_add(&dropped, 1);
+	losts = bpf_map_lookup_elem(&dropped, &k);
+	if (losts != NULL)
+		__sync_fetch_and_add(losts, 1);
+
 	return 0;
 }
 
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
author maintainer
#12 lore raw

PATCH v3 6/8 perf record: Fix a potential error handling issue

In reply to Namhyung Kim

The evlist is allocated at the beginning of cmd_record(). Also free-ing thread masks should be paired with record__init_thread_masks() which is called right before __cmd_record().

Let's change the order of these functions to release the resources correctly in case of errors. This is maybe fine as the process exits, but it might be a problem if it manages some system-wide resources that live longer than the process.

---
 tools/perf/builtin-record.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tools/perf/builtin-record.c +3 -3
index e855a7688008..a473000f3599 100644
@@ -4242,13 +4242,13 @@ int cmd_record(int argc, const char **argv)
 
 	err = __cmd_record(&record, argc, argv);
 out:
-	evlist__delete(rec->evlist);
+	record__free_thread_masks(rec, rec->nr_threads);
+	rec->nr_threads = 0;
 	symbol__exit();
 	auxtrace_record__free(rec->itr);
 out_opts:
-	record__free_thread_masks(rec, rec->nr_threads);
-	rec->nr_threads = 0;
 	evlist__close_control(rec->opts.ctl_fd, rec->opts.ctl_fd_ack, &rec->opts.ctl_fd_close);
+	evlist__delete(rec->evlist);
 	return err;
 }
 
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
author maintainer
#13 lore raw

PATCH v3 7/8 perf record: Add --setup-filter option

In reply to Namhyung Kim

To allow BPF filters for unprivileged users it needs to pin the BPF objects to BPF-fs first. Let's add a new option to pin and unpin the objects easily. I'm not sure 'perf record' is a right place to do this but I don't have a better idea right now.

  $ sudo perf record --setup-filter pin

The above command would pin BPF program and maps for the filter when the system has BPF-fs (usually at /sys/fs/bpf/). To unpin the objects, users can run the following command (as root).

  $ sudo perf record --setup-filter unpin
---
 tools/perf/Documentation/perf-record.txt |  5 +++++
 tools/perf/builtin-record.c              | 15 +++++++++++++++
 2 files changed, 20 insertions(+)
tools/perf/Documentation/perf-record.txt +5 -0
index d6532ed97c02..41e36b4dc765 100644
@@ -828,6 +828,11 @@ filtered through the mask provided by -C option.
 	only, as of now.  So the applications built without the frame
 	pointer might see bogus addresses.
 
+--setup-filter=<action>::
+	Prepare BPF filter to be used by regular users.  The action should be
+	either "pin" or "unpin".  The filter can be used after it's pinned.
+
+
 include::intel-hybrid.txt[]
 
 SEE ALSO
tools/perf/builtin-record.c +15 -0
index a473000f3599..e88dedc0391b 100644
@@ -171,6 +171,7 @@ struct record {
 	bool			timestamp_filename;
 	bool			timestamp_boundary;
 	bool			off_cpu;
+	const char		*filter_action;
 	struct switch_output	switch_output;
 	unsigned long long	samples;
 	unsigned long		output_max_size;	/* = 0: unlimited */
@@ -3557,6 +3558,8 @@ static struct option __record_options[] = {
 			    "write collected trace data into several data files using parallel threads",
 			    record__parse_threads),
 	OPT_BOOLEAN(0, "off-cpu", &record.off_cpu, "Enable off-cpu analysis"),
+	OPT_STRING(0, "setup-filter", &record.filter_action, "pin|unpin",
+		   "BPF filter action"),
 	OPT_END()
 };
 
@@ -4086,6 +4089,18 @@ int cmd_record(int argc, const char **argv)
 		pr_warning("WARNING: --timestamp-filename option is not available in parallel streaming mode.\n");
 	}
 
+	if (rec->filter_action) {
+		if (!strcmp(rec->filter_action, "pin"))
+			err = perf_bpf_filter__pin();
+		else if (!strcmp(rec->filter_action, "unpin"))
+			err = perf_bpf_filter__unpin();
+		else {
+			pr_warning("Unknown BPF filter action: %s\n", rec->filter_action);
+			err = -EINVAL;
+		}
+		goto out_opts;
+	}
+
 	/*
 	 * Allow aliases to facilitate the lookup of symbols for address
 	 * filters. Refer to auxtrace_parse_filters().
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
author maintainer
#14 lore raw

PATCH v3 8/8 perf test: Update sample filtering test

In reply to Namhyung Kim

Now it can run the BPF filtering test with normal user if the BPF objects are pinned by 'sudo perf record --setup-filter pin'. Let's update the test case to verify the behavior. It'll skip the test if the filter check is failed from a normal user, but it shows a message how to set up the filters.

First, run the test as a normal user and it fails.

  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 425677
  Checking BPF-filter privilege
  try 'sudo perf record --setup-filter pin' first.       <<<--- here
  bpf-filter test [Skipped permission]
  ---- end(-2) ----
   95: perf record sample filtering (by BPF) tests                     : Skip

According to the message, run the perf record command to pin the BPF objects.

  $ sudo perf record --setup-filter pin

And re-run the test as a normal user.

  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 424486
  Checking BPF-filter privilege
  Basic bpf-filter test
  Basic bpf-filter test [Success]
  Failing bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Failing bpf-filter test [Success]
  Group bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Error: task-clock event does not have PERF_SAMPLE_CODE_PAGE_SIZE
  Group bpf-filter test [Success]
  ---- end(0) ----
   95: perf record sample filtering (by BPF) tests                     : Ok
---
 tools/perf/tests/shell/record_bpf_filter.sh | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
tools/perf/tests/shell/record_bpf_filter.sh +7 -6
index 31c593966e8c..c5882d620db7 100755
@@ -22,15 +22,16 @@ trap trap_cleanup EXIT TERM INT
 test_bpf_filter_priv() {
   echo "Checking BPF-filter privilege"
 
-  if [ "$(id -u)" != 0 ]
-  then
-    echo "bpf-filter test [Skipped permission]"
-    err=2
-    return
-  fi
   if ! perf record -e task-clock --filter 'period > 1' \
 	  -o /dev/null --quiet true 2>&1
   then
+    if [ "$(id -u)" != 0 ]
+    then
+      echo "try 'sudo perf record --setup-filter pin' first."
+      echo "bpf-filter test [Skipped permission]"
+      err=2
+      return
+    fi
     echo "bpf-filter test [Skipped missing BPF support]"
     err=2
     return
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
#15 lore raw

Re: [PATCH v3 8/8] perf test: Update sample filtering test

Quoted 25 lines · Namhyung Kim

Now it can run the BPF filtering test with normal user if the BPF objects are pinned by 'sudo perf record --setup-filter pin'. Let's update the test case to verify the behavior. It'll skip the test if the filter check is failed from a normal user, but it shows a message how to set up the filters.

First, run the test as a normal user and it fails.

  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 425677
  Checking BPF-filter privilege
  try 'sudo perf record --setup-filter pin' first.       <<<--- here
  bpf-filter test [Skipped permission]
  ---- end(-2) ----
   95: perf record sample filtering (by BPF) tests                     : Skip

According to the message, run the perf record command to pin the BPF objects.

  $ sudo perf record --setup-filter pin

And re-run the test as a normal user.

Quoted 16 lines
  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 424486
  Checking BPF-filter privilege
  Basic bpf-filter test
  Basic bpf-filter test [Success]
  Failing bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Failing bpf-filter test [Success]
  Group bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Error: task-clock event does not have PERF_SAMPLE_CODE_PAGE_SIZE
  Group bpf-filter test [Success]
  ---- end(0) ----
   95: perf record sample filtering (by BPF) tests                     : Ok

Ok, so I tested one of the examples you provide as a root user:

root@number:~# perf record -o- -e cycles:u --filter 'period < 10' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.024 MB - ]
       perf-exec  228020 53029.825757:          1 cpu_core/cycles/u:      7fe361d1cc11 [unknown] ([unknown])
       perf-exec  228020 53029.825760:          1 cpu_core/cycles/u:      7fe361d1cc11 [unknown] ([unknown])
            perf  228020 53029.826313:          1 cpu_atom/cycles/u:      7fd80d7ba040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228020 53029.826316:          1 cpu_atom/cycles/u:      7fd80d7ba040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228020 53029.838051:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.838054:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.838055:          9 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.844137:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228020 53029.844139:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
root@number:~# perf record -o- -e cycles:u --filter 'period < 100000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.025 MB - ]
       perf-exec  228084 53076.760776:          1 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760779:          1 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760779:         10 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760780:        497 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760781:      27924 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
            perf  228084 53076.761318:          1 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761320:          1 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761321:         14 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761322:        518 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761322:      20638 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768070:          1 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768072:          1 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768073:         17 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768073:        836 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768074:      44346 cpu_core/cycles/u:      7f317056e89b _dl_relocate_object+0x1db (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.843976:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843978:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843979:         13 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843979:        563 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843980:      26519 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53077.482090:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482092:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482093:         15 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482093:        746 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482094:      38315 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
root@number:~#

Filtering by period works as advertised, now I have done as root;

root@number:~# perf record --setup-filter pin
root@number:~# ls -la /sys/fs/bpf/perf_filter/
total 0
drwxr-xr-x. 2 root root 0 Jul 31 10:43 .
drwxr-xr-t. 3 root root 0 Jul 31 10:43 ..
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 dropped
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 filters
-rwxrwxrwx. 1 root root 0 Jul 31 10:43 perf_sample_filter
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 pid_hash
-rw-------. 1 root root 0 Jul 31 10:43 sample_f_rodata
root@number:~# ls -la /sys/fs/bpf/perf_filter/perf_sample_filter 
-rwxrwxrwx. 1 root root 0 Jul 31 10:43 /sys/fs/bpf/perf_filter/perf_sample_filter
root@number:~#

And as a normal user I try:

acme@number:~$ perf record -o- -e cycles:u perf test -w noploop | perf script -i- | head
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.204 MB - ]
            perf  228218 53158.670585:          1 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670590:          1 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670592:          7 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670593:        117 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670595:       2152 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670604:      38977 cpu_atom/cycles/u:  ffffffff99201280 [unknown] ([unknown])
            perf  228218 53158.670650:     167064 cpu_atom/cycles/u:      7f2fb1b67d7c intel_check_word.constprop.0+0x16c (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.671472:     232830 cpu_atom/cycles/u:      7f2fb1b75d98 strcmp+0x78 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.672710:     191183 cpu_atom/cycles/u:      7f2fb1b59311 _dl_map_object_from_fd+0xea1 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.673461:     158125 cpu_atom/cycles/u:      7f2fb1b77148 strcmp+0x1428 (/usr/lib64/ld-linux-x86-64.so.2)
acme@number:~$

Ok, no filtering, bot samples, lets try to use filtering as with root:

acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$
acme@number:~$ perf record -v -e cycles:u --filter 'period < 10000000' perf test -w noploop 
Using CPUID GenuineIntel-6-B7-1
DEBUGINFOD_URLS=
nr_cblocks: 0
affinity: SYS
mmap flush: 1
comp level: 0
Problems creating module maps, continuing anyway...
pid hash: 228434 -> 13
pid hash: 228434 -> 14
mmap size 528384B
Control descriptor is not initialized
Couldn't start the BPF side band thread:
BPF programs starting from now on won't be annotatable
[ perf record: Woken up 1 times to write data ]
failed to write feature CPU_PMU_CAPS
[ perf record: Captured and wrote 0.009 MB perf.data ]
acme@number:~$

I also tried with task-clock:

acme@number:~$ perf record -o- -e task-clock -c 10000 perf test -w noploop | perf script -i- | head
            perf  229784 54146.473644:      10000 task-clock:u:      7faf38f1c622 get_common_indices.constprop.0+0xa2 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473654:      10000 task-clock:u:      7faf38f1d323 update_active.constprop.0+0x383 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473664:      10000 task-clock:u:      7faf38f1cd32 intel_check_word.constprop.0+0x122 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473674:      10000 task-clock:u:      7faf38f1cd7c intel_check_word.constprop.0+0x16c (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473684:      10000 task-clock:u:      7faf38f19de5 __tunable_get_val+0x75 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473704:      10000 task-clock:u:      7faf38f190d0 rtld_mutex_dummy+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473754:      10000 task-clock:u:      7faf38f1a80e _dl_cache_libcmp+0xe (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473864:      10000 task-clock:u:      7faf38f2adb9 strcmp+0x99 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473954:      10000 task-clock:u:      7faf38f1aa02 search_cache+0x112 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.474024:      10000 task-clock:u:      7faf38f0de38 _dl_map_object_from_fd+0x9c8 (/usr/lib64/ld-linux-x86-64.so.2)
acme@number:~$ 
acme@number:~$ perf record -o- -e task-clock -c 10000 --filter 'ip < 0xffffffff00000000' perf test -w noploop | perf script -i- 
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.127 MB - ]
acme@number:~$

Ideas?

I'm keeping it in my local tree so that I run it through the container build tests meanwhile we try to understand this, what am I missing?

- Arnaldo

⬢[acme@toolbox perf-tools-next]$ uname -a
Linux toolbox 6.9.10-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jul 18 21:39:30 UTC 2024 x86_64 GNU/Linux ⬢[acme@toolbox perf-tools-next]$ perf -vv
perf version 6.11.rc1.g77a71e434cf4

                 dwarf: [ on  ]  # HAVE_DWARF_SUPPORT
    dwarf_getlocations: [ on  ]  # HAVE_DWARF_GETLOCATIONS_SUPPORT
         syscall_table: [ on  ]  # HAVE_SYSCALL_TABLE_SUPPORT
                libbfd: [ OFF ]  # HAVE_LIBBFD_SUPPORT
            debuginfod: [ on  ]  # HAVE_DEBUGINFOD_SUPPORT
                libelf: [ on  ]  # HAVE_LIBELF_SUPPORT
               libnuma: [ on  ]  # HAVE_LIBNUMA_SUPPORT
numa_num_possible_cpus: [ on  ]  # HAVE_LIBNUMA_SUPPORT
               libperl: [ on  ]  # HAVE_LIBPERL_SUPPORT
             libpython: [ on  ]  # HAVE_LIBPYTHON_SUPPORT
              libslang: [ on  ]  # HAVE_SLANG_SUPPORT
             libcrypto: [ on  ]  # HAVE_LIBCRYPTO_SUPPORT
             libunwind: [ on  ]  # HAVE_LIBUNWIND_SUPPORT
    libdw-dwarf-unwind: [ on  ]  # HAVE_DWARF_SUPPORT
           libcapstone: [ on  ]  # HAVE_LIBCAPSTONE_SUPPORT
                  zlib: [ on  ]  # HAVE_ZLIB_SUPPORT
                  lzma: [ on  ]  # HAVE_LZMA_SUPPORT
             get_cpuid: [ on  ]  # HAVE_AUXTRACE_SUPPORT
                   bpf: [ on  ]  # HAVE_LIBBPF_SUPPORT
                   aio: [ on  ]  # HAVE_AIO_SUPPORT
                  zstd: [ on  ]  # HAVE_ZSTD_SUPPORT
               libpfm4: [ on  ]  # HAVE_LIBPFM
         libtraceevent: [ on  ]  # HAVE_LIBTRACEEVENT
         bpf_skeletons: [ on  ]  # HAVE_BPF_SKEL
  dwarf-unwind-support: [ on  ]  # HAVE_DWARF_UNWIND_SUPPORT
            libopencsd: [ on  ]  # HAVE_CSTRACE_SUPPORT

⬢[acme@toolbox perf-tools-next]$ git log --oneline -10
2a24133dc55000b3 (HEAD -> perf-tools-next) perf test: Update sample filtering test d6fed13469889202 perf record: Add --setup-filter option
d8a2ec627150b7a4 perf record: Fix a potential error handling issue b0313e52f43035b5 perf bpf-filter: Support separate lost counts for each filter eb29dacbaf215fda perf bpf-filter: Support pin/unpin BPF object
086e7d06af7ce4eb perf bpf-filter: Split per-task filter use case d3453d1bb80cdbb2 perf bpf-filter: Pass 'target' to perf_bpf_filter__prepare() 736cd1c7a7105e1d perf bpf-filter: Make filters map a single entry hashmap 96ff640908b9808e perf jevents: Don't stop at the first matched pmu when searching a events table 379fe1f78ed5ceaf perf jevents: Use name for special find value (PMU_EVENTS__NOT_FOUND) ⬢[acme@toolbox perf-tools-next]$

Quoted 35 lines
---
 tools/perf/tests/shell/record_bpf_filter.sh | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
tools/perf/tests/shell/record_bpf_filter.sh +7 -6
index 31c593966e8c..c5882d620db7 100755
@@ -22,15 +22,16 @@ trap trap_cleanup EXIT TERM INT
 test_bpf_filter_priv() {
   echo "Checking BPF-filter privilege"
 
-  if [ "$(id -u)" != 0 ]
-  then
-    echo "bpf-filter test [Skipped permission]"
-    err=2
-    return
-  fi
   if ! perf record -e task-clock --filter 'period > 1' \
 	  -o /dev/null --quiet true 2>&1
   then
+    if [ "$(id -u)" != 0 ]
+    then
+      echo "try 'sudo perf record --setup-filter pin' first."
+      echo "bpf-filter test [Skipped permission]"
+      err=2
+      return
+    fi
     echo "bpf-filter test [Skipped missing BPF support]"
     err=2
     return
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
author maintainer
#16 lore raw

Re: [PATCH v3 8/8] perf test: Update sample filtering test

Quoted 122 lines · Arnaldo Carvalho de Melo
Quoted 25 lines · Namhyung Kim

Now it can run the BPF filtering test with normal user if the BPF objects are pinned by 'sudo perf record --setup-filter pin'. Let's update the test case to verify the behavior. It'll skip the test if the filter check is failed from a normal user, but it shows a message how to set up the filters.

First, run the test as a normal user and it fails.

  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 425677
  Checking BPF-filter privilege
  try 'sudo perf record --setup-filter pin' first.       <<<--- here
  bpf-filter test [Skipped permission]
  ---- end(-2) ----
   95: perf record sample filtering (by BPF) tests                     : Skip

According to the message, run the perf record command to pin the BPF objects.

  $ sudo perf record --setup-filter pin

And re-run the test as a normal user.

Quoted 16 lines
  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 424486
  Checking BPF-filter privilege
  Basic bpf-filter test
  Basic bpf-filter test [Success]
  Failing bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Failing bpf-filter test [Success]
  Group bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Error: task-clock event does not have PERF_SAMPLE_CODE_PAGE_SIZE
  Group bpf-filter test [Success]
  ---- end(0) ----
   95: perf record sample filtering (by BPF) tests                     : Ok

Ok, so I tested one of the examples you provide as a root user:

root@number:~# perf record -o- -e cycles:u --filter 'period < 10' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.024 MB - ]
       perf-exec  228020 53029.825757:          1 cpu_core/cycles/u:      7fe361d1cc11 [unknown] ([unknown])
       perf-exec  228020 53029.825760:          1 cpu_core/cycles/u:      7fe361d1cc11 [unknown] ([unknown])
            perf  228020 53029.826313:          1 cpu_atom/cycles/u:      7fd80d7ba040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228020 53029.826316:          1 cpu_atom/cycles/u:      7fd80d7ba040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228020 53029.838051:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.838054:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.838055:          9 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.844137:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228020 53029.844139:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
root@number:~# perf record -o- -e cycles:u --filter 'period < 100000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.025 MB - ]
       perf-exec  228084 53076.760776:          1 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760779:          1 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760779:         10 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760780:        497 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760781:      27924 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
            perf  228084 53076.761318:          1 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761320:          1 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761321:         14 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761322:        518 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761322:      20638 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768070:          1 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768072:          1 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768073:         17 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768073:        836 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768074:      44346 cpu_core/cycles/u:      7f317056e89b _dl_relocate_object+0x1db (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.843976:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843978:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843979:         13 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843979:        563 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843980:      26519 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53077.482090:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482092:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482093:         15 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482093:        746 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482094:      38315 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
root@number:~#

Filtering by period works as advertised, now I have done as root;

root@number:~# perf record --setup-filter pin
root@number:~# ls -la /sys/fs/bpf/perf_filter/
total 0
drwxr-xr-x. 2 root root 0 Jul 31 10:43 .
drwxr-xr-t. 3 root root 0 Jul 31 10:43 ..
(quoted diff) +0 -5
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 dropped
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 filters
-rwxrwxrwx. 1 root root 0 Jul 31 10:43 perf_sample_filter
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 pid_hash
-rw-------. 1 root root 0 Jul 31 10:43 sample_f_rodata
root@number:~# ls -la /sys/fs/bpf/perf_filter/perf_sample_filter 
-rwxrwxrwx. 1 root root 0 Jul 31 10:43 /sys/fs/bpf/perf_filter/perf_sample_filter
root@number:~#

And as a normal user I try:

acme@number:~$ perf record -o- -e cycles:u perf test -w noploop | perf script -i- | head
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.204 MB - ]
            perf  228218 53158.670585:          1 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670590:          1 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670592:          7 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670593:        117 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670595:       2152 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670604:      38977 cpu_atom/cycles/u:  ffffffff99201280 [unknown] ([unknown])
            perf  228218 53158.670650:     167064 cpu_atom/cycles/u:      7f2fb1b67d7c intel_check_word.constprop.0+0x16c (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.671472:     232830 cpu_atom/cycles/u:      7f2fb1b75d98 strcmp+0x78 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.672710:     191183 cpu_atom/cycles/u:      7f2fb1b59311 _dl_map_object_from_fd+0xea1 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.673461:     158125 cpu_atom/cycles/u:      7f2fb1b77148 strcmp+0x1428 (/usr/lib64/ld-linux-x86-64.so.2)
acme@number:~$

Ok, no filtering, bot samples, lets try to use filtering as with root:

Quoted 13 lines
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$

Hmm.. strange. The above command works well for me.

Quoted 11 lines
acme@number:~$ perf record -v -e cycles:u --filter 'period < 10000000' perf test -w noploop 
Using CPUID GenuineIntel-6-B7-1
DEBUGINFOD_URLS=
nr_cblocks: 0
affinity: SYS
mmap flush: 1
comp level: 0
Problems creating module maps, continuing anyway...
pid hash: 228434 -> 13
pid hash: 228434 -> 14

This part is a little strange as it's using two entries. Hmm, are you using a hybrid machine? Anyway I think it should work there too..

Also the number is too high.. I expect 1 or 2. Maybe it didn't release all the entries. Let me think about the case.

Thanks,
Namhyung

Quoted 113 lines

mmap size 528384B
Control descriptor is not initialized Couldn't start the BPF side band thread:
BPF programs starting from now on won't be annotatable [ perf record: Woken up 1 times to write data ] failed to write feature CPU_PMU_CAPS
[ perf record: Captured and wrote 0.009 MB perf.data ] acme@number:~$

I also tried with task-clock:

acme@number:~$ perf record -o- -e task-clock -c 10000 perf test -w noploop | perf script -i- | head
            perf  229784 54146.473644:      10000 task-clock:u:      7faf38f1c622 get_common_indices.constprop.0+0xa2 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473654:      10000 task-clock:u:      7faf38f1d323 update_active.constprop.0+0x383 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473664:      10000 task-clock:u:      7faf38f1cd32 intel_check_word.constprop.0+0x122 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473674:      10000 task-clock:u:      7faf38f1cd7c intel_check_word.constprop.0+0x16c (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473684:      10000 task-clock:u:      7faf38f19de5 __tunable_get_val+0x75 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473704:      10000 task-clock:u:      7faf38f190d0 rtld_mutex_dummy+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473754:      10000 task-clock:u:      7faf38f1a80e _dl_cache_libcmp+0xe (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473864:      10000 task-clock:u:      7faf38f2adb9 strcmp+0x99 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473954:      10000 task-clock:u:      7faf38f1aa02 search_cache+0x112 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.474024:      10000 task-clock:u:      7faf38f0de38 _dl_map_object_from_fd+0x9c8 (/usr/lib64/ld-linux-x86-64.so.2)
acme@number:~$ 
acme@number:~$ perf record -o- -e task-clock -c 10000 --filter 'ip < 0xffffffff00000000' perf test -w noploop | perf script -i- 
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.127 MB - ]
acme@number:~$

Ideas?

I'm keeping it in my local tree so that I run it through the container build tests meanwhile we try to understand this, what am I missing?

- Arnaldo

⬢[acme@toolbox perf-tools-next]$ uname -a
Linux toolbox 6.9.10-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jul 18 21:39:30 UTC 2024 x86_64 GNU/Linux ⬢[acme@toolbox perf-tools-next]$ perf -vv
perf version 6.11.rc1.g77a71e434cf4

                 dwarf: [ on  ]  # HAVE_DWARF_SUPPORT
    dwarf_getlocations: [ on  ]  # HAVE_DWARF_GETLOCATIONS_SUPPORT
         syscall_table: [ on  ]  # HAVE_SYSCALL_TABLE_SUPPORT
                libbfd: [ OFF ]  # HAVE_LIBBFD_SUPPORT
            debuginfod: [ on  ]  # HAVE_DEBUGINFOD_SUPPORT
                libelf: [ on  ]  # HAVE_LIBELF_SUPPORT
               libnuma: [ on  ]  # HAVE_LIBNUMA_SUPPORT
numa_num_possible_cpus: [ on  ]  # HAVE_LIBNUMA_SUPPORT
               libperl: [ on  ]  # HAVE_LIBPERL_SUPPORT
             libpython: [ on  ]  # HAVE_LIBPYTHON_SUPPORT
              libslang: [ on  ]  # HAVE_SLANG_SUPPORT
             libcrypto: [ on  ]  # HAVE_LIBCRYPTO_SUPPORT
             libunwind: [ on  ]  # HAVE_LIBUNWIND_SUPPORT
    libdw-dwarf-unwind: [ on  ]  # HAVE_DWARF_SUPPORT
           libcapstone: [ on  ]  # HAVE_LIBCAPSTONE_SUPPORT
                  zlib: [ on  ]  # HAVE_ZLIB_SUPPORT
                  lzma: [ on  ]  # HAVE_LZMA_SUPPORT
             get_cpuid: [ on  ]  # HAVE_AUXTRACE_SUPPORT
                   bpf: [ on  ]  # HAVE_LIBBPF_SUPPORT
                   aio: [ on  ]  # HAVE_AIO_SUPPORT
                  zstd: [ on  ]  # HAVE_ZSTD_SUPPORT
               libpfm4: [ on  ]  # HAVE_LIBPFM
         libtraceevent: [ on  ]  # HAVE_LIBTRACEEVENT
         bpf_skeletons: [ on  ]  # HAVE_BPF_SKEL
  dwarf-unwind-support: [ on  ]  # HAVE_DWARF_UNWIND_SUPPORT
            libopencsd: [ on  ]  # HAVE_CSTRACE_SUPPORT

⬢[acme@toolbox perf-tools-next]$ git log --oneline -10
2a24133dc55000b3 (HEAD -> perf-tools-next) perf test: Update sample filtering test d6fed13469889202 perf record: Add --setup-filter option
d8a2ec627150b7a4 perf record: Fix a potential error handling issue b0313e52f43035b5 perf bpf-filter: Support separate lost counts for each filter eb29dacbaf215fda perf bpf-filter: Support pin/unpin BPF object
086e7d06af7ce4eb perf bpf-filter: Split per-task filter use case d3453d1bb80cdbb2 perf bpf-filter: Pass 'target' to perf_bpf_filter__prepare() 736cd1c7a7105e1d perf bpf-filter: Make filters map a single entry hashmap 96ff640908b9808e perf jevents: Don't stop at the first matched pmu when searching a events table 379fe1f78ed5ceaf perf jevents: Use name for special find value (PMU_EVENTS__NOT_FOUND) ⬢[acme@toolbox perf-tools-next]$

Quoted 35 lines
---
 tools/perf/tests/shell/record_bpf_filter.sh | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
tools/perf/tests/shell/record_bpf_filter.sh +7 -6
index 31c593966e8c..c5882d620db7 100755
@@ -22,15 +22,16 @@ trap trap_cleanup EXIT TERM INT
 test_bpf_filter_priv() {
   echo "Checking BPF-filter privilege"
 
-  if [ "$(id -u)" != 0 ]
-  then
-    echo "bpf-filter test [Skipped permission]"
-    err=2
-    return
-  fi
   if ! perf record -e task-clock --filter 'period > 1' \
 	  -o /dev/null --quiet true 2>&1
   then
+    if [ "$(id -u)" != 0 ]
+    then
+      echo "try 'sudo perf record --setup-filter pin' first."
+      echo "bpf-filter test [Skipped permission]"
+      err=2
+      return
+    fi
     echo "bpf-filter test [Skipped missing BPF support]"
     err=2
     return
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
#17 lore raw

Re: [PATCH v3 8/8] perf test: Update sample filtering test

Quoted 151 lines · Namhyung Kim
Quoted 122 lines · Arnaldo Carvalho de Melo
Quoted 25 lines · Namhyung Kim

Now it can run the BPF filtering test with normal user if the BPF objects are pinned by 'sudo perf record --setup-filter pin'. Let's update the test case to verify the behavior. It'll skip the test if the filter check is failed from a normal user, but it shows a message how to set up the filters.

First, run the test as a normal user and it fails.

  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 425677
  Checking BPF-filter privilege
  try 'sudo perf record --setup-filter pin' first.       <<<--- here
  bpf-filter test [Skipped permission]
  ---- end(-2) ----
   95: perf record sample filtering (by BPF) tests                     : Skip

According to the message, run the perf record command to pin the BPF objects.

  $ sudo perf record --setup-filter pin

And re-run the test as a normal user.

Quoted 16 lines
  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 424486
  Checking BPF-filter privilege
  Basic bpf-filter test
  Basic bpf-filter test [Success]
  Failing bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Failing bpf-filter test [Success]
  Group bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Error: task-clock event does not have PERF_SAMPLE_CODE_PAGE_SIZE
  Group bpf-filter test [Success]
  ---- end(0) ----
   95: perf record sample filtering (by BPF) tests                     : Ok

Ok, so I tested one of the examples you provide as a root user:

root@number:~# perf record -o- -e cycles:u --filter 'period < 10' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.024 MB - ]
       perf-exec  228020 53029.825757:          1 cpu_core/cycles/u:      7fe361d1cc11 [unknown] ([unknown])
       perf-exec  228020 53029.825760:          1 cpu_core/cycles/u:      7fe361d1cc11 [unknown] ([unknown])
            perf  228020 53029.826313:          1 cpu_atom/cycles/u:      7fd80d7ba040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228020 53029.826316:          1 cpu_atom/cycles/u:      7fd80d7ba040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228020 53029.838051:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.838054:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.838055:          9 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.844137:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228020 53029.844139:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
root@number:~# perf record -o- -e cycles:u --filter 'period < 100000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.025 MB - ]
       perf-exec  228084 53076.760776:          1 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760779:          1 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760779:         10 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760780:        497 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760781:      27924 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
            perf  228084 53076.761318:          1 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761320:          1 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761321:         14 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761322:        518 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761322:      20638 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768070:          1 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768072:          1 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768073:         17 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768073:        836 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768074:      44346 cpu_core/cycles/u:      7f317056e89b _dl_relocate_object+0x1db (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.843976:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843978:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843979:         13 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843979:        563 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843980:      26519 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53077.482090:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482092:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482093:         15 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482093:        746 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482094:      38315 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
root@number:~#

Filtering by period works as advertised, now I have done as root;

root@number:~# perf record --setup-filter pin
root@number:~# ls -la /sys/fs/bpf/perf_filter/
total 0
drwxr-xr-x. 2 root root 0 Jul 31 10:43 .
drwxr-xr-t. 3 root root 0 Jul 31 10:43 ..
(quoted diff) +0 -5
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 dropped
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 filters
-rwxrwxrwx. 1 root root 0 Jul 31 10:43 perf_sample_filter
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 pid_hash
-rw-------. 1 root root 0 Jul 31 10:43 sample_f_rodata
root@number:~# ls -la /sys/fs/bpf/perf_filter/perf_sample_filter 
-rwxrwxrwx. 1 root root 0 Jul 31 10:43 /sys/fs/bpf/perf_filter/perf_sample_filter
root@number:~#

And as a normal user I try:

acme@number:~$ perf record -o- -e cycles:u perf test -w noploop | perf script -i- | head
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.204 MB - ]
            perf  228218 53158.670585:          1 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670590:          1 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670592:          7 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670593:        117 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670595:       2152 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670604:      38977 cpu_atom/cycles/u:  ffffffff99201280 [unknown] ([unknown])
            perf  228218 53158.670650:     167064 cpu_atom/cycles/u:      7f2fb1b67d7c intel_check_word.constprop.0+0x16c (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.671472:     232830 cpu_atom/cycles/u:      7f2fb1b75d98 strcmp+0x78 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.672710:     191183 cpu_atom/cycles/u:      7f2fb1b59311 _dl_map_object_from_fd+0xea1 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.673461:     158125 cpu_atom/cycles/u:      7f2fb1b77148 strcmp+0x1428 (/usr/lib64/ld-linux-x86-64.so.2)
acme@number:~$

Ok, no filtering, bot samples, lets try to use filtering as with root:

Quoted 13 lines
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$

Hmm.. strange. The above command works well for me.

Quoted 11 lines
acme@number:~$ perf record -v -e cycles:u --filter 'period < 10000000' perf test -w noploop 
Using CPUID GenuineIntel-6-B7-1
DEBUGINFOD_URLS=
nr_cblocks: 0
affinity: SYS
mmap flush: 1
comp level: 0
Problems creating module maps, continuing anyway...
pid hash: 228434 -> 13
pid hash: 228434 -> 14
Quoted 2 lines

This part is a little strange as it's using two entries. Hmm, are you using a hybrid machine? Anyway I think it should work there too..

Yes, I'll try it again on a 5950x since it isn't hybrid.

Quoted 2 lines

Also the number is too high.. I expect 1 or 2. Maybe it didn't release all the entries. Let me think about the case.

I'm inclined for now to keep this series merged and then take fixes on top, please advise if this isn't ok with you.

- Arnaldo

Quoted 117 lines

Thanks,
Namhyung

Quoted 113 lines

mmap size 528384B
Control descriptor is not initialized Couldn't start the BPF side band thread:
BPF programs starting from now on won't be annotatable [ perf record: Woken up 1 times to write data ] failed to write feature CPU_PMU_CAPS
[ perf record: Captured and wrote 0.009 MB perf.data ] acme@number:~$

I also tried with task-clock:

acme@number:~$ perf record -o- -e task-clock -c 10000 perf test -w noploop | perf script -i- | head
            perf  229784 54146.473644:      10000 task-clock:u:      7faf38f1c622 get_common_indices.constprop.0+0xa2 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473654:      10000 task-clock:u:      7faf38f1d323 update_active.constprop.0+0x383 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473664:      10000 task-clock:u:      7faf38f1cd32 intel_check_word.constprop.0+0x122 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473674:      10000 task-clock:u:      7faf38f1cd7c intel_check_word.constprop.0+0x16c (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473684:      10000 task-clock:u:      7faf38f19de5 __tunable_get_val+0x75 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473704:      10000 task-clock:u:      7faf38f190d0 rtld_mutex_dummy+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473754:      10000 task-clock:u:      7faf38f1a80e _dl_cache_libcmp+0xe (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473864:      10000 task-clock:u:      7faf38f2adb9 strcmp+0x99 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.473954:      10000 task-clock:u:      7faf38f1aa02 search_cache+0x112 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  229784 54146.474024:      10000 task-clock:u:      7faf38f0de38 _dl_map_object_from_fd+0x9c8 (/usr/lib64/ld-linux-x86-64.so.2)
acme@number:~$ 
acme@number:~$ perf record -o- -e task-clock -c 10000 --filter 'ip < 0xffffffff00000000' perf test -w noploop | perf script -i- 
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.127 MB - ]
acme@number:~$

Ideas?

I'm keeping it in my local tree so that I run it through the container build tests meanwhile we try to understand this, what am I missing?

- Arnaldo

⬢[acme@toolbox perf-tools-next]$ uname -a
Linux toolbox 6.9.10-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jul 18 21:39:30 UTC 2024 x86_64 GNU/Linux ⬢[acme@toolbox perf-tools-next]$ perf -vv
perf version 6.11.rc1.g77a71e434cf4

                 dwarf: [ on  ]  # HAVE_DWARF_SUPPORT
    dwarf_getlocations: [ on  ]  # HAVE_DWARF_GETLOCATIONS_SUPPORT
         syscall_table: [ on  ]  # HAVE_SYSCALL_TABLE_SUPPORT
                libbfd: [ OFF ]  # HAVE_LIBBFD_SUPPORT
            debuginfod: [ on  ]  # HAVE_DEBUGINFOD_SUPPORT
                libelf: [ on  ]  # HAVE_LIBELF_SUPPORT
               libnuma: [ on  ]  # HAVE_LIBNUMA_SUPPORT
numa_num_possible_cpus: [ on  ]  # HAVE_LIBNUMA_SUPPORT
               libperl: [ on  ]  # HAVE_LIBPERL_SUPPORT
             libpython: [ on  ]  # HAVE_LIBPYTHON_SUPPORT
              libslang: [ on  ]  # HAVE_SLANG_SUPPORT
             libcrypto: [ on  ]  # HAVE_LIBCRYPTO_SUPPORT
             libunwind: [ on  ]  # HAVE_LIBUNWIND_SUPPORT
    libdw-dwarf-unwind: [ on  ]  # HAVE_DWARF_SUPPORT
           libcapstone: [ on  ]  # HAVE_LIBCAPSTONE_SUPPORT
                  zlib: [ on  ]  # HAVE_ZLIB_SUPPORT
                  lzma: [ on  ]  # HAVE_LZMA_SUPPORT
             get_cpuid: [ on  ]  # HAVE_AUXTRACE_SUPPORT
                   bpf: [ on  ]  # HAVE_LIBBPF_SUPPORT
                   aio: [ on  ]  # HAVE_AIO_SUPPORT
                  zstd: [ on  ]  # HAVE_ZSTD_SUPPORT
               libpfm4: [ on  ]  # HAVE_LIBPFM
         libtraceevent: [ on  ]  # HAVE_LIBTRACEEVENT
         bpf_skeletons: [ on  ]  # HAVE_BPF_SKEL
  dwarf-unwind-support: [ on  ]  # HAVE_DWARF_UNWIND_SUPPORT
            libopencsd: [ on  ]  # HAVE_CSTRACE_SUPPORT

⬢[acme@toolbox perf-tools-next]$ git log --oneline -10
2a24133dc55000b3 (HEAD -> perf-tools-next) perf test: Update sample filtering test d6fed13469889202 perf record: Add --setup-filter option
d8a2ec627150b7a4 perf record: Fix a potential error handling issue b0313e52f43035b5 perf bpf-filter: Support separate lost counts for each filter eb29dacbaf215fda perf bpf-filter: Support pin/unpin BPF object
086e7d06af7ce4eb perf bpf-filter: Split per-task filter use case d3453d1bb80cdbb2 perf bpf-filter: Pass 'target' to perf_bpf_filter__prepare() 736cd1c7a7105e1d perf bpf-filter: Make filters map a single entry hashmap 96ff640908b9808e perf jevents: Don't stop at the first matched pmu when searching a events table 379fe1f78ed5ceaf perf jevents: Use name for special find value (PMU_EVENTS__NOT_FOUND) ⬢[acme@toolbox perf-tools-next]$

Quoted 35 lines
---
 tools/perf/tests/shell/record_bpf_filter.sh | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
tools/perf/tests/shell/record_bpf_filter.sh +7 -6
index 31c593966e8c..c5882d620db7 100755
@@ -22,15 +22,16 @@ trap trap_cleanup EXIT TERM INT
 test_bpf_filter_priv() {
   echo "Checking BPF-filter privilege"
 
-  if [ "$(id -u)" != 0 ]
-  then
-    echo "bpf-filter test [Skipped permission]"
-    err=2
-    return
-  fi
   if ! perf record -e task-clock --filter 'period > 1' \
 	  -o /dev/null --quiet true 2>&1
   then
+    if [ "$(id -u)" != 0 ]
+    then
+      echo "try 'sudo perf record --setup-filter pin' first."
+      echo "bpf-filter test [Skipped permission]"
+      err=2
+      return
+    fi
     echo "bpf-filter test [Skipped missing BPF support]"
     err=2
     return
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
author maintainer
#18 lore raw

Re: [PATCH v3 8/8] perf test: Update sample filtering test

Quoted 155 lines · Arnaldo Carvalho de Melo
Quoted 151 lines · Namhyung Kim
Quoted 122 lines · Arnaldo Carvalho de Melo
Quoted 25 lines · Namhyung Kim

Now it can run the BPF filtering test with normal user if the BPF objects are pinned by 'sudo perf record --setup-filter pin'. Let's update the test case to verify the behavior. It'll skip the test if the filter check is failed from a normal user, but it shows a message how to set up the filters.

First, run the test as a normal user and it fails.

  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 425677
  Checking BPF-filter privilege
  try 'sudo perf record --setup-filter pin' first.       <<<--- here
  bpf-filter test [Skipped permission]
  ---- end(-2) ----
   95: perf record sample filtering (by BPF) tests                     : Skip

According to the message, run the perf record command to pin the BPF objects.

  $ sudo perf record --setup-filter pin

And re-run the test as a normal user.

Quoted 16 lines
  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 424486
  Checking BPF-filter privilege
  Basic bpf-filter test
  Basic bpf-filter test [Success]
  Failing bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Failing bpf-filter test [Success]
  Group bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Error: task-clock event does not have PERF_SAMPLE_CODE_PAGE_SIZE
  Group bpf-filter test [Success]
  ---- end(0) ----
   95: perf record sample filtering (by BPF) tests                     : Ok

Ok, so I tested one of the examples you provide as a root user:

root@number:~# perf record -o- -e cycles:u --filter 'period < 10' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.024 MB - ]
       perf-exec  228020 53029.825757:          1 cpu_core/cycles/u:      7fe361d1cc11 [unknown] ([unknown])
       perf-exec  228020 53029.825760:          1 cpu_core/cycles/u:      7fe361d1cc11 [unknown] ([unknown])
            perf  228020 53029.826313:          1 cpu_atom/cycles/u:      7fd80d7ba040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228020 53029.826316:          1 cpu_atom/cycles/u:      7fd80d7ba040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228020 53029.838051:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.838054:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.838055:          9 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.844137:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228020 53029.844139:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
root@number:~# perf record -o- -e cycles:u --filter 'period < 100000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.025 MB - ]
       perf-exec  228084 53076.760776:          1 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760779:          1 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760779:         10 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760780:        497 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760781:      27924 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
            perf  228084 53076.761318:          1 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761320:          1 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761321:         14 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761322:        518 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761322:      20638 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768070:          1 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768072:          1 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768073:         17 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768073:        836 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768074:      44346 cpu_core/cycles/u:      7f317056e89b _dl_relocate_object+0x1db (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.843976:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843978:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843979:         13 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843979:        563 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843980:      26519 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53077.482090:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482092:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482093:         15 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482093:        746 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482094:      38315 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
root@number:~#

Filtering by period works as advertised, now I have done as root;

root@number:~# perf record --setup-filter pin
root@number:~# ls -la /sys/fs/bpf/perf_filter/
total 0
drwxr-xr-x. 2 root root 0 Jul 31 10:43 .
drwxr-xr-t. 3 root root 0 Jul 31 10:43 ..
(quoted diff) +0 -5
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 dropped
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 filters
-rwxrwxrwx. 1 root root 0 Jul 31 10:43 perf_sample_filter
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 pid_hash
-rw-------. 1 root root 0 Jul 31 10:43 sample_f_rodata
root@number:~# ls -la /sys/fs/bpf/perf_filter/perf_sample_filter 
-rwxrwxrwx. 1 root root 0 Jul 31 10:43 /sys/fs/bpf/perf_filter/perf_sample_filter
root@number:~#

And as a normal user I try:

acme@number:~$ perf record -o- -e cycles:u perf test -w noploop | perf script -i- | head
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.204 MB - ]
            perf  228218 53158.670585:          1 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670590:          1 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670592:          7 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670593:        117 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670595:       2152 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670604:      38977 cpu_atom/cycles/u:  ffffffff99201280 [unknown] ([unknown])
            perf  228218 53158.670650:     167064 cpu_atom/cycles/u:      7f2fb1b67d7c intel_check_word.constprop.0+0x16c (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.671472:     232830 cpu_atom/cycles/u:      7f2fb1b75d98 strcmp+0x78 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.672710:     191183 cpu_atom/cycles/u:      7f2fb1b59311 _dl_map_object_from_fd+0xea1 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.673461:     158125 cpu_atom/cycles/u:      7f2fb1b77148 strcmp+0x1428 (/usr/lib64/ld-linux-x86-64.so.2)
acme@number:~$

Ok, no filtering, bot samples, lets try to use filtering as with root:

Quoted 13 lines
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$

Hmm.. strange. The above command works well for me.

Quoted 11 lines
acme@number:~$ perf record -v -e cycles:u --filter 'period < 10000000' perf test -w noploop 
Using CPUID GenuineIntel-6-B7-1
DEBUGINFOD_URLS=
nr_cblocks: 0
affinity: SYS
mmap flush: 1
comp level: 0
Problems creating module maps, continuing anyway...
pid hash: 228434 -> 13
pid hash: 228434 -> 14
Quoted 2 lines

This part is a little strange as it's using two entries. Hmm, are you using a hybrid machine? Anyway I think it should work there too..

Quoted 7 lines

Yes, I'll try it again on a 5950x since it isn't hybrid.

Quoted 2 lines

Also the number is too high.. I expect 1 or 2. Maybe it didn't release all the entries. Let me think about the case.

I'm inclined for now to keep this series merged and then take fixes on top, please advise if this isn't ok with you.

No objections, I'll investigate why it failed on your machine..

Thanks,
Namhyung

author maintainer
#19 lore raw

Re: [PATCH v3 8/8] perf test: Update sample filtering test

Quoted 158 lines · Namhyung Kim
Quoted 157 lines · Arnaldo Carvalho de Melo
Quoted 151 lines · Namhyung Kim
Quoted 122 lines · Arnaldo Carvalho de Melo
Quoted 25 lines · Namhyung Kim

Now it can run the BPF filtering test with normal user if the BPF objects are pinned by 'sudo perf record --setup-filter pin'. Let's update the test case to verify the behavior. It'll skip the test if the filter check is failed from a normal user, but it shows a message how to set up the filters.

First, run the test as a normal user and it fails.

  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 425677
  Checking BPF-filter privilege
  try 'sudo perf record --setup-filter pin' first.       <<<--- here
  bpf-filter test [Skipped permission]
  ---- end(-2) ----
   95: perf record sample filtering (by BPF) tests                     : Skip

According to the message, run the perf record command to pin the BPF objects.

  $ sudo perf record --setup-filter pin

And re-run the test as a normal user.

Quoted 16 lines
  $ perf test -vv filtering
   95: perf record sample filtering (by BPF) tests:
  --- start ---
  test child forked, pid 424486
  Checking BPF-filter privilege
  Basic bpf-filter test
  Basic bpf-filter test [Success]
  Failing bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Failing bpf-filter test [Success]
  Group bpf-filter test
  Error: task-clock event does not have PERF_SAMPLE_CPU
  Error: task-clock event does not have PERF_SAMPLE_CODE_PAGE_SIZE
  Group bpf-filter test [Success]
  ---- end(0) ----
   95: perf record sample filtering (by BPF) tests                     : Ok

Ok, so I tested one of the examples you provide as a root user:

root@number:~# perf record -o- -e cycles:u --filter 'period < 10' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.024 MB - ]
       perf-exec  228020 53029.825757:          1 cpu_core/cycles/u:      7fe361d1cc11 [unknown] ([unknown])
       perf-exec  228020 53029.825760:          1 cpu_core/cycles/u:      7fe361d1cc11 [unknown] ([unknown])
            perf  228020 53029.826313:          1 cpu_atom/cycles/u:      7fd80d7ba040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228020 53029.826316:          1 cpu_atom/cycles/u:      7fd80d7ba040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228020 53029.838051:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.838054:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.838055:          9 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228020 53029.844137:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228020 53029.844139:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
root@number:~# perf record -o- -e cycles:u --filter 'period < 100000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.025 MB - ]
       perf-exec  228084 53076.760776:          1 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760779:          1 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760779:         10 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760780:        497 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
       perf-exec  228084 53076.760781:      27924 cpu_core/cycles/u:      7f7e7691cc11 [unknown] ([unknown])
            perf  228084 53076.761318:          1 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761320:          1 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761321:         14 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761322:        518 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.761322:      20638 cpu_atom/cycles/u:      7f317057d040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768070:          1 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768072:          1 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768073:         17 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768073:        836 cpu_core/cycles/u:      7f317056e898 _dl_relocate_object+0x1d8 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.768074:      44346 cpu_core/cycles/u:      7f317056e89b _dl_relocate_object+0x1db (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228084 53076.843976:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843978:          1 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843979:         13 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843979:        563 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53076.843980:      26519 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
            perf  228084 53077.482090:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482092:          1 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482093:         15 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482093:        746 cpu_core/cycles/u:            53b062 noploop+0x62 (/home/acme/bin/perf)
            perf  228084 53077.482094:      38315 cpu_core/cycles/u:            53b05c noploop+0x5c (/home/acme/bin/perf)
root@number:~#

Filtering by period works as advertised, now I have done as root;

root@number:~# perf record --setup-filter pin
root@number:~# ls -la /sys/fs/bpf/perf_filter/
total 0
drwxr-xr-x. 2 root root 0 Jul 31 10:43 .
drwxr-xr-t. 3 root root 0 Jul 31 10:43 ..
(quoted diff) +0 -5
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 dropped
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 filters
-rwxrwxrwx. 1 root root 0 Jul 31 10:43 perf_sample_filter
-rw-rw-rw-. 1 root root 0 Jul 31 10:43 pid_hash
-rw-------. 1 root root 0 Jul 31 10:43 sample_f_rodata
root@number:~# ls -la /sys/fs/bpf/perf_filter/perf_sample_filter 
-rwxrwxrwx. 1 root root 0 Jul 31 10:43 /sys/fs/bpf/perf_filter/perf_sample_filter
root@number:~#

And as a normal user I try:

acme@number:~$ perf record -o- -e cycles:u perf test -w noploop | perf script -i- | head
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.204 MB - ]
            perf  228218 53158.670585:          1 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670590:          1 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670592:          7 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670593:        117 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670595:       2152 cpu_atom/cycles/u:      7f2fb1b6e040 _start+0x0 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.670604:      38977 cpu_atom/cycles/u:  ffffffff99201280 [unknown] ([unknown])
            perf  228218 53158.670650:     167064 cpu_atom/cycles/u:      7f2fb1b67d7c intel_check_word.constprop.0+0x16c (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.671472:     232830 cpu_atom/cycles/u:      7f2fb1b75d98 strcmp+0x78 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.672710:     191183 cpu_atom/cycles/u:      7f2fb1b59311 _dl_map_object_from_fd+0xea1 (/usr/lib64/ld-linux-x86-64.so.2)
            perf  228218 53158.673461:     158125 cpu_atom/cycles/u:      7f2fb1b77148 strcmp+0x1428 (/usr/lib64/ld-linux-x86-64.so.2)
acme@number:~$

Ok, no filtering, bot samples, lets try to use filtering as with root:

Quoted 13 lines
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$ perf record -o- -e cycles:u --filter 'period < 10000000' perf test -w noploop | perf script -i-
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.019 MB - ]
acme@number:~$

Hmm.. strange. The above command works well for me.

Quoted 11 lines
acme@number:~$ perf record -v -e cycles:u --filter 'period < 10000000' perf test -w noploop 
Using CPUID GenuineIntel-6-B7-1
DEBUGINFOD_URLS=
nr_cblocks: 0
affinity: SYS
mmap flush: 1
comp level: 0
Problems creating module maps, continuing anyway...
pid hash: 228434 -> 13
pid hash: 228434 -> 14
Quoted 2 lines

This part is a little strange as it's using two entries. Hmm, are you using a hybrid machine? Anyway I think it should work there too..

Yes, I'll try it again on a 5950x since it isn't hybrid.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Also the number is too high.. I expect 1 or 2. Maybe it didn't release all the entries. Let me think about the case.

I'm inclined for now to keep this series merged and then take fixes on top, please advise if this isn't ok with you.

No objections, I'll investigate why it failed on your machine..

I've sent out a fix for multiple events. Can you please check out if it fixes your problem? It's on top of this series.

https://lore.kernel.org/r/[email protected]

Thanks,
Namhyung

author maintainer
#20 lore raw
In reply to Namhyung Kim

Ping!

Quoted 81 lines · Namhyung Kim

Hello,

This is to support the unprivileged BPF filter for profiling per-task events. Until now only root (or any user with CAP_BPF) can use the filter and we cannot add a new unprivileged BPF program types. After talking with the BPF folks at LSF/MM/BPF 2024, I was told that this is the way to go. Finally I managed to make it working with pinned BPF objects. :)

v3 changes)
* rebased onto latest perf-tools-next

v2 changes)
* rebased onto Ian's UID/GID (non-sample data based) filter term change
* support separate lost counts for each use case
* update the test case to allow normal users (if supported)

This only supports the per-task mode for normal users and root still uses its own instance of the same BPF program - not shared with other users. But it requires the one-time setup (by root) before using it by normal users like below.

  $ sudo perf record --setup-filter pin

This will load the BPF program and maps and pin them in the BPF-fs. Then normal users can use the filter.

  $ perf record -o- -e cycles:u --filter 'period < 10000' perf test -w noploop | perf script -i-
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.011 MB - ]
        perf  759982 448227.214189:          1 cycles:u:      7f153719f4d0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
        perf  759982 448227.214195:          1 cycles:u:      7f153719f4d0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
        perf  759982 448227.214196:          7 cycles:u:      7f153719f4d0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
        perf  759982 448227.214196:        223 cycles:u:      7f153719f4d0 _start+0x0 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
        perf  759982 448227.214198:       9475 cycles:u:  ffffffff8ee012a0 [unknown] ([unknown])
        perf  759982 448227.548608:          1 cycles:u:      559a9f03c81c noploop+0x5c (/home/namhyung/linux/tools/perf/perf)
        perf  759982 448227.548611:          1 cycles:u:      559a9f03c81c noploop+0x5c (/home/namhyung/linux/tools/perf/perf)
        perf  759982 448227.548612:         12 cycles:u:      559a9f03c81c noploop+0x5c (/home/namhyung/linux/tools/perf/perf)
        perf  759982 448227.548613:        466 cycles:u:      559a9f03c81c noploop+0x5c (/home/namhyung/linux/tools/perf/perf)

It's also possible to unload (and unpin, of course) using this command:

  $ sudo perf record --setup-filter unpin

The code is avaiable in 'perf/pinned-filter-v3' branch at

git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git

Thanks,
Namhyung

Namhyung Kim (8):
perf bpf-filter: Make filters map a single entry hashmap perf bpf-filter: Pass 'target' to perf_bpf_filter__prepare() perf bpf-filter: Split per-task filter use case perf bpf-filter: Support pin/unpin BPF object
perf bpf-filter: Support separate lost counts for each filter perf record: Fix a potential error handling issue perf record: Add --setup-filter option
perf test: Update sample filtering test

 tools/perf/Documentation/perf-record.txt     |   5 +
 tools/perf/builtin-record.c                  |  23 +-
 tools/perf/builtin-stat.c                    |   2 +-
 tools/perf/builtin-top.c                     |   2 +-
 tools/perf/builtin-trace.c                   |   2 +-
 tools/perf/tests/shell/record_bpf_filter.sh  |  13 +-
 tools/perf/util/bpf-filter.c                 | 406 +++++++++++++++++--
 tools/perf/util/bpf-filter.h                 |  19 +-
 tools/perf/util/bpf_skel/sample-filter.h     |   2 +
 tools/perf/util/bpf_skel/sample_filter.bpf.c |  75 +++-
 tools/perf/util/evlist.c                     |   5 +-
 tools/perf/util/evlist.h                     |   4 +-
 12 files changed, 483 insertions(+), 75 deletions(-)

--
2.45.2.803.g4e1b14247a-goog