Skip to contentSkip to conversation

perf tools: API probing fixes

PATCH Applied Adrian Hunter 6 participants 15 messages Aug 12, 2014 – Aug 14, 2014 perf events

lore
Share & tools
lore

Adrian Hunter

author reviewer
#0 lore raw

cover perf tools: API probing fixes

Hi

Here are some fixes for API probing issues that I have run into, and a second attempt at the jump-label problem.

Patches apply to your tmp.perf/core branch.

Adrian Hunter (4):
perf tools: Fix CLOEXEC probe for perf_event_paranoid == 2 perf tools: Fix one of the probe events to exclude kernel perf tools: Fix probing the kernel API with cpu-wide events perf tools: Prefer to use a cpu-wide event for probing CLOEXEC

Regards
Adrian

author reviewer
#1 lore raw

PATCH 1/4 perf tools: Fix CLOEXEC probe for perf_event_paranoid == 2

With /proc/sys/kernel/perf_event_paranoid set to 2, the probe of PERF_FLAG_FD_CLOEXEC would fail. Fix by excluding kernel profiling from the probe event.

---
 tools/perf/util/cloexec.c | 1 +
 1 file changed, 1 insertion(+)
tools/perf/util/cloexec.c +1 -0
index 5073c01..000047c 100644
@@ -11,6 +11,7 @@ static int perf_flag_probe(void)
 	struct perf_event_attr attr = {
 		.type = PERF_TYPE_SOFTWARE,
 		.config = PERF_COUNT_SW_CPU_CLOCK,
+		.exclude_kernel = 1,
 	};
 	int fd;
 	int err;
Signature · 1 line
1.8.3.2
maintainer Acked-by on v1 1/4
#2 lore raw

Re: [PATCH 1/4] perf tools: Fix CLOEXEC probe for perf_event_paranoid == 2

Hi Adrian,

Quoted 5 lines · Adrian Hunter

With /proc/sys/kernel/perf_event_paranoid set to 2, the probe of PERF_FLAG_FD_CLOEXEC would fail. Fix by excluding kernel profiling from the probe event.

Thanks,
Namhyung

Quoted 16 lines
---
 tools/perf/util/cloexec.c | 1 +
 1 file changed, 1 insertion(+)
tools/perf/util/cloexec.c +1 -0
index 5073c01..000047c 100644
@@ -11,6 +11,7 @@ static int perf_flag_probe(void)
 	struct perf_event_attr attr = {
 		.type = PERF_TYPE_SOFTWARE,
 		.config = PERF_COUNT_SW_CPU_CLOCK,
+		.exclude_kernel = 1,
 	};
 	int fd;
 	int err;
#3 lore raw

[tip:perf/core] perf tools: Fix CLOEXEC probe for perf_event_paranoid == 2

In reply to Adrian Hunter
perf tools: Fix CLOEXEC probe for perf_event_paranoid == 2
Commit-ID:  a5b0153c880c2775bf5bdd78306f0a47e860ea04
Gitweb:     http://git.kernel.org/tip/a5b0153c880c2775bf5bdd78306f0a47e860ea04
Author:     Adrian Hunter <[email protected]>
AuthorDate: Tue, 12 Aug 2014 18:04:28 +0300
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 13 Aug 2014 15:58:08 -0300

perf tools: Fix CLOEXEC probe for perf_event_paranoid == 2

With /proc/sys/kernel/perf_event_paranoid set to 2, the probe of PERF_FLAG_FD_CLOEXEC would fail. Fix by excluding kernel profiling from the probe event.

---
 tools/perf/util/cloexec.c | 1 +
 1 file changed, 1 insertion(+)
tools/perf/util/cloexec.c +1 -0
index 5073c01..000047c 100644
@@ -11,6 +11,7 @@ static int perf_flag_probe(void)
 	struct perf_event_attr attr = {
 		.type = PERF_TYPE_SOFTWARE,
 		.config = PERF_COUNT_SW_CPU_CLOCK,
+		.exclude_kernel = 1,
 	};
 	int fd;
 	int err;
author reviewer
#4 lore raw

PATCH 2/4 perf tools: Fix one of the probe events to exclude kernel

In reply to Adrian Hunter

When probing the kernel API the kernel should be excluded otherwise the probe will fail for users with insufficient privilege to profile the kernel.

---
 tools/perf/util/record.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/record.c +1 -1
index 787b882..68ad2d0 100644
@@ -47,7 +47,7 @@ out_delete:
 
 static bool perf_probe_api(setup_probe_fn_t fn)
 {
-	const char *try[] = {"cycles:u", "instructions:u", "cpu-clock", NULL};
+	const char *try[] = {"cycles:u", "instructions:u", "cpu-clock:u", NULL};
 	struct cpu_map *cpus;
 	int cpu, ret, i = 0;
 
Signature · 1 line
1.8.3.2
maintainer Acked-by on v1 2/4
#5 lore raw

Re: [PATCH 2/4] perf tools: Fix one of the probe events to exclude kernel

Quoted 5 lines · Adrian Hunter

When probing the kernel API the kernel should be excluded otherwise the probe will fail for users with insufficient privilege to profile the kernel.

Thanks,
Namhyung

Quoted 16 lines
---
 tools/perf/util/record.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/record.c +1 -1
index 787b882..68ad2d0 100644
@@ -47,7 +47,7 @@ out_delete:
 
 static bool perf_probe_api(setup_probe_fn_t fn)
 {
-	const char *try[] = {"cycles:u", "instructions:u", "cpu-clock", NULL};
+	const char *try[] = {"cycles:u", "instructions:u", "cpu-clock:u", NULL};
 	struct cpu_map *cpus;
 	int cpu, ret, i = 0;
#6 lore raw

[tip:perf/core] perf tools: Fix one of the probe events to exclude kernel

In reply to Adrian Hunter
perf tools: Fix one of the probe events to exclude kernel
Commit-ID:  c6fa35659c5fae5f9aeb6874b177baeb2adbc02e
Gitweb:     http://git.kernel.org/tip/c6fa35659c5fae5f9aeb6874b177baeb2adbc02e
Author:     Adrian Hunter <[email protected]>
AuthorDate: Tue, 12 Aug 2014 18:04:29 +0300
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 13 Aug 2014 16:20:33 -0300

perf tools: Fix one of the probe events to exclude kernel

When probing the kernel API the kernel should be excluded otherwise the probe will fail for users with insufficient privilege to profile the kernel.

---
 tools/perf/util/record.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/record.c +1 -1
index fe8079e..58267a8 100644
@@ -47,7 +47,7 @@ out_delete:
 
 static bool perf_probe_api(setup_probe_fn_t fn)
 {
-	const char *try[] = {"cycles:u", "instructions:u", "cpu-clock", NULL};
+	const char *try[] = {"cycles:u", "instructions:u", "cpu-clock:u", NULL};
 	struct cpu_map *cpus;
 	int cpu, ret, i = 0;
 
author reviewer
#7 lore raw

PATCH 3/4 perf tools: Fix probing the kernel API with cpu-wide events

In reply to Adrian Hunter

Fall back to probing with the current pid if cpu-wide probing fails. This primarily affects the setting of comm_exec flag when the user is un-privileged and /proc/sys/kernel/perf_event_paranoid > 0. The change to comm_exec can be observed by using -vv with perf record and a kernel that supports comm_exec.

---
 tools/perf/util/record.c | 36 +++++++++++++++++++++++++++---------
 1 file changed, 27 insertions(+), 9 deletions(-)
tools/perf/util/record.c +27 -9
index 68ad2d0..cf69325 100644
@@ -14,6 +14,7 @@ static int perf_do_probe_api(setup_probe_fn_t fn, int cpu, const char *str)
 	struct perf_evsel *evsel;
 	unsigned long flags = perf_event_open_cloexec_flag();
 	int err = -EAGAIN, fd;
+	static pid_t pid = -1;
 
 	evlist = perf_evlist__new();
 	if (!evlist)
@@ -24,14 +25,22 @@ static int perf_do_probe_api(setup_probe_fn_t fn, int cpu, const char *str)
 
 	evsel = perf_evlist__first(evlist);
 
-	fd = sys_perf_event_open(&evsel->attr, -1, cpu, -1, flags);
-	if (fd < 0)
-		goto out_delete;
+	while (1) {
+		fd = sys_perf_event_open(&evsel->attr, pid, cpu, -1, flags);
+		if (fd < 0) {
+			if (pid == -1 && errno == EACCES) {
+				pid = 0;
+				continue;
+			}
+			goto out_delete;
+		}
+		break;
+	}
 	close(fd);
 
 	fn(evsel);
 
-	fd = sys_perf_event_open(&evsel->attr, -1, cpu, -1, flags);
+	fd = sys_perf_event_open(&evsel->attr, pid, cpu, -1, flags);
 	if (fd < 0) {
 		if (errno == EINVAL)
 			err = -EINVAL;
@@ -201,6 +210,7 @@ bool perf_evlist__can_select_event(struct perf_evlist *evlist, const char *str)
 	struct perf_evsel *evsel;
 	int err, fd, cpu;
 	bool ret = false;
+	pid_t pid = -1;
 
 	temp_evlist = perf_evlist__new();
 	if (!temp_evlist)
@@ -221,12 +231,20 @@ bool perf_evlist__can_select_event(struct perf_evlist *evlist, const char *str)
 		cpu = evlist->cpus->map[0];
 	}
 
-	fd = sys_perf_event_open(&evsel->attr, -1, cpu, -1,
-				 perf_event_open_cloexec_flag());
-	if (fd >= 0) {
-		close(fd);
-		ret = true;
+	while (1) {
+		fd = sys_perf_event_open(&evsel->attr, pid, cpu, -1,
+					 perf_event_open_cloexec_flag());
+		if (fd < 0) {
+			if (pid == -1 && errno == EACCES) {
+				pid = 0;
+				continue;
+			}
+			goto out_delete;
+		}
+		break;
 	}
+	close(fd);
+	ret = true;
 
 out_delete:
 	perf_evlist__delete(temp_evlist);
Signature · 1 line
1.8.3.2
#8 lore raw

Re: [PATCH 3/4] perf tools: Fix probing the kernel API with cpu-wide events

Quoted 6 lines · Adrian Hunter

Fall back to probing with the current pid if cpu-wide probing fails. This primarily affects the setting of comm_exec flag when the user is un-privileged and /proc/sys/kernel/perf_event_paranoid > 0. The change to comm_exec can be observed by using -vv with perf record and a kernel that supports comm_exec.

Maybe I'm missing something - why not just try process-wide probing in the first place?

Thanks,
Namhyung

author reviewer
#9 lore raw

Re: [PATCH 3/4] perf tools: Fix probing the kernel API with cpu-wide events

Quoted 10 lines · Namhyung Kim
Quoted 6 lines · Adrian Hunter

Fall back to probing with the current pid if cpu-wide probing fails. This primarily affects the setting of comm_exec flag when the user is un-privileged and /proc/sys/kernel/perf_event_paranoid > 0. The change to comm_exec can be observed by using -vv with perf record and a kernel that supports comm_exec.

Maybe I'm missing something - why not just try process-wide probing in the first place?

Same reason as patch 4 i.e. to avoid a jump label change when doing system-wide tracing.

#10 lore raw

[tip:perf/core] perf tools: Fix probing the kernel API with cpu-wide events

In reply to Adrian Hunter
perf tools: Fix probing the kernel API with cpu-wide events
Commit-ID:  46ec69add5df60310147ce3ce01a662053d82a1e
Gitweb:     http://git.kernel.org/tip/46ec69add5df60310147ce3ce01a662053d82a1e
Author:     Adrian Hunter <[email protected]>
AuthorDate: Tue, 12 Aug 2014 18:04:30 +0300
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 13 Aug 2014 16:22:06 -0300

perf tools: Fix probing the kernel API with cpu-wide events

Fall back to probing with the current pid if cpu-wide probing fails. This primarily affects the setting of comm_exec flag when the user is un-privileged and /proc/sys/kernel/perf_event_paranoid > 0.

The change to comm_exec can be observed by using -vv with perf record and a kernel that supports comm_exec.

---
 tools/perf/util/record.c | 36 +++++++++++++++++++++++++++---------
 1 file changed, 27 insertions(+), 9 deletions(-)
tools/perf/util/record.c +27 -9
index 58267a8..e778afd 100644
@@ -14,6 +14,7 @@ static int perf_do_probe_api(setup_probe_fn_t fn, int cpu, const char *str)
 	struct perf_evsel *evsel;
 	unsigned long flags = perf_event_open_cloexec_flag();
 	int err = -EAGAIN, fd;
+	static pid_t pid = -1;
 
 	evlist = perf_evlist__new();
 	if (!evlist)
@@ -24,14 +25,22 @@ static int perf_do_probe_api(setup_probe_fn_t fn, int cpu, const char *str)
 
 	evsel = perf_evlist__first(evlist);
 
-	fd = sys_perf_event_open(&evsel->attr, -1, cpu, -1, flags);
-	if (fd < 0)
-		goto out_delete;
+	while (1) {
+		fd = sys_perf_event_open(&evsel->attr, pid, cpu, -1, flags);
+		if (fd < 0) {
+			if (pid == -1 && errno == EACCES) {
+				pid = 0;
+				continue;
+			}
+			goto out_delete;
+		}
+		break;
+	}
 	close(fd);
 
 	fn(evsel);
 
-	fd = sys_perf_event_open(&evsel->attr, -1, cpu, -1, flags);
+	fd = sys_perf_event_open(&evsel->attr, pid, cpu, -1, flags);
 	if (fd < 0) {
 		if (errno == EINVAL)
 			err = -EINVAL;
@@ -201,6 +210,7 @@ bool perf_evlist__can_select_event(struct perf_evlist *evlist, const char *str)
 	struct perf_evsel *evsel;
 	int err, fd, cpu;
 	bool ret = false;
+	pid_t pid = -1;
 
 	temp_evlist = perf_evlist__new();
 	if (!temp_evlist)
@@ -221,12 +231,20 @@ bool perf_evlist__can_select_event(struct perf_evlist *evlist, const char *str)
 		cpu = evlist->cpus->map[0];
 	}
 
-	fd = sys_perf_event_open(&evsel->attr, -1, cpu, -1,
-				 perf_event_open_cloexec_flag());
-	if (fd >= 0) {
-		close(fd);
-		ret = true;
+	while (1) {
+		fd = sys_perf_event_open(&evsel->attr, pid, cpu, -1,
+					 perf_event_open_cloexec_flag());
+		if (fd < 0) {
+			if (pid == -1 && errno == EACCES) {
+				pid = 0;
+				continue;
+			}
+			goto out_delete;
+		}
+		break;
 	}
+	close(fd);
+	ret = true;
 
 out_delete:
 	perf_evlist__delete(temp_evlist);
author reviewer
#11 lore raw

PATCH 4/4 perf tools: Prefer to use a cpu-wide event for probing CLOEXEC

In reply to Adrian Hunter

When doing a system-wide trace with Intel PT, the jump label set up as a result of probing CLOEXEC gets reset while the trace is running. That causes an Intel PT decoding error because the object code (obtained from /proc/kcore) does not match the running code at that point. While we can't expect there never to be jump label changes, we can avoid cases that the perf tool itself creates.

The problem is avoided by first trying a cpu-wide event (pid = -1) for probing the PERF_FLAG_FD_CLOEXEC flag and falling back to an event for the current process (pid = 0).

---
 tools/perf/util/cloexec.c | 22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)
tools/perf/util/cloexec.c +18 -4
index 000047c..4945aa5 100644
@@ -1,3 +1,4 @@
+#include <sched.h>
 #include "util.h"
 #include "../perf.h"
 #include "cloexec.h"
@@ -15,10 +16,23 @@ static int perf_flag_probe(void)
 	};
 	int fd;
 	int err;
+	int cpu;
+	pid_t pid = -1;
 
-	/* check cloexec flag */
-	fd = sys_perf_event_open(&attr, 0, -1, -1,
-				 PERF_FLAG_FD_CLOEXEC);
+	cpu = sched_getcpu();
+	if (cpu < 0)
+		cpu = 0;
+
+	while (1) {
+		/* check cloexec flag */
+		fd = sys_perf_event_open(&attr, pid, cpu, -1,
+					 PERF_FLAG_FD_CLOEXEC);
+		if (fd < 0 && pid == -1 && errno == EACCES) {
+			pid = 0;
+			continue;
+		}
+		break;
+	}
 	err = errno;
 
 	if (fd >= 0) {
@@ -31,7 +45,7 @@ static int perf_flag_probe(void)
 		  err, strerror(err));
 
 	/* not supported, confirm error related to PERF_FLAG_FD_CLOEXEC */
-	fd = sys_perf_event_open(&attr, 0, -1, -1, 0);
+	fd = sys_perf_event_open(&attr, pid, cpu, -1, 0);
 	err = errno;
 
 	if (WARN_ONCE(fd < 0 && err != EBUSY,
Signature · 1 line
1.8.3.2
#12 lore raw

[tip:perf/core] perf tools: Prefer to use a cpu-wide event for probing CLOEXEC

perf tools: Prefer to use a cpu-wide event for probing CLOEXEC
Commit-ID:  f6edb53c4993ffe92ce521fb449d1c146cea6ec2
Gitweb:     http://git.kernel.org/tip/f6edb53c4993ffe92ce521fb449d1c146cea6ec2
Author:     Adrian Hunter <[email protected]>
AuthorDate: Tue, 12 Aug 2014 18:04:31 +0300
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 13 Aug 2014 16:24:13 -0300

perf tools: Prefer to use a cpu-wide event for probing CLOEXEC

When doing a system-wide trace with Intel PT, the jump label set up as a result of probing CLOEXEC gets reset while the trace is running. That causes an Intel PT decoding error because the object code (obtained from /proc/kcore) does not match the running code at that point. While we can't expect there never to be jump label changes, we can avoid cases that the perf tool itself creates.

The problem is avoided by first trying a cpu-wide event (pid = -1) for probing the PERF_FLAG_FD_CLOEXEC flag and falling back to an event for the current process (pid = 0).

---
 tools/perf/util/cloexec.c | 22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)
tools/perf/util/cloexec.c +18 -4
index 000047c..4945aa5 100644
@@ -1,3 +1,4 @@
+#include <sched.h>
 #include "util.h"
 #include "../perf.h"
 #include "cloexec.h"
@@ -15,10 +16,23 @@ static int perf_flag_probe(void)
 	};
 	int fd;
 	int err;
+	int cpu;
+	pid_t pid = -1;
 
-	/* check cloexec flag */
-	fd = sys_perf_event_open(&attr, 0, -1, -1,
-				 PERF_FLAG_FD_CLOEXEC);
+	cpu = sched_getcpu();
+	if (cpu < 0)
+		cpu = 0;
+
+	while (1) {
+		/* check cloexec flag */
+		fd = sys_perf_event_open(&attr, pid, cpu, -1,
+					 PERF_FLAG_FD_CLOEXEC);
+		if (fd < 0 && pid == -1 && errno == EACCES) {
+			pid = 0;
+			continue;
+		}
+		break;
+	}
 	err = errno;
 
 	if (fd >= 0) {
@@ -31,7 +45,7 @@ static int perf_flag_probe(void)
 		  err, strerror(err));
 
 	/* not supported, confirm error related to PERF_FLAG_FD_CLOEXEC */
-	fd = sys_perf_event_open(&attr, 0, -1, -1, 0);
+	fd = sys_perf_event_open(&attr, pid, cpu, -1, 0);
 	err = errno;
 
 	if (WARN_ONCE(fd < 0 && err != EBUSY,
Acked-by on the v1 series
#13 lore raw
In reply to Adrian Hunter
Quoted 5 lines · Adrian Hunter

Hi

Here are some fixes for API probing issues that I have run into, and a second attempt at the jump-label problem.

Quoted 8 lines

Patches apply to your tmp.perf/core branch.

Adrian Hunter (4):
perf tools: Fix CLOEXEC probe for perf_event_paranoid == 2 perf tools: Fix one of the probe events to exclude kernel perf tools: Fix probing the kernel API with cpu-wide events perf tools: Prefer to use a cpu-wide event for probing CLOEXEC

for the patchset:

thanks,
jirka

maintainer Applied
#14 lore raw

Em Wed, Aug 13, 2014 at 01:28:47PM +0200, Jiri Olsa escreveu:

Quoted 15 lines · Adrian Hunter
Quoted 5 lines

Hi

Here are some fixes for API probing issues that I have run into, and a second attempt at the jump-label problem.

Quoted 8 lines

Patches apply to your tmp.perf/core branch.

Adrian Hunter (4):
perf tools: Fix CLOEXEC probe for perf_event_paranoid == 2 perf tools: Fix one of the probe events to exclude kernel perf tools: Fix probing the kernel API with cpu-wide events perf tools: Prefer to use a cpu-wide event for probing CLOEXEC

Quoted 3 lines

for the patchset:

Thanks, applied.

- Arnaldo